[DISCUSSION] Support Avro Datatype Conversion to Carbon

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[DISCUSSION] Support Avro Datatype Conversion to Carbon

Indhumathi
Hi All,

I am working on supporting Avro Datatype Conversion to carbon format.Please
find the
scope below for the same.

**Scope:**
  1. Support Avro Complex Types: Enum, Union, Fixed.
  2. Support Avro Logical Types: TimeMillis, TimeMicros, Decimal.

Please feel free to provide your comments and suggestions. I am working on
the design document and will upload soon.
**Jira Link:**
https://issues.apache.org/jira/browse/CARBONDATA-2876
<https://issues.apache.org/jira/browse/CARBONDATA-2876>  

Regards
Indhumathi M



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Support Avro Datatype Conversion to Carbon

kanaka
Hi Indhumathi,

Thanks for initiating the discussion.

Please list the internal storage types in carbon how you are planning to
store and retrieve these types in query.



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Support Avro Datatype Conversion to Carbon

Indhumathi
In reply to this post by Indhumathi
Hello all,

Please find below plan for avro datatype conversion to carbon storage type.

1. Avro Complex DataType *Union* can be internally stored as *Struct of
Distinct Value types*.

2. Avro Complex DataType *Enum* can be internally stored as *Array of Value
type*.

3. Avro *Fixed* DataType can be internally stored as *Binary*.

4. Avro Logical DataTypes *TimeMillis* and *TimeMicros* can be internally
stored
    as *DataTypes.STRING*.

5. Avro Decimal DataType *Decimal* can be internally stored as
    *DataTypes.createDecimalType(precision, scale)*.

Please provide your inputs and comments. Any suggestion from community is
most
welcomed.

Regards,
Indhumathi



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Support Avro Datatype Conversion to Carbon

Indhumathi
In reply to this post by Indhumathi