Add more metadata to footer

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Add more metadata to footer

Hi Community,

With the current metadata in carbondata file footer, we can add more
metadata to footer file to Improve carbon maintainability. we can add info
like who has written the file and what is the carbon version in which this
file is written, which will help to identify or fix any performance bugs in
that version or any other functional issue and may be compatibility.

This is required because now carbon is has SDK support, file format, and
have multiple version support.So by writing these info it would be easy to
identify and fix any bugs.

Currently we can write the above info in file footer and include these info
in cli tool summary. May be later we can expose a DDL which will help us to
get these info in beeline session only instead of running the java command
separately for cli tool.

For SDK we can expose two APIs for writing and reading these info which
will allow to write these info and  read these info and get details in some
proper format string format.

Any suggestiom or inputs are welcomed.