[jira] [Updated] (CARBONDATA-4158) Make Secondary Index as a coarse grain datamap and use secondary indexes for Presto queries

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (CARBONDATA-4158) Make Secondary Index as a coarse grain datamap and use secondary indexes for Presto queries

Akshay (Jira)

     [ https://issues.apache.org/jira/browse/CARBONDATA-4158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Venugopal Reddy K updated CARBONDATA-4158:
------------------------------------------
    Description:
*Background:*

Secondary Indexes are created as carbon tables and are managed as child tables to the main table. And these indexes are leveraged for query pruning via spark plan modification during optimizer/execution phases of query execution. In order to make use of Secondary Indexes for queries from engines other than spark like presto etc, it is not feasible to modify the engine specific query execution plans as we desire in the current approach. It makes Secondary Indexes not usable for presto query pruning. Thus need arises for an engine agnostic approach to use Secondary Indexes for presto queries.

*Description:*

Current Secondary Index pruning is tightly coupled with spark because the query plan modification is specific to the spark engine. It is hard to reuse the solution for presto queries. Need a new solution to use secondary indexes with Presto queries. And it  shouldn’t affect the existing customer using secondary index with spark.

  was:
*Background:*

Secondary Indexes are created as carbon tables and are managed as child tables to the main table. And these indexes are leveraged for query pruning via spark plan modification during optimizer/execution phases of query execution. In order to make use of Secondary Indexes for queries from engines other than spark like presto etc, it is not feasible to modify the engine specific query execution plans as we desire in the current approach. It makes Secondary Indexes not usable for presto query pruning. Thus need arises for an engine agnostic approach to use Secondary Indexes for presto queries.

*Description:*

                          Current Secondary Index pruning is tightly coupled with spark because the query plan modification is specific to the spark engine. It is hard to reuse the solution for presto queries. Need a new solution to use secondary indexes with Presto queries. And it  shouldn’t affect the existing customer using secondary index with spark.


> Make Secondary Index as a coarse grain datamap and use secondary indexes for Presto queries
> -------------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-4158
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-4158
>             Project: CarbonData
>          Issue Type: New Feature
>            Reporter: Venugopal Reddy K
>            Priority: Minor
>
> *Background:*
> Secondary Indexes are created as carbon tables and are managed as child tables to the main table. And these indexes are leveraged for query pruning via spark plan modification during optimizer/execution phases of query execution. In order to make use of Secondary Indexes for queries from engines other than spark like presto etc, it is not feasible to modify the engine specific query execution plans as we desire in the current approach. It makes Secondary Indexes not usable for presto query pruning. Thus need arises for an engine agnostic approach to use Secondary Indexes for presto queries.
> *Description:*
> Current Secondary Index pruning is tightly coupled with spark because the query plan modification is specific to the spark engine. It is hard to reuse the solution for presto queries. Need a new solution to use secondary indexes with Presto queries. And it  shouldn’t affect the existing customer using secondary index with spark.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)