[jira] [Resolved] (CARBONDATA-4126) Concurrent Compaction fails with Load on table with SI

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Resolved] (CARBONDATA-4126) Concurrent Compaction fails with Load on table with SI

Karan (Jira)

     [ https://issues.apache.org/jira/browse/CARBONDATA-4126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kunal Kapoor resolved CARBONDATA-4126.
--------------------------------------
    Fix Version/s: 2.1.1
       Resolution: Fixed

> Concurrent Compaction fails with Load on table with SI
> ------------------------------------------------------
>
>                 Key: CARBONDATA-4126
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-4126
>             Project: CarbonData
>          Issue Type: Bug
>          Components: data-load
>    Affects Versions: 2.1.0
>         Environment: Spark 2.4.5
>            Reporter: Chetan Bhat
>            Priority: Major
>             Fix For: 2.1.1
>
>          Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> [Steps] :-
> Create table, load data and create SI.
> create table brinjal (imei string,AMSize string,channelsId string,ActiveCountry string, Activecity string,gamePointId double,deviceInformationId double,productionDate Timestamp,deliveryDate timestamp,deliverycharge double) stored as carbondata TBLPROPERTIES('table_blocksize'='1');
> LOAD DATA INPATH 'hdfs://hacluster/chetan/vardhandaterestruct.csv' INTO TABLE brinjal OPTIONS('DELIMITER'=',', 'QUOTECHAR'= '"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'= 'imei,deviceInformationId,AMSize,channelsId,ActiveCountry,Activecity,gamePointId,productionDate,deliveryDate,deliverycharge');
> create index indextable1 ON TABLE brinjal (AMSize) AS 'carbondata';
>  
> From one terminal load data to table and other terminal perform minor and major compaction on the table concurrently for some time.
> LOAD DATA INPATH 'hdfs://hacluster/chetan/vardhandaterestruct.csv' INTO TABLE brinjal OPTIONS('DELIMITER'=',', 'QUOTECHAR'= '"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'= 'imei,deviceInformationId,AMSize,channelsId,ActiveCountry,Activecity,gamePointId,productionDate,deliveryDate,deliverycharge');
> alter table brinjal compact 'minor';
> alter table brinjal compact 'major';
>  
> [Expected Result] :-  Concurrent Compaction should be success with Load on table with SI
>  
> [Actual Issue] : - Concurrent Compaction fails with Load on table with SI
> *0: jdbc:hive2://linux-32:22550/> alter table brinjal compact 'major';*
> *Error: org.apache.spark.sql.AnalysisException: Compaction failed. Please check logs for more info. Exception in compaction Failed to acquire lock on segment 2, during compaction of table test.brinjal; (state=,code=0)*



--
This message was sent by Atlassian Jira
(v8.3.4#803005)