[GitHub] [carbondata] QiangCai opened a new pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan

classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] QiangCai opened a new pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan

GitBox

QiangCai opened a new pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097


    ### Why is this PR needed?
    1.  in spark version 3, org.apache.spark.sql.sources.Filter is sealed
    2.  CarbonLateDecodeStrategy class's name is incorrect
    3.  CarbonDataSourceScan should be the same for 2.3 and 2.4
     
    ### What changes were proposed in this PR?
   1. translate spark Expression to carbon Expression directly, skip spark Filter step.  Remove all spark Filter in carbon code.
   2. separate CarbonLateDecodeStrategy to CarbonSourceStrategy and DMLStrategy, and simplify the code of CarbonSourceStrategy
   3.  move CarbonDataSourceScan back to the source folder
   
    ### Does this PR introduce any user interface change?
    - No
   
    ### Is any new testcase added?
    - No
   
       
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan

GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-783401216


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5503/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-783406803


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3738/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-784035438


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5504/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-784040262


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3739/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-784270040


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5506/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-784270298


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3741/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-784788475


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3744/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-784798556


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5509/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan for spark3

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-784962948


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5511/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan for spark3

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-784963097


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3746/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan for spark3

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-785231539


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5514/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan for spark3

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-785239855


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3749/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan for spark3

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-785541812


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5515/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan for spark3

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-785544498


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3750/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan for spark3

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-786339618






----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] QiangCai commented on pull request #4097: [WIP] Refactor CarbonLateDecodeStrategy and CarbonDataSourceScan for spark3

GitBox
In reply to this post by GitBox

QiangCai commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-786371903


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [CARBONDATA-4137] Refactor CarbonDataSourceScan without the soruces.Filter of Spark 3

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-786408503


   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12444/job/ApacheCarbon_PR_Builder_2.4.5/3754/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [CARBONDATA-4137] Refactor CarbonDataSourceScan without the soruces.Filter of Spark 3

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-786409127


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5519/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [carbondata] CarbonDataQA2 commented on pull request #4097: [CARBONDATA-4137] Refactor CarbonDataSourceScan without the soruces.Filter of Spark 3

GitBox
In reply to this post by GitBox

CarbonDataQA2 commented on pull request #4097:
URL: https://github.com/apache/carbondata/pull/4097#issuecomment-786477934


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12444/job/ApacheCarbonPRBuilder2.3/5520/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


12