DISCUSSION: removing support for Spark 2.1 and 2.2 in CarbonData 2.0
For following reasons, I suggest to deprecate the support of spark 2.1 and
2.2 integration in CarbonData 2.0:
1. As spark community is moving to 3.0, versions below 2.3 will be out of
maintaining lifecycle in spark community.
2. In PR #3378 (Integrating with Spark 2.4.4), the carbondata-spark2 module
becomes very complex since new folder to extract out common code for Spark
2.4.4 is needed. To avoid these complexity, it is better to drop support for
2.1 and 2.2
3. We only have 2 set of CI environment (currently is for 2.2 and 2.3
integration), after we integrate with Spark 2.4, we do not have enough
resources to run for 2.2 integration.