CDH 5.2.1 Release Notes
The following lists all Lightning-Fast Cluster Computing Jiras included in CDH 5.2.1
that are not included in the Lightning-Fast Cluster Computing base version 1.1.0. The
file lists all changes included in CDH 5.2.1. The patch for each
change can be found in the cloudera/patches directory in the release tarball.
Changes Not In Lightning-Fast Cluster Computing 1.1.0
- [SPARK-3788] - Yarn dist cache code is not friendly to HDFS HA, Federation
- [SPARK-3661] - spark.*.memory is ignored in cluster mode
- [SPARK-3979] - Yarn backend's default file replication should match HDFS's default one
- [SPARK-1719] - spark.executor.extraLibraryPath isn't applied on yarn
- [SPARK-3606] - Spark-on-Yarn AmIpFilter does not work with Yarn HA.
- [SPARK-3560] - In yarn-cluster mode, the same jars are distributed through multiple mechanisms.
- [SPARK-3039] - Spark assembly for new hadoop API (hadoop 2) contains avro-mapred for hadoop 1 API
- [SPARK-3286] - Cannot view ApplicationMaster UI when Yarns url scheme is https
- [SPARK-3260] - Yarn - pass acls along with executor launch
- [SPARK-3452] - Maven build should skip publishing artifacts people shouldn't depend on
- [SPARK-3465] - Task metrics are not aggregated correctly in local mode
- [SPARK-3429] - Don't include the empty string "" as a defaultAclUser
- [SPARK-2140] - yarn stable client doesn't properly handle MEMORY_OVERHEAD for AM
- [SPARK-2425] - Standalone Master is too aggressive in removing Applications
- [SPARK-3394] - TakeOrdered crashes when limit is 0
- [SPARK-3211] - .take() is OOM-prone when there are empty partitions
- [SPARK-3401] - Wrong usage of tee command in python/run-tests
- [SPARK-3233] - Executor never stop its SparnEnv, BlockManager, ConnectionManager etc.
- [SPARK-1912] - Compression memory issue during reduce
- [SPARK-3404] - SparkSubmitSuite fails with "spark-submit exits with code 1"
- [SPARK-3014] - Log a more informative messages in a couple failure scenarios
- [SPARK-3337] - Paranoid quoting in shell to allow install dirs with spaces within.
- [SPARK-3193] - output error info when Process exitcode not zero
- [SPARK-2419] - Misc updates to streaming programming guide
- [SPARK-2871] - Missing API in PySpark
- [SPARK-3052] - Misleading and spurious FileSystem closed errors whenever a job fails while reading from Hadoop
- [SPARK-2288] - Hide ShuffleBlockManager behind ShuffleManager
- [SPARK-3073] - improve large sort (external sort) for PySpark
- [SPARK-3256] - Enable :cp to add JARs in spark-shell (Scala 2.10)