CDH 5.16.2 Release Notes

Changes Not In Spark Netlib 1.1

Bigtop

Bug

• [BIGTOP-1050] - Permissions on YARN LCE should be 4754
• [BIGTOP-1194] - redhat-lsb-core is sufficient in spec files
• [BIGTOP-1589] - Prioritization is broken in bigtop-detect-javahome
• [BIGTOP-1356] - Generate hive.install on the fly
• [BIGTOP-1438] - asciidoc is no longer needed by Sqoop
• [BIGTOP-1398] - hadoop-hdfs-fuse can't be installed on Ubuntu 14.04
• [BIGTOP-1371] - Spark-master service fails to start due to missing /usr/lib/spark/work
• [BIGTOP-1308] - Installing HBase removes chrony if it is configured
• [BIGTOP-1306] - RPM licensing metadata is inconsistent and wrong
• [BIGTOP-1266] - Zookeeper init script doesn't source /etc/default/zookeeper
• [BIGTOP-1215] - Refactor Tomcat deployment
• [BIGTOP-1193] - Update Spark packaging for various conventions
• [BIGTOP-1189] - spark-python dependency needs to be updated to spark-core
• [BIGTOP-1187] - Spark service packages have extraneous core in the name
• [BIGTOP-1182] - Location of HCat binary default file not overridable
• [BIGTOP-1170] - Add an easier tool for uploading / creating the sharelib
• [BIGTOP-1027] - 'bigtop-detect-classpath' helper script missing from bigtop-utils
• [BIGTOP-1150] - update Spark debian packaging to the latest format
• [BIGTOP-1125] - Return value does not reflect status checks
• [BIGTOP-1114] - Hue metastore app is missing from core registration and Debian packages
• [BIGTOP-1112] - sqoop-client should require bigtop-utils
• [BIGTOP-1044] - hive and hive-jdbc versions should be kept in sync
• [BIGTOP-1045] - Be consistent with shells, bigtop-detect-javahome, and bigtop-utils versions
• [BIGTOP-1037] - Provide a mechanism to control the sourcing of defaults files
• [BIGTOP-1005] - Create versionless symlinks in hadoop client directory
• [BIGTOP-1017] - Revert a commit that introduces some unneeded dependencies
• [BIGTOP-988] - HCatalog does not launch daemon process in a known directory
• [BIGTOP-975] - HBase regionserver init script does not implement "condrestart" command
• [BIGTOP-984] - bigtop-utils doesn't detect JAVA_HOME for Oracle JDK 7 when package by Debian's tool
• [BIGTOP-983] - Oozie-client package ships its docs twice
• [BIGTOP-982] - hcatalog needs to be given HCAT_HOME and HIVE_HOME

Improvement

• [BIGTOP-1085] - spark packages needs to be split
• [BIGTOP-1139] - bigtop-tomcat and bigtop-jsvc don't depend on bigtop-utils
• [BIGTOP-1138] - hadoop-conf-pseudo package uses the deprecated fs.default.name parameter
• [BIGTOP-1137] - provide a way to request a major version of JDK

New Feature

• [BIGTOP-811] - Add /var/lib/bigtop as a location to install SQL connectors and other plug-ins
• [BIGTOP-39] - Include Apache Avro in Bigtop

Task

• [BIGTOP-715] - Add Spark packaging

MapReduce

Bug

• [MAPREDUCE-4649] - mr-jobhistory-daemon.sh needs to be updated post YARN-1

Common

Improvement

• [HADOOP-14141] - Store KMS SSL keystore password in catalina.properties

Oozie

Improvement

• [OOZIE-1560] - Log messages should have a way of identifying which server they came from when using HA

CDH

Bug

• [DISTRO-533] - Solr server watchdog makes heartbeat request using server short hostname, not FQDN

Sentry

Bug

• [SENTRY-24] - NOTICE.txt missing from generated tarball

Impala

Bug

• [IMPALA-3529] - Package the minidump collection script along with Impala.

Improvement

• [IMPALA-2889] - Generate one combined binary for impalad, statestored, catalogd
• [IMPALA-2632] - Link LLVM bytecode into impalad binary

Task

• [IMPALA-3778] - ASF Ubuntu 14.04 packaging build fails