CDH 4 Release Notes

The following lists all Apache Hadoop Jiras included in CDH 4 that are not included in the Apache Hadoop base version 2.0.0. The hadoop-2.0.0+91.CHANGES.txt file lists all changes included in CDH 4. The patch for each change can be found in the cloudera/patches directory in the release tarball.

Changes Not In Apache Hadoop 2.0.0

Common

Bug

[HADOOP-8466] - hadoop-client POM incorrectly excludes avro
[HADOOP-8452] - DN logs backtrace when running under jsvc and /jmx is loaded
[HADOOP-8400] - All commands warn "Kerberos krb5 configuration not found" when security is not enabled
[HADOOP-8422] - Deprecate FileSystem#getDefault* and getServerDefault methods that don't take a Path argument
[HADOOP-8329] - Build fails with Java 7
[HADOOP-8408] - MR doesn't work with a non-default ViewFS mount table and security enabled
[HADOOP-8287] - etc/hadoop is missing hadoop-env.sh
[HADOOP-8406] - CompressionCodecFactory.CODEC_PROVIDERS iteration is thread-unsafe
[HADOOP-8405] - ZKFC tests leak ZK instances
[HADOOP-8393] - hadoop-config.sh missing variable exports, causes Yarn jobs to fail with ClassNotFoundException MRAppMaster
[HADOOP-8316] - Audit logging should be disabled by default
[HADOOP-7868] - Hadoop native fails to compile when default linker option is -Wl,--as-needed
[HADOOP-8257] - Auto-HA: TestZKFailoverControllerStress occasionally fails with Mockito error
[HADOOP-8245] - Fix flakiness in TestZKFailoverController
[HADOOP-8220] - ZKFailoverController doesn't handle failure to become active correctly

Improvement

[HADOOP-8361] - Avoid out-of-memory problems when deserializing strings
[HADOOP-8224] - Don't hardcode hdfs.audit.logger in the scripts
[HADOOP-8353] - hadoop-daemon.sh and yarn-daemon.sh can be misleading on stop
[HADOOP-8276] - Auto-HA: add config for java options to pass to zkfc daemon
[HADOOP-8279] - Auto-HA: Allow manual failover to be invoked from zkfc.
[HADOOP-8306] - ZKFC: improve error message when ZK is not running
[HADOOP-8247] - Auto-HA: add a config to enable auto-HA, which disables manual FC
[HADOOP-8246] - Auto-HA: automatically scope znode by nameservice ID
[HADOOP-8215] - Security support for ZK Failover controller

Test

[HADOOP-8260] - Auto-HA: Replace ClientBaseWithFixes with our own modified copy of the class
[HADOOP-8228] - Auto HA: Refactor tests and add stress tests

HDFS

Bug

[HDFS-3501] - Checkpointing with security enabled will stop working after ticket lifetime expires
[HDFS-3487] - offlineimageviewer should give byte offset information when it encounters an exception
[HDFS-3486] - offlineimageviewer can't read fsimage files that contain persistent delegation tokens
[HDFS-3484] - hdfs fsck doesn't work if NN HTTP address is set to 0.0.0.0 even if NN RPC address is configured
[HDFS-3460] - HttpFS proxyuser validation with Kerberos ON uses full principal name
[HDFS-3413] - TestFailureToReadEdits timing out
[HDFS-2982] - Startup performance suffers when there are many edit log segments
[HDFS-3440] - should more effectively limit stream memory consumption when reading corrupt edit logs
[HDFS-3444] - hdfs groups command doesn't work with security enabled
[HDFS-2800] - HA: TestStandbyCheckpoints.testCheckpointCancellation is racy
[HDFS-3434] - InvalidProtocolBufferException when visiting DN browseDirectory.jsp
[HDFS-3433] - GetImageServlet should allow administrative requestors when security is enabled
[HDFS-3432] - TestDFSZKFailoverController tries to fail over too early
[HDFS-3422] - TestStandbyIsHot timeouts too aggressive
[HDFS-2759] - Pre-allocate HDFS edit log files after writing version number
[HDFS-3031] - HA: Error (failed to close file) when uploading large file + kill active NN + manual failover
[HDFS-3395] - NN doesn't start with HA+security enabled and HTTP address set to 0.0.0.0
[HDFS-3414] - Balancer does not find NameNode if rpc-address or servicerpc-address are not set in client configs
[HDFS-3026] - HA: Handle failure during HA state transition
[HDFS-3396] - FUSE build fails on Ubuntu 12.04
[HDFS-3328] - NPE in DataNode.getIpcPort
[HDFS-3261] - TestHASafeMode fails on HDFS-3042 branch
[HDFS-3037] - TestMulitipleNNDataBlockScanner#testBlockScannerAfterRestart is racy
[HDFS-2976] - Remove unnecessary method (tokenRefetchNeeded) in DFSClient

Improvement

[HDFS-3454] - Balancer unconditionally logs InterruptedException at INFO level on shutdown if security is enabled
[HDFS-3341] - Change minimum RPC versions to 2.0.0-SNAPSHOT instead of 2.0.0
[HDFS-3438] - BootstrapStandby should not require a rollEdits on active node
[HDFS-2885] - Remove "federation" from the nameservice config options
[HDFS-3134] - Harden edit log loader against malformed or malicious input
[HDFS-3335] - check for edit log corruption at the end of the log
[HDFS-3404] - Make putImage in GetImageServlet infer remote address to fetch from request
[HDFS-3400] - DNs should be able start with jsvc even if security is disabled
[HDFS-3390] - DFSAdmin should print full stack traces of errors when DEBUG logging is enabled
[HDFS-3223] - Auto HA: add zkfc to hadoop-daemon.sh script

New Feature

[HDFS-3159] - Document NN auto-failover setup and configuration
[HDFS-3200] - Auto-HA: Scope all ZKFC configs by nameservice
[HDFS-2185] - HA: HDFS portion of ZK-based FailoverController

Wish

[HDFS-860] - fuse-dfs truncate behavior causes issues with scp

MapReduce

Bug

[MAPREDUCE-4250] - hadoop-config.sh missing variable exports, causes Yarn jobs to fail with ClassNotFoundException MRAppMaster
[MAPREDUCE-4237] - TestNodeStatusUpdater can fail if localhost has a domain associated with it