CDH 4 Release Notes

Changes Not In Apache HBase 0.92.1

HBase

Bug

• [HBASE-6011] - Unable to start master in local mode
• [HBASE-6803] - script hbase should add JAVA_LIBRARY_PATH to LD_LIBRARY_PATH
• [HBASE-5920] - New Compactions Logic can silently prevent user-initiated compactions from occurring
• [HBASE-6165] - Replication can overrun .META. scans on cluster re-start
• [HBASE-6638] - Move DaemonThreadFactory into Threads (0.94)
• [HBASE-6525] - bin/replication/copy_tables_desc.rb references non-existent class
• [HBASE-3443] - ICV optimization to look in memstore first and then store files (HBASE-3082) does not work when deletes are in the mix
• [HBASE-6297] - Backport HBASE-6195 to 0.92
• [HBASE-6686] - HFile Quarantine fails with missing dirs in hadoop 2.0
• [HBASE-6662] - Region server incorrectly reports its own address as master's address
• [HBASE-5806] - Handle split region related failures on master restart and RS restart
• [HBASE-6608] - Fix for HBASE-6160, META entries from daughters can be deleted before parent entries, shouldn't compare HRegionInfo's
• [HBASE-6160] - META entries from daughters can be deleted before parent entries
• [HBASE-6565] - Coprocessor exec result Map is not thread safe
• [HBASE-6663] - NPE race in HConnection if zookeeper is reset
• [HBASE-6115] - NullPointerException is thrown when root and meta table regions are assigning to another RS.
• [HBASE-6236] - Offline meta repair fails if the HBase base mount point is on a different cluster/volume than its parent in a ViewFS or similar FS
• [HBASE-5606] - SplitLogManger async delete node hangs log splitting when ZK connection is lost
• [HBASE-5871] - Usability regression, we don't parse compression algos anymore
• [HBASE-5680] - Improve compatibility warning about HBase with Hadoop 0.23.x
• [HBASE-6576] - HBaseAdmin.createTable should wait until the table is enabled
• [HBASE-5058] - Allow HBaseAdmin to use an existing connection
• [HBASE-6604] - Bump log4j to 1.2.17
• [HBASE-6471] - Performance regression caused by HBASE-4054
• [HBASE-6211] - Put latencies in jmx
• [HBASE-5114] - TestMetaReaderEditor can timeout during test execution
• [HBASE-6552] - TestAcidGuarantees system test should flush more aggressively
• [HBASE-6325] - [replication] Race in ReplicationSourceManager.init can initiate a failover even if the node is alive
• [HBASE-6319] - ReplicationSource can call terminate on itself and deadlock
• [HBASE-5586] - [replication] NPE in ReplicationSource when creating a stream to an inexistent cluster
• [HBASE-5757] - TableInputFormat should handle as many errors as possible
• [HBASE-6440] - SplitLogManager - log the exception when failed to finish split log file
• [HBASE-6420] - Gracefully shutdown logsyncer
• [HBASE-5922] - HalfStoreFileReader seekBefore causes StackOverflowError
• [HBASE-5773] - HtablePool constructor not reading config files in certain cases
• [HBASE-6200] - KeyComparator.compareWithoutRow can be wrong when families have the same prefix
• [HBASE-4470] - ServerNotRunningException coming out of assignRootAndMeta kills the Master
• [HBASE-5724] - Row cache of KeyValue should be cleared in readFields().
• [HBASE-5603] - rolling-restart.sh script hangs when attempting to detect expiration of /hbase/master znode.
• [HBASE-4379] - [hbck] Does not complain about tables with no end region [Z,]
• [HBASE-5874] - When 'fs.default.name' not configured, the hbck tool and Merge tool throw IllegalArgumentException.
• [HBASE-6392] - UnknownRegionException blocks hbck from sideline big overlap regions
• [HBASE-6397] - [hbck] print out bulk load commands for sidelined regions if necessary
• [HBASE-6394] - verifyrep MR job map tasks throws NullPointerException
• [HBASE-6357] - Failed distributed log splitting stuck on master web UI
• [HBASE-6107] - Distributed log splitting hangs even there is no task under /hbase/splitlog
• [HBASE-6097] - TestHRegion.testBatchPut is flaky on 0.92
• [HBASE-6068] - Secure HBase cluster : Client not able to call some admin APIs
• [HBASE-5908] - TestHLogSplit.testTralingGarbageCorruptionFileSkipErrorsPasses should not use append to corrupt the HLog
• [HBASE-5488] - OfflineMetaRepair doesn't support hadoop 0.20's fs.default.name property
• [HBASE-5740] - Compaction interruption may be due to balacing
• [HBASE-5486] - Warn message in HTable: Stringify the byte[]
• [HBASE-5484] - Spelling mistake in error message in HMasterCommandLine
• [HBASE-5833] - 0.92 build has been failing pretty consistently on TestMasterFailover....
• [HBASE-6095] - ActiveMasterManager NullPointerException
• [HBASE-5849] - On first cluster startup, RS aborts if root znode is not available
• [HBASE-5213] - "hbase master stop" does not bring down backup masters
• [HBASE-3690] - Option to Exclude Bulk Import Files from Minor Compaction
• [HBASE-6018] - hbck fails with a RejectedExecutionException when >50 regions present
• [HBASE-5928] - Hbck shouldn't npe when there are no tables.
• [HBASE-6029] - HBCK doesn't recover Balance switch if exception occurs in onlineHbck()
• [HBASE-5821] - Incorrect handling of null value in Coprocessor aggregation function min()
• [HBASE-5669] - AggregationClient fails validation for open stoprow scan
• [HBASE-5966] - MapReduce based tests broken on Hadoop 2.0.0-alpha
• [HBASE-5975] - Failed suppression of fs shutdown hook with Hadoop 2.0.0
• [HBASE-5963] - ClassCastException: FileSystem$Cache$ClientFinalizer cannot be cast to Thread
• [HBASE-5952] - Sync hardcoded default flush size and max file size with hbase-default.xml
• [HBASE-5635] - If getTaskList() returns null, splitlogWorker would go down and it won't serve any requests
• [HBASE-5787] - Table owner can't disable/delete his/her own table
• [HBASE-5780] - Fix race in HBase regionserver startup vs ZK SASL authentication
• [HBASE-5781] - Zookeeper session got closed while trying to assign the region to RS using hbck -fix
• [HBASE-5665] - Repeated split causes HRegionServer failures and breaks table
• [HBASE-5638] - Backport to 0.90 and 0.92 - NPE reading ZK config in HBase
• [HBASE-5624] - Aborting regionserver when splitting region, may cause daughter region not assigned by ServerShutdownHandler.
• [HBASE-5596] - Few minor bugs from HBASE-5209
• [HBASE-5563] - HRegionInfo#compareTo should compare regionId as well

Improvement

• [HBASE-6689] - HRegion.increment comment nits
• [HBASE-5682] - Allow HConnectionImplementation to recover from ZK connection loss (for 0.94 only)
• [HBASE-6643] - Accept encoded region name in compacting/spliting region from shell
• [HBASE-6314] - Fast fail behavior for unauthenticated user
• [HBASE-4746] - Use a random ZK client port in unit tests so we can run them in parallel
• [HBASE-6586] - Quarantine Corrupted HFiles with hbck
• [HBASE-5735] - Clearer warning message when connecting a non-secure HBase client to a secure HBase server
• [HBASE-6644] - HBaseAdmin.createTable should wait more till table is enabled.
• [HBASE-5714] - Add write permissions check before any hbck run that modifies hdfs.
• [HBASE-5728] - Methods Missing in HTableInterface
• [HBASE-5887] - Make TestAcidGuarantees usable for system testing.
• [HBASE-3373] - Allow regions to be load-balanced by table
• [HBASE-6444] - Expose the ability to set custom HTTP Request Headers for the REST client used by RemoteHTable
• [HBASE-5189] - Add metrics to keep track of region-splits in RS
• [HBASE-5618] - SplitLogManager - prevent unnecessary attempts to resubmits
• [HBASE-6334] - TestImprovement for TestHRegion.testWritesWhileGetting
• [HBASE-6013] - Polish sharp edges from CopyTable
• [HBASE-5360] - [uberhbck] Add options for how to handle offline split parents.
• [HBASE-6173] - hbck check specified tables only
• [HBASE-5892] - [hbck] Refactor parallel WorkItem* to Futures.
• [HBASE-6384] - hbck should group together those sidelined regions need to be bulk loaded later
• [HBASE-6067] - HBase won't start when hbase.rootdir uses ViewFileSystem
• [HBASE-6373] - Add more context information to audit log messages
• [HBASE-6382] - Upgrade Jersey to 1.8 to match Hadoop 1 and 2
• [HBASE-6283] - [region_mover.rb] Add option to exclude list of hosts on unload instead of just assuming the source node.
• [HBASE-6124] - Backport HBASE-6033 to 0.90, 0.92 and 0.94
• [HBASE-6062] - preCheckAndPut/Delete() checks for READ when also a WRITE is performed
• [HBASE-4720] - Implement atomic update operations (checkAndPut, checkAndDelete) for REST client/server
• [HBASE-6061] - Fix ACL "Admin" Table inconsistent permission check
• [HBASE-4820] - Distributed log splitting coding enhancement to make it easier to understand, no semantics change
• [HBASE-5310] - HConnectionManager server cache key enhancement
• [HBASE-5342] - Grant/Revoke global permissions
• [HBASE-5712] - Parallelize load of .regioninfo files in diagnostic/repair portion of hbck.
• [HBASE-5823] - Hbck should be able to print help
• [HBASE-5734] - Change hbck sideline root
• [HBASE-6004] - Adding more logging to help debugging MR job
• [HBASE-5973] - Add ability for potentially long-running IPC calls to abort if client disconnects
• [HBASE-5739] - Upgrade guava to 11.0.2
• [HBASE-5533] - Add more metrics to HBase
• [HBASE-5655] - Cap space usage of default log4j rolling policy
• [HBASE-5589] - Add of the offline call to the Master Interface
• [HBASE-5588] - Deprecate/remove AssignmentManager#clearRegionFromTransition
• [HBASE-4440] - add an option to presplit table to PerformanceEvaluation

New Feature

• [HBASE-4391] - Add ability to start RS as root and call mlockall
• [HBASE-6550] - Refactoring ReplicationSink to make it more responsive of cluster health
• [HBASE-5719] - Enhance hbck to sideline overlapped mega regions
• [HBASE-5599] - [hbck] handle NO_VERSION_FILE and SHOULD_NOT_BE_DEPLOYED inconsistencies
• [HBASE-5886] - Add new metric for possible data loss due to puts without WAL
• [HBASE-5128] - [uber hbck] Online automated repair of table integrity and region consistency problems

Task

• [HBASE-5758] - Forward port "HBASE-4109 Hostname returned via reverse dns lookup contains trailing period if configured interface is not 'default'"
• [HBASE-6606] - Test for reconnecting with HBaseAdmin using unmanaged HConnection
• [HBASE-6553] - Remove Avro Gateway
• [HBASE-6131] - Add attribution for code added by HBASE-5533 metrics
• [HBASE-6034] - Upgrade Hadoop dependencies
• [HBASE-5793] - TestHBaseFsck#TestNoHdfsTable test hangs after client retries increased
• [HBASE-6001] - Upgrade slf4j to 1.6.1

Test

• [HBASE-6593] - TestAdmin times out sometimes
• [HBASE-6507] - [hbck] TestHBaseFsck ran into TableNotEnabledException
• [HBASE-5792] - HLog Performance Evaluation Tool
• [HBASE-5223] - TestMetaReaderEditor is missing call to CatalogTracker.stop()
• [HBASE-5985] - TestMetaMigrationRemovingHTD failed with HADOOP 2.0.0

Umbrella

• [HBASE-6157] - Revoke of Global permission is not taking effect without restart.