CDH 4 Release Notes

The following lists all Apache HBase Jiras included in CDH 4 that are not included in the Apache HBase base version 0.94.2. The hbase-0.94.2-cdh4.2.0.CHANGES.txt file lists all changes included in CDH 4. The patch for each change can be found in the cloudera/patches directory in the release tarball.

Changes Not In Apache HBase 0.94.2

HBase

Brainstorming

[HBASE-7518] - Move AuthResult out of AccessController

Bug

[HBASE-6972] - HBase Shell deleteall should not require column to be defined
[HBASE-7829] - zookeeper kerberos conf keytab and principal parameters interchanged
[HBASE-7788] - [snapshot 130201 merge] Fix flakey TestRestore*SnapshotFromClient#testCloneSnapshot
[HBASE-7730] - HBaseAdmin#synchronousBalanceSwitch is not compatible with 0.92
[HBASE-7685] - Closing socket connection can't be removed from SecureClient
[HBASE-7684] - NullPointerException in SecureClient when Call is cleaned up due to RPC timeout
[HBASE-7720] - improve logging messages of failed snapshot attempts.
[HBASE-7703] - Eventually all online snapshots failing due to Timeout at same regionserver.
[HBASE-7694] - Secure HBase should use replication call queue
[HBASE-7693] - Hostname returned by TableInputFormatBase.reverseDNS contains trailing period
[HBASE-7657] - Make ModifyTableHandler synchronous
[HBASE-7666] - More logging improvements in online snapshots code.
[HBASE-7651] - RegionServerSnapshotManager fails with CancellationException if previous snapshot fails in per region task
[HBASE-7644] - Port HBASE-4802 'Disable show table metrics in bulk loader' to 0.94
[HBASE-7647] - 0.94 hfiles v2.1 are not backwards compatible with HFilev2.0
[HBASE-7643] - HFileArchiver.resolveAndArchive() race condition may lead to snapshot data loss
[HBASE-7336] - HFileBlock.readAtOffset does not work well with multiple threads
[HBASE-7648] - TestAcidGuarantees.testMixedAtomicity hangs sometimes
[HBASE-7646] - Make forkedProcessTimeoutInSeconds configurable
[HBASE-7513] - HDFSBlocksDistribution shouldn't send NPEs when something goes wrong
[HBASE-7627] - UnsupportedOperationException in CatalogJanitor thread.
[HBASE-7545] - [replication] Break out TestReplication into manageable classes
[HBASE-5458] - Thread safety issues with Compression.Algorithm.GZ and CompressionTest
[HBASE-7616] - NPE in ZKProcedureCoordinator$1.nodeCreated
[HBASE-7562] - ZKUtil: missing "else condition" in multi processing
[HBASE-7596] - Redundant FSTableDescriptor update
[HBASE-7592] - HConnectionManager.getHTableDescriptor() compares too much
[HBASE-7172] - TestSplitLogManager.testVanishingTaskZNode() fails when run individually and is flaky
[HBASE-7575] - FSUtils#getTableStoreFilePathMap should all ignore non-table folders
[HBASE-7560] - TestCompactionState failures
[HBASE-6622] - TestUpgradeFromHFileV1ToEncoding#testUpgrade fails in trunk
[HBASE-7536] - Add test that confirms that multiple concurrent snapshot requests are rejected.
[HBASE-7538] - Improve snapshot related error and exception messages
[HBASE-7548] - Fix javadoc warnings in snapshot classes
[HBASE-7501] - Introduce MetaEditor method that both adds and deletes rows in .META. table
[HBASE-7552] - Pass bufferSize param to FileLinkInputStream constructor within FileLink.open method, and remove unnecessary import packages.
[HBASE-7537] - .regioninfo not created by createHRegion()
[HBASE-7558] - Client not able to access /hbase/unassigned in secure cluster
[HBASE-7550] - Synchronization problem in AssignmentManager
[HBASE-7523] - Snapshot attempt with the name of a previously taken snapshots fails sometimes.
[HBASE-7214] - CleanerChore logs too much, so much so it obscures all else that is going on
[HBASE-7467] - CleanerChore checkAndDeleteDirectory not deleting empty directories
[HBASE-7299] - TestMultiParallel fails intermittently in trunk builds
[HBASE-7502] - TestScannerTimeout fails on snapshot branch
[HBASE-7498] - Make REST server thread pool size configurable
[HBASE-7496] - TestZKProcedure fails interrmittently.
[HBASE-7321] - Simple Flush Snapshot
[HBASE-6864] - Online snapshots scaffolding
[HBASE-7212] - Globally Barriered Procedure mechanism
[HBASE-7423] - HFileArchiver should not use the configuration from the Filesystem
[HBASE-5672] - TestLruBlockCache#testBackgroundEvictionThread fails occasionally
[HBASE-7440] - ReplicationZookeeper#addPeer is racy
[HBASE-7412] - Fix how HTableDescriptor handles default max file size and flush size
[HBASE-7422] - MasterFS doesn't set configuration for internal FileSystem
[HBASE-7421] - TestHFileCleaner->testHFileCleaning has an aggressive timeout
[HBASE-7418] - HFileLink flaky tests
[HBASE-6796] - Backport HBASE-5547, Don't delete HFiles in backup mode.
[HBASE-7077] - Test for: CheckAndPut should properly read MVCC
[HBASE-7051] - CheckAndPut should properly read MVCC
[HBASE-7103] - Need to fail split if SPLIT znode is deleted even before the split is completed.
[HBASE-6846] - BitComparator bug - ArrayIndexOutOfBoundsException
[HBASE-7017] - Backport "[replication] The replication-executor should make sure the file that it is replicating is closed before declaring success on that file" to 0.94
[HBASE-6700] - [replication] empty znodes created during queue failovers aren't deleted
[HBASE-6733] - [0.92 UNIT TESTS] TestReplication.queueFailover occasionally fails [Part-2]
[HBASE-6826] - [WINDOWS] TestFromClientSide failures
[HBASE-7343] - Fix flaky condition for TestDrainingServer
[HBASE-7338] - Fix flaky condition for org.apache.hadoop.hbase.TestRegionRebalancing.testRebalanceOnRegionServerNumberChange
[HBASE-6528] - Raise the wait time for TestSplitLogWorker#testAcquireTaskAtStartup to reduce the failure probability
[HBASE-7314] - Can't start REST/Thrift server if HBASE_JMX_OPTS not set
[HBASE-7301] - Force ipv4 for unit tests
[HBASE-7300] - HbckTestingUtil needs to keep a static executor to lower the number of threads used
[HBASE-6327] - HLog can be null when create table
[HBASE-7069] - HTable.batch does not have to be synchronized
[HBASE-7277] - Thrift default JMX port should be 10103 instead of 8093
[HBASE-6963] - unable to run hbck on a secure cluster
[HBASE-7130] - NULL qualifier is ignored
[HBASE-6958] - TestAssignmentManager sometimes fails
[HBASE-6904] - In the HBase shell, an error is thrown that states replication-related znodes already exist
[HBASE-6996] - HRegion.mutateRowsWithLocks should call checkResources/checkReadOnly
[HBASE-7018] - Fix and Improve TableDescriptor caching for bulk assignment
[HBASE-6858] - Fix the incorrect BADVERSION checking in the recoverable zookeeper
[HBASE-6950] - TestAcidGuarantees system test now flushes too aggressively
[HBASE-6604] - Bump log4j to 1.2.17
[HBASE-5740] - Compaction interruption may be due to balacing
[HBASE-6853] - IllegalArgumentException is thrown when an empty region is splitted

Improvement

[HBASE-7699] - Replace LOG.info() with LOG.debug() in FSUtils.listStatus()
[HBASE-6372] - Add scanner batching to Export job
[HBASE-6459] - improve speed of table creation
[HBASE-7540] - Make znode dump to print a dump of replication znodes
[HBASE-5778] - Fix HLog compression's incompatibilities
[HBASE-7526] - create table does not log the table name in audit log
[HBASE-5548] - Add ability to get a table in the shell
[HBASE-6386] - Audit log messages do not include column family / qualifier information consistently
[HBASE-5987] - HFileBlockIndex improvement
[HBASE-6836] - [89-fb] Parallel deletes in HBase Thrift server
[HBASE-7197] - Add multi get to RemoteHTable
[HBASE-6852] - SchemaMetrics.updateOnCacheHit costs too much while full scanning a table with all of its fields
[HBASE-7199] - hbck should check lingering reference hfile and have option to sideline them automatically
[HBASE-6585] - Audit log messages should contain info about the higher level operation being executed
[HBASE-6775] - Use ZK.multi when available for HBASE-6710 0.92/0.94 compatibility fix
[HBASE-7377] - Clean up TestHBase7051
[HBASE-4791] - Allow Secure Zookeeper JAAS configuration to be programmatically set (rather than only by reading JAAS configuration file)
[HBASE-5898] - Consider double-checked locking for block cache lock
[HBASE-5616] - Make compaction code standalone
[HBASE-5693] - When creating a region, the master initializes it and creates a memstore within the master server
[HBASE-7274] - Enable JMX metrics collection for REST Server
[HBASE-6804] - [replication] lower the amount of logging to a more human-readable level
[HBASE-7204] - Make hbck ErrorReporter pluggable
[HBASE-7190] - Add an option to hbck to check only meta and assignment
[HBASE-7178] - Compression tests
[HBASE-7156] - Add Data Block Encoding and -D opts to Performance Evaluation
[HBASE-5934] - Add the ability for Performance Evaluation to set the table compression
[HBASE-4913] - Per-CF compaction Via the Shell
[HBASE-7151] - Better log message for Per-CF compactions
[HBASE-5257] - Allow INCLUDE_AND_NEXT_COL in filters and use it in ColumnPaginationFilter
[HBASE-7073] - OperationMetrics needs to cache the value of hbase.metrics.exposeOperationTimes
[HBASE-6951] - Allow the master info server to be started in a read only mode.
[HBASE-5879] - Enable JMX metrics collection for the Thrift proxy
[HBASE-6979] - recovered.edits file should not break distributed log splitting
[HBASE-6689] - HRegion.increment comment nits
[HBASE-5655] - Cap space usage of default log4j rolling policy

New Feature

[HBASE-7795] - Race in the Restore Archiving
[HBASE-2611] - Handle RS that fails while processing the failure of another one
[HBASE-7689] - CloneTableHandler notify completion too early
[HBASE-7365] - Safer table creation and deletion using .tmp dir
[HBASE-7622] - Add table descriptor verification after snapshot restore
[HBASE-7625] - Remove duplicated logFSTree() from TestRestoreFlushSnapshotFromClient
[HBASE-7604] - Remove duplicated code from HFileLink
[HBASE-7583] - Fixes and cleanups
[HBASE-7547] - Fix findbugs warnings in snapshot classes
[HBASE-7360] - Snapshot 0.94 Backport
[HBASE-7535] - Fix restore reference files
[HBASE-7480] - Explicit message for not allowed snapshot on meta tables
[HBASE-7419] - revisit hfilelink file name format.
[HBASE-7484] - Fix Restore with schema changes
[HBASE-7453] - HBASE-7423 snapshot followup
[HBASE-7354] - Snapshot Info/Debug Tool
[HBASE-7294] - Check for snapshot file cleaners on start
[HBASE-7352] - clone operation from HBaseAdmin can hang forever.
[HBASE-7400] - ExportSnapshot mapper closes the FileSystem
[HBASE-7207] - Consolidate snapshot related classes into fewer packages
[HBASE-7208] - Transition Offline Snapshots to ForeignExceptions
[HBASE-7452] - Change ForeignException#receive(String, FE) to only be #receive(FE)
[HBASE-7439] - HFileLink should not use the configuration from the Filesystem
[HBASE-7339] - Splitting a hfilelink causes region servers to go down.
[HBASE-7436] - Improve stack trace info dumped by ForeignExceptionSnare#rethrowException
[HBASE-7430] - TestSnapshotDescriptionUtils break compaction/scanner tests (EnvironmentEdge issue)
[HBASE-7206] - Foreign Exception framework v2 (simplifies and replaces HBASE-6571)
[HBASE-7420] - TestSnapshotExceptionSnare and TestWALReferenceTask missing test annotation failing TestCheckTestClasses
[HBASE-7311] - Add snapshot information to hbase master webui
[HBASE-7367] - Snapshot coprocessor and ACL security
[HBASE-7174] - [snapshots] Refactor snapshot file cleaner cache to use the Snapshot FileVisitor
[HBASE-7240] - Cleanup old snapshots on start
[HBASE-6802] - Export Snapshot
[HBASE-6230] - [brainstorm] "Restore" snapshots for HBase 0.96
[HBASE-6777] - Snapshot Restore interface
[HBASE-6865] - Snapshot File Cleaners
[HBASE-7107] - Snapshot References Utils (FileSystem Visitor)
[HBASE-6353] - Snapshots shell
[HBASE-6571] - Generic multi-thread/cross-process error handling framework
[HBASE-6765] - 'Take a snapshot' interface
[HBASE-6610] - HFileLink: Hardlink alternative for snapshot restore
[HBASE-7253] - Compaction Tool
[HBASE-4391] - Add ability to start RS as root and call mlockall

Task

[HBASE-6694] - Test scanner batching in export job feature HBASE-6372 AND report on improvement HBASE-6372 adds
[HBASE-7283] - Backport HBASE-6564 + HBASE-7202 to 0.94
[HBASE-7016] - port HBASE-6518 'Bytes.toBytesBinary() incorrect trailing backslash escape' to 0.94
[HBASE-7038] - Port HBASE-5970 Improve the AssignmentManager#updateTimer and speed up handling opened event to 0.94
[HBASE-7170] - [0.94 branch] Allow HConnectionImplementation to reconnect to master multiple times
[HBASE-6202] - Medium tests fail with jdk1.7
[HBASE-6606] - Test for reconnecting with HBaseAdmin using unmanaged HConnection
[HBASE-6553] - Remove Avro Gateway

Test

[HBASE-7142] - TestSplitLogManager#testDeadWorker may fail because of hard limit on the TimeoutMonitor's timeout period
[HBASE-7152] - testShouldCheckMasterFailOverWhenMETAIsInOpenedState times out occasionally
[HBASE-7080] - TestHFileOutputFormat.testColumnFamilyCompression failing due to UnsatisfiedLinkError

Umbrella

[HBASE-6305] - TestLocalHBaseCluster hangs with hadoop 2.0/0.23 builds.
[HBASE-7113] - TestGzipFilter is flaky with jdk1.7
[HBASE-6921] - String and ConcurrentHashMap sizes change on jdk7; makes TestHeapSize fail; second attempt
[HBASE-6601] - TestImportExport on Hadoop 2 must copy YARN configuration vars