CDH 3 Release Notes

Changes Not In Apache HBase 0.90.4

HBase

Bug

• [HBASE-5196] - Failure in region split after PONR could cause region hole
• [HBASE-4684] - REST server is leaking ZK connections in 0.90
• [HBASE-5081] - Distributed log splitting deleteNode races against splitLog retry
• [HBASE-5009] - Failure of creating split dir if it already exists prevents splits from happening further
• [HBASE-4562] - When split doing offlineParentInMeta encounters error, it'll cause data loss
• [HBASE-4563] - When error occurs in this.parent.close(false) of split, the split region cannot write or read
• [HBASE-2312] - Possible data loss when RS goes into GC pause while rolling HLog
• [HBASE-5077] - SplitLogWorker fails to let go of a task, kills the RS
• [HBASE-5078] - DistributedLogSplitter failing to split file because it has edits for lots of regions
• [HBASE-4997] - SplitLogManager can have a race on batch.installed
• [HBASE-4987] - wrong use of incarnation var in SplitLogManager
• [HBASE-5049] - TestHLogSplit.testLogRollAfterSplitStart not working due to HBASE-5006
• [HBASE-4395] - EnableTableHandler races with itself
• [HBASE-4351] - If from Admin we try to unassign a region forcefully, though a valid region name is given the master is not able to identify the region to unassign.
• [HBASE-4325] - Improve error message when using STARTROW for meta scans.
• [HBASE-4341] - HRS#closeAllRegions should take care of HRS#onlineRegions's weak consistency
• [HBASE-4253] - Intermittent test failure because of missing config parameter in new HTable(tablename)
• [HBASE-4234] - Let regionserver abort if rollback fail after point-of-no-return on branch
• [HBASE-4065] - TableOutputFormat ignores failure to create table instance
• [HBASE-4186] - No region is added to regionsInTransitionInRS
• [HBASE-4168] - A client continues to try and connect to a powered down regionserver
• [HBASE-4144] - RS does not abort if the initialization of RS fails
• [HBASE-4945] - NPE in HRegion.bulkLoadHFiles(...)
• [HBASE-4277] - HRS.closeRegion should be able to close regions with only the encoded name
• [HBASE-4171] - HBase shell broken in trunk
• [HBASE-3065] - Retry all 'retryable' zk operations; e.g. connection loss
• [HBASE-4238] - CatalogJanitor can clear a daughter that split before processing its parent
• [HBASE-4848] - TestScanner failing because hostname can't be null
• [HBASE-4799] - Catalog Janitor logic bug causes region leackage
• [HBASE-4423] - HBASE-4238 broke TestCatalogJanitor#testCleanParent test
• [HBASE-4790] - Occasional TestDistributedLogSplitting failure
• [HBASE-4862] - Splitting hlog and opening region concurrently may cause data loss
• [HBASE-4855] - SplitLogManager hangs on cluster restart due to batch.installed doubly counted
• [HBASE-4685] - TestDistributedLogSplitting.testOrphanLogCreation failing because of ArithmeticException: / by zero.
• [HBASE-4007] - distributed log splitting can get indefinitely stuck
• [HBASE-3889] - NPE in Distributed Log Splitting
• [HBASE-3820] - Splitlog() executed while the namenode was in safemode may cause data-loss
• [HBASE-3843] - splitLogWorker starts too early
• [HBASE-3819] - TestSplitLogWorker has too many SLWs running -- makes for contention and occasional failures
• [HBASE-3674] - Treat ChecksumException as we would a ParseException splitting logs; else we replay split on every restart
• [HBASE-3806] - distributed log splitting double escapes task names
• [HBASE-4854] - it seems that CLASSPATH elements coming from Hadoop change HBase behaviour
• [HBASE-4806] - Fix logging message in HbaseObjectWritable
• [HBASE-4718] - Backport HBASE-4552 to 0.90 branch
• [HBASE-4378] - [hbck] Does not complain about regions with startkey==endkey.
• [HBASE-4508] - Backport HBASE-3777 to 0.90 branch
• [HBASE-4386] - NPE in TaskMonitor
• [HBASE-4570] - Scan ACID problem with concurrent puts.
• [HBASE-4209] - The HBase hbase-daemon.sh SIGKILLs master when stopping it
• [HBASE-4406] - TestOpenRegionHandler failing after HBASE-4287
• [HBASE-4270] - IOE ignored during flush-on-close causes dataloss
• [HBASE-4294] - HLogSplitter sleeps with 1-second granularity
• [HBASE-4290] - HLogSplitter doesn't mark its MonitoredTask as complete in non-distributed case
• [HBASE-3560] - the hbase-default entry of "hbase.defaults.for.version" causes tests not to run via not-maven
• [HBASE-4156] - ZKConfig defaults clientPort improperly
• [HBASE-3465] - Hbase should use a HADOOP_HOME environment variable if available.
• [HBASE-4196] - TableRecordReader may skip first row of region
• [HBASE-4160] - HBase shell move and online may be unusable if region name or server includes binary-encoded data
• [HBASE-4148] - HFileOutputFormat doesn't fill in TIMERANGE_KEY metadata
• [HBASE-3853] - TestInfoServers failing after HBASE-3835
• [HBASE-3440] - Clean out load_table.rb and make sure all roads lead to completebulkload tool
• [HBASE-3864] - Rename of hfile.min.blocksize.size in HBASE-2899 reverted in HBASE-1861

Improvement

• [HBASE-1364] - [performance] Distributed splitting of regionserver commit logs
• [HBASE-4159] - HBaseServer - IPC Reader threads are not daemons
• [HBASE-1938] - Make in-memory table scanning faster
• [HBASE-4944] - Optionally verify bulk loaded HFiles
• [HBASE-4820] - Distributed log splitting coding enhancement to make it easier to understand, no semantics change
• [HBASE-4869] - Backport to 0.92: HBASE-4797 [availability] Skip recovered.edits files with edits older than what region currently has
• [HBASE-4194] - RegionSplitter: Split on under-loaded region servers first
• [HBASE-3963] - Schedule all log-spliiting at startup all at once
• [HBASE-3592] - combine bulk loading tools with hbase shell
• [HBASE-4509] - [hbck] Improve region map output
• [HBASE-4322] - [hbck] Update checkIntegrity/checkRegionChain to present more accurate region split problem summary
• [HBASE-4321] - Add more comprehensive region split calculator
• [HBASE-4313] - Refactor TestHBaseFsck to make adding individual hbck tests easier
• [HBASE-4287] - If region opening fails, try to transition region back to "offline" in ZK
• [HBASE-4257] - Limit the number of regions in transitions displayed on master webpage.
• [HBASE-4272] - Add hbck feature to only inspect and try to repair META and ROOT
• [HBASE-4323] - Add debug logging when AssignmentManager can't make a plan for a region
• [HBASE-4293] - More verbose logging in ServerShutdownHandler for meta/root cases
• [HBASE-4275] - RS should communicate fatal "aborts" back to the master
• [HBASE-4291] - Improve display of regions in transition in UI to be more readable
• [HBASE-4276] - AssignmentManager debug logs should be at INFO level for META/ROOT regions
• [HBASE-4299] - Upgrade to Avro 1.5.3 & use Avro Maven plugin to generate Avro classes
• [HBASE-4269] - Add tests and restore semantics to TableInputFormat/TableRecordReader
• [HBASE-4010] - HMaster.createTable could be heavily optimized
• [HBASE-3835] - Switch web pages to Jamon template engine instead of JSP
• [HBASE-3240] - Improve documentation of importtsv and bulk loads
• [HBASE-3871] - Speedup LoadIncrementalHFiles by parallelizing HFile splitting
• [HBASE-3474] - HFileOutputFormat to use column family's compression algorithm
• [HBASE-3873] - Mavenize Hadoop Snappy JAR/SOs project dependencies
• [HBASE-1861] - Multi-Family support for bulk upload tools (HFileOutputFormat / loadtable.rb)
• [HBASE-3533] - Allow HBASE_LIBRARY_PATH env var to specify extra locations of native libs

New Feature

• [HBASE-2233] - Support both Hadoop 0.20, 0.21, and 0.22
• [HBASE-4377] - [hbck] Offline rebuild .META. from fs data only.
• [HBASE-4375] - [hbck] Add region coverage visualization to hbck
• [HBASE-4292] - Add a debugging dump servlet to the master and regionserver
• [HBASE-4281] - Add facility to dump current state of all executors
• [HBASE-3837] - Expose regionsInTransition on master UI
• [HBASE-4000] - You can't specify split points when you create a table in the shell
• [HBASE-3839] - Expose in-progress tasks on web UIs
• [HBASE-3836] - Add facility to track currently progressing actions/workflows

Task

• [HBASE-5006] - Move hbase 0.92RC1 on to hadoop 1.0.0RC2
• [HBASE-4384] - Hard to tell what causes failure in CloseRegionHandler#getCurrentVersion
• [HBASE-3393] - Update Avro gateway to use Avro 1.4.1 and the new server.join() method
• [HBASE-3691] - Add compressor support for 'snappy', google's compressor

Test

• [HBASE-5112] - TestReplication#queueFailover flaky due to potentially uninitialized Scan
• [HBASE-4302] - Only run Snappy compression tests if Snappy is available