CDH 3 Release Notes

Changes Not In Flume NG 1.2.0

Flume

Bug

• [FLUME-1616] - FileChannel will lose data in when rollback fails with IOException
• [FLUME-1567] - Avro source should expose the number of active connections through JMX
• [FLUME-1610] - HDFSEventSink and bucket writer have a race condition
• [FLUME-1606] - Rollbacks of Put transactions does not clear the transaction from inflight puts.
• [FLUME-1583] - FileChannel fast full replay will always be used if enabled
• [FLUME-1577] - CHECKPOINT_INCOMPLETE should be synced to disk before starting the checkpoint.
• [FLUME-1572] - Add batching to FILE_ROLL sink
• [FLUME-1557] - It would be nice if SequenceGeneratorSource could do batching
• [FLUME-1555] - StressSource outputs bad log messages that reference (Sequence generator)
• [FLUME-1545] - File channel missing implicit dependency on commons-lang
• [FLUME-1541] - Implement a SinkSelector for LoadBalancingSinkProcessor that includes failover mechanics
• [FLUME-1482] - Flume should support exposing metrics via HTTP in JSON/some other web service format.
• [FLUME-1500] - Upgrade flume to use latest version of Avro - v1.7
• [FLUME-1534] - CheckpointRebuilder$ComparableFlumeEventPointer#equals does not work correctly.
• [FLUME-1527] - Seed TransactionID/WriteOrderID in the case no logs need to be replayed
• [FLUME-1513] - File Channel log close() method should not be synchronized
• [FLUME-1428] - File Channel should not consider a file as inactive until all takes are committed.
• [FLUME-1512] - File Channel should not stop during a checkpoint.
• [FLUME-1504] - Test file channel times out randomly
• [FLUME-1498] - File channel - Log updates and queue updates should be atomic
• [FLUME-1437] - Checkpoint can miss pending takes.
• [FLUME-1432] - FileChannel should replay logs in the order they were written
• [FLUME-1417] - File Channel checkpoint can be bad leading to the channel being unable to start.
• [FLUME-1401] - Asynchbase sink should be configurable to support timeout
• [FLUME-1398] - Improve concurrency for async hbase sink
• [FLUME-1414] - VersionInfo should not create a log instance
• [FLUME-1412] - Commons collections is used in file channel - even though it is not in pom.xml
• [FLUME-1386] - Ganglia metrics plugin should filter MBean name
• [FLUME-1373] - Remove hardcoded file separator in HDFSEventSink
• [FLUME-1380] - File channel log can record the op code and not the operation in some cases
• [FLUME-1377] - ChannelProcessor clobbers exception with NPE if Channel.getTransaction() throws
• [FLUME-1376] - StaticInterceptor doc update
• [FLUME-1374] - Support ganglia reporting
• [FLUME-1369] - Code and user guide documentation out of sync for interceptors
• [FLUME-1368] - In user guide, property sink.directory for file roller sink should be bold
• [FLUME-1362] - Port retrying in TestThriftLegacySource not working
• [FLUME-1364] - Document the necessity of the timestamp header when using time-related escapes for hdfs sink paths
• [FLUME-1363] - flume-ng-node - TestNetcatSource doesn't try multiple ports
• [FLUME-1310] - Make Asynch hbase sink test work with other versions of Hbase
• [FLUME-1276] - Create a static header interceptor

Improvement

• [FLUME-1523] - Allow -X java opts to be passed to the agent on the command line
• [FLUME-1538] - Channels should expose channel fill ratio through JMX
• [FLUME-1536] - Support for batch size in StressSource
• [FLUME-1526] - LogFile log message is scary when no harm has been done
• [FLUME-1490] - Option to limit number of events sent in Stress source
• [FLUME-1528] - File Channel replay when no checkpoint is present can be faster.
• [FLUME-1505] - TestFileChannel needs to be able to force a checkpoint
• [FLUME-1434] - Distinguish background worker with channel name
• [FLUME-1387] - Ganglia graphs should work with Flume
• [FLUME-1383] - Improve various log messages in FileChannel and HDFSSink
• [FLUME-1360] - Provide documentation for static interceptor
• [FLUME-1361] - Add event batching to ExecSource

New Feature

• [FLUME-1537] - Dump RollingFileSink's counter status when agent stops
• [FLUME-1488] - Load Balancing RPC client should support exponential backoff of failed nodes