CDH 3 Release Notes

Changes Not In Flume NG 1.1.0

Flume

Bug

• [FLUME-1182] - Syslog source cannot read format correctly from configuration
• [FLUME-1180] - ChannelSelectorFactory creates incorrect selector for multiplexing selector type
• [FLUME-1088] - TestWAL.testThreadedAppend fails on jenkins build server
• [FLUME-1156] - If config file has empty sources, then throws NPE
• [FLUME-1112] - HDFSCompressedDataStream append does not work
• [FLUME-1148] - Refactor logging
• [FLUME-1149] - All sources get same channel list even if configuration is different.
• [FLUME-1142] - Seq source fails with multiplexing channel selector
• [FLUME-1096] - Add support to pass headers through AvroCLIClient
• [FLUME-1135] - flume-docs exclude is not sufficient for rat
• [FLUME-1130] - flume-ng script bad ordering on FLUME_HOME var
• [FLUME-1136] - Remove from executor service does not always remove the runnables from the queue
• [FLUME-1129] - change foo to agent in sample config
• [FLUME-1124] - Lifecycle supervisor can cause thread contention, sometimes causing components to not startup.
• [FLUME-1125] - flume-ng script allows flume-env.sh to clobber some command-line arguments
• [FLUME-1098] - Hadoop jars from compilation step included in assembly build
• [FLUME-1119] - Remove default ports for syslog sources
• [FLUME-1116] - Extra event created for max payload size of 2500 bytes in Flume syslogtcp source
• [FLUME-1114] - Syslog Sources does not implement maxsize
• [FLUME-1110] - HDFS Sink throws IllegalStateException when flume-daemon shuts down
• [FLUME-1109] - Syslog sources need to be refactored
• [FLUME-1104] - HDFS rolls the first file incorrectly
• [FLUME-1017] - syslog classes missing
• [FLUME-1102] - HDFSEventSink rollInterval is broken
• [FLUME-1099] - copy-paste issue with RecoverableMemoryChannel
• [FLUME-1094] - hadoop.profile=23 build is broken by slf4j-jcl dependencies
• [FLUME-1074] - AvroSink if any non-caught exception is thrown, an exception is thrown in finally clause
• [FLUME-1070] - Fix javadoc for configuring hdfsSink
• [FLUME-1075] - HDFSEventSink begin is called when transaction opened due to other error
• [FLUME-1087] - Restore Client API compat with v1.1.0
• [FLUME-1080] - Issue with HDFSEventSink for append support
• [FLUME-1079] - Flume agent reconfiguration enters permanent bad state
• [FLUME-1040] - Release-Notes says Apache Ivy instead of Apache Flume
• [FLUME-1037] - NETCAT handler theads terminate under stress test
• [FLUME-1047] - Client SDK has dependency on apache commons
• [FLUME-1036] - Reconfiguration of AVRO or NETCAT source causes port bind exception
• [FLUME-1046] - invoking flume-ng script from bin directory fails
• [FLUME-1041] - maven warns of duplicate dependencies
• [FLUME-1026] - Document Thread Safety Guarantees
• [FLUME-1002] - FailoverSinkProcessor replaces sinks with same priority
• [FLUME-1035] - slf4j error in flume sdk unit tests

Improvement

• [FLUME-1126] - Support RFC 3164 and 5424 syslog format timestamp parsing
• [FLUME-1122] - Flume documentation layout should be refactored
• [FLUME-1127] - Add configuration support to AbstractAvroEventSerializer for Avro sync interval and compression support
• [FLUME-1078] - flume-ng script has no way to add, not replace, classpath
• [FLUME-1117] - Support output to files in Avro container format
• [FLUME-1113] - JDBC Channel invokes size query on every put
• [FLUME-1011] - AvroSource should have a configurable max thread count
• [FLUME-1107] - Configuration keys for JDBC channel contain redundant prefix.
• [FLUME-1105] - Allow the optional disabling of foreign keys
• [FLUME-1090] - JDBC Channel: Minimize logging under nominal conditions
• [FLUME-881] - Would be nice if HDFS Sink would automatically choose best writableFormat based on fileType
• [FLUME-985] - All HDFS Operations in HDFSEventSink should have a timeout
• [FLUME-1048] - speed up mvn package: stop building .zip packages
• [FLUME-1030] - Retry logic for failover sink processor to handle downstream exceptions in a predictable manner.
• [FLUME-1049] - Use hadoop-1.0.0 as basis for default Flume build instead of 0.20.205
• [FLUME-1043] - SDK should mark slf4j deps as optional
• [FLUME-1020] - Implement Kerberos security for HDFS Sink
• [FLUME-828] - LoggerSink representation of the event's body isn't too useful
• [FLUME-979] - ExecSource should optionally restart the command when it exits

New Feature

• [FLUME-896] - Implement a Durable Memory Channel
• [FLUME-962] - Failover capability for Client SDK

Task

• [FLUME-1052] - Core configuration component
• [FLUME-999] - Updating init scripts and variables to fit the term agent
• [FLUME-1073] - Default Log4j configuration file should have a rolling policy
• [FLUME-1050] - Update version of surefire plugin

Test

• [FLUME-1082] - Add User and dev guide to Flume site