CDH 4 Release Notes

Changes Not In Apache Oozie 3.2.0

Oozie

Bug

• [OOZIE-1003] - TestOozieCLI.testSubmitDoAs() should disable anonymous request
• [OOZIE-1004] - Oozie client needs to bundle slf4j JARs
• [OOZIE-989] - Testcases failing intermittently where coordinator jobs are in catchup mode
• [OOZIE-981] - Subworkflow lib not found in classpath when parent workflow lib overwrites it
• [OOZIE-988] - Improve verification of TestJavaActionExecutor.testLibFileArchives
• [OOZIE-975] - Test cases should not load all the services classes
• [OOZIE-971] - TestRecoveryService failing very often in pre-commit builds
• [OOZIE-969] - Unit tests in TestStatusTransitService failing due to change in CoordKillX
• [OOZIE-914] - Make sure all commands do their JPA writes within a single JPA executor
• [OOZIE-930] - Bundle not pass SUSPEND command to PAUSED coord job
• [OOZIE-966] - Fix formatting in CLI output when GMT-#### and GMT-##:## formatted timezones are used
• [OOZIE-960] - TestStatusTransitService failing intermittently
• [OOZIE-965] - Allow the timezone attribute in coordinator jobs to use a format like GMT-####
• [OOZIE-957] - TestStatusTransitService and TestCoordKillXCommand are failing randomly; replace Thread.sleep()
• [OOZIE-477] - Adding configurable filesystem support instead of hardcoded "hdfs"
• [OOZIE-243] - Workflow nodes START/END/KILL/FORK/JOIN should create rows in the action DB table
• [OOZIE-923] - Improve error message when a user tries to start a coordinator job
• [OOZIE-955] - TestCoordELFunctions and TestELConstantFunctions failing
• [OOZIE-708] - Update the parent entity status dynamically
• [OOZIE-954] - Global section xsd should allow job-xml elements and update documentation
• [OOZIE-951] - Global section should make JT and NN optional
• [OOZIE-926] - handling of global configuration is not correct
• [OOZIE-921] - Changes in global section for the Name Node in FS action
• [OOZIE-939] - JT_PRINCIPAL and NN_PRINCIPAL must be added back to XOozieClient
• [OOZIE-904] - TestBundleStartX Failing
• [OOZIE-933] - Modify the Oozie CLI help to escape the semi-colon when multiple filters are specified
• [OOZIE-936] - TestActionFailover failing
• [OOZIE-931] - inconsistency on timestamp format between CLI command and web UI
• [OOZIE-903] - Workflow action status 'Ok' but the workflow job remains in 'RUNNING'
• [OOZIE-920] - Incorrect error message for multiple start instances in coordinator xml
• [OOZIE-915] - Obsolete NN/JT kerberos principal property names not removed from some places
• [OOZIE-707] - Suspending a paused coordinator job suspends the children but the status of job remains unchanged
• [OOZIE-917] - Coordinator Job from 'KILLED' or 'FAILED' doesn't move to 'DONEWITHERROR' if the job materialization is not done
• [OOZIE-916] - aggregator example uses hardcoded 'examples' value instead of parameterized (examplesRoot) value in path
• [OOZIE-907] - Default ACL settings for Oozie launcher
• [OOZIE-890] - Support for map-reduce in OozieCLI
• [OOZIE-885] - A race condition can cause the workflow/coordinator to run even after the bundle job is killed
• [OOZIE-899] - NullPointerException on string check - Missing input dependency
• [OOZIE-882] - CoordELEvaluator.createDataEvaluator doesn't set timezone for coord action
• [OOZIE-850] - apache-rat report should be a single global report file
• [OOZIE-898] - Oozie help [subcommand] should only print help for that subcommand
• [OOZIE-924] - CATALINA_OPTS should include $CATALINA_OPTS in oozie-env.sh
• [OOZIE-892] - increase default JVM max memory to 1GB to avoid OOM
• [OOZIE-922] - Global section code doesn't properly handle extension actions
• [OOZIE-874] - Eliminate redundancies in xml
• [OOZIE-865] - ForkJoin validator checks total lengths of forks vs. joins instead of actual paths
• [OOZIE-896] - TestFsActionExecutor.java failing
• [OOZIE-901] - When Submitting a job without an app path, Oozie doesn't properly handle multiple paths from oozie.libpath
• [OOZIE-902] - XLogService doesn't properly disable WS log streaming if log4j.appender.oozie.File is missing
• [OOZIE-824] - ooziedb tool creates derby.log in current directory
• [OOZIE-880] - oozie.job.acl is not working
• [OOZIE-563] - Missing dependency showing feeds that are already present
• [OOZIE-879] - streaming sharelib brings in lots of unwanted dependencies
• [OOZIE-876] - distcp packaging with Hadoop 1 brings in lots of unwanted dependencies
• [OOZIE-831] - POMs cleanup/fixing of hadoop version and to be able to publish oozie JARs to maven repo
• [OOZIE-801] - Accepts coordinator with start time > end time
• [OOZIE-764] - remove log warnings when credentials are null
• [OOZIE-603] - Invalid regex expression for IDENTIFIER type in xsd files
• [OOZIE-835] - XTestCase Minicluster hangs when shutting down with an exception using Hadoop 2
• [OOZIE-834] - Testcases for MapReduceMain, StreamingMain, PigMain, HiveMain fail with Hadoop 2
• [OOZIE-783] - Upgrade to junit4
• [OOZIE-812] - ooziedb tool does not create VALIDATE_CONN table
• [OOZIE-823] - datelist-java-main example has namenode harcoded in app path the job.properties
• [OOZIE-817] - Fix the unreasonable definition of the schema to fs action
• [OOZIE-871] - actions from subworkflows that use oozie.libpath have duplicate classpath
• [OOZIE-818] - CoordChangeXCommand deletes past coordinator actions
• [OOZIE-839] - materialization of action does not take into account the lookupInterval for the calculations

Improvement

• [OOZIE-977] - NotificationXCommand (job.notification queue entry) should set a timeout in the HTTP connections it makes
• [OOZIE-654] - Provide a way to use 'uber' jars with Oozie MR actions
• [OOZIE-948] - Add support for Oozie coordinator to work in an UTC offset
• [OOZIE-942] - Add formal Parameters to bundle XML
• [OOZIE-239] - Add formal parameters to WF & COORD XML
• [OOZIE-938] - Remove some duplicated code in FsActionExecutor
• [OOZIE-769] - Adding "-debug" option to Oozie CLI for debug statements to stdout
• [OOZIE-937] - Mention how to use the sharelib in the Quick Start page of the docs
• [OOZIE-886] - Display request header info when trace is enabled
• [OOZIE-928] - Clarify the documentation for submitting coordinator jobs using web services API
• [OOZIE-900] - Indicate that the exception thrown from the db-cli during an error is for debug purposes
• [OOZIE-860] - start-instance and end-instance should limit 1
• [OOZIE-861] - allow for use of multiple <java-opts>
• [OOZIE-870] - Parameterize Credentials
• [OOZIE-913] - Add Name Node, job-xml, and configuration Elements to FS action
• [OOZIE-905] - Clarify and improve Oozie logging configuration and streaming
• [OOZIE-881] - Support for recursive chmod
• [OOZIE-872] - Support touchz in fs node of oozie worklfow
• [OOZIE-748] - Add support for oozie.libpath to be set on a per-action basis in workflow.xml
• [OOZIE-895] - Oozie Hive-action should use Hive var replacement
• [OOZIE-875] - Support for multiple job-xml elements in extension action nodes
• [OOZIE-12] - Support for multiple job-xml elements in action nodes
• [OOZIE-559] - Create DistCp WF example
• [OOZIE-649] - Fail fast when a date doesn't parse correctly
• [OOZIE-773] - AuthorizationService should be able to use a group to identify superusers

New Feature

• [OOZIE-887] - Support for choosing timezone in Oozie UI
• [OOZIE-830] - add support for multiple/configurable sharelibs for each action type
• [OOZIE-637] - parametrization of 'name' attribute in workflow/coordinator/bundle

Task

• [OOZIE-888] - Change default logging levels for oozieops and hadoop.security.authentication.server to INFO from DEBUG
• [OOZIE-766] - Verify distcp action works with Hadoop 0.23
• [OOZIE-867] - update version of branch-2 to 3.2.1-incubating-SNAPSHOT