CDH 4 Release Notes

Changes Not In Apache Oozie 3.1.3

Oozie

Bug

• [OOZIE-802] - Add minicluster config as hadoop-site.xml for testcases
• [OOZIE-799] - Testcases failing due to missing action-conf directory
• [OOZIE-798] - it would be nice to keep action specific configs in action-conf instead of hadoop-conf
• [OOZIE-793] - remove HiveMain check for HADOOP_HOME
• [OOZIE-789] - a few testcases using waitFor are timing out with YARN MiniCluster
• [OOZIE-788] - JavaActionExecutor should not set yarn.resourcemanager.address
• [OOZIE-779] - Oozie cannot submit jobs to hadoop 0.23 if running in non-default host:port
• [OOZIE-777] - Hadoop 0.23 removed org.apache.hadoop.mapred.Task$Counter group name breaking EL counters()
• [OOZIE-776] - custom-main example job.properties is missing the activation of sharelib
• [OOZIE-775] - sharelibs assembly does not include deps with compile scope
• [OOZIE-772] - enable Hive and Email actions by default
• [OOZIE-771] - oozied.sh should exec catalina.sh instead of forking it off
• [OOZIE-768] - latest Hadoop 0.23.3 minicluster is not creating /tmp in HDFS
• [OOZIE-760] - oozied.sh needs to be able to pick up CATALINA_HOME
• [OOZIE-749] - oozie tests doesn't delete the files in tmp directories
• [OOZIE-761] - Fixes in CoordELFunctions and testcases for Hadoop 0.23
• [OOZIE-755] - remove hadoop.job.ugi and kerberos principal hardcoded injections
• [OOZIE-757] - hive-contrib version should use ${hive.version}
• [OOZIE-753] - default hadoop config file does not endup in the distro and hadoop config doc typo
• [OOZIE-752] - if group.name/oozie.job.acl is not defined in job.properties job submission fails
• [OOZIE-746] - JobConf/Configuration creation is inconsistent and makes things fail in odd ways
• [OOZIE-744] - HadoopAccessorService hadoop-configs loading logic is not 100% correct
• [OOZIE-743] - HadoopAccessorService hadoop configurations should not use VARs from properties defined oozie-site.xml
• [OOZIE-738] - HadoopAccessorService configs typo/missed value
• [OOZIE-737] - ooziedb tool does not register SLAEventBean resulting in an incorrect DB creation
• [OOZIE-729] - SubmitMRCommand & SubmitMRXCommant testcases fail with Hadoop 0.23
• [OOZIE-715] - Fix TestLocalOozieExample
• [OOZIE-724] - TestClassUtils fails as looks for hadoop-core
• [OOZIE-701] - Oozie notification URLs don't get replaced with the taken transition
• [OOZIE-228] - For authorization we should use the ACL model
• [OOZIE-726] - Removing switch from default runtime scope to compile scope for hadooplib.xml assembly due to it pulling in JDK tools.jar with compile scope.
• [OOZIE-712] - Hive testcases fail with 0.23
• [OOZIE-704] - Sharelib action JARs for testcases should be collected leveraging Maven dependency information
• [OOZIE-702] - XTestCase minicluster fails with Hadoop 1.x
• [OOZIE-696] - scope of oozie-hadoop-test artifact is wrong
• [OOZIE-691] - Fix testcases using launcher using Hadoop 0.23.x
• [OOZIE-499] - Broken link in maven docs to config files
• [OOZIE-636] - Check fork and join in the workflow in the submission time
• [OOZIE-673] - offset and len options not working as expected
• [OOZIE-668] - Adding license header into minitest/pom.xml
• [OOZIE-665] - Shell action doesn't capture multiple key-value pairs
• [OOZIE-666] - Oozie's Tomcat admin port is hardcoded
• [OOZIE-662] - Unit test failing: TestHostnameFilter
• [OOZIE-651] - Coordinator rerun fails due to NPE in some cases
• [OOZIE-642] - YEAR support in coord:dataOffset() el function
• [OOZIE-639] - Hive sharelib POM must exclude hadoop-core
• [OOZIE-635] - ShellMain closes the STD/ERR stream while shell is processing
• [OOZIE-629] - Oozie server to prevent usage of dataset initial-instance earlier than the system-defined default value
• [OOZIE-8] - variable not replaced in workflow rerun
• [OOZIE-15] - Coordinator input event instance should limit 1 instance
• [OOZIE-634] - Add DB-site.xml to example project
• [OOZIE-625] - Oozie server not starting due to missing sl4j library for Authenticationfilter
• [OOZIE-620] - POMs clean/tune up before 3.2 release
• [OOZIE-613] - Update documentation for execution order
• [OOZIE-156] - Add support for a SQOOP action
• [OOZIE-77] - Oozie should support Kerberos authentication on its HTTP REST API
• [OOZIE-622] - Remove system sharelib tests from TestLiteWorkflowAppService
• [OOZIE-68] - Add Hive action
• [OOZIE-609] - Oozie services fail to start with log enabled
• [OOZIE-607] - Pig POM brings in several unneeded dependencies
• [OOZIE-480] - OOZIE-63: In Oozie-site.xml, if we specify oozie.services.ext property is not overriding the services.
• [OOZIE-587] - Out of memory issues due to current query design

Improvement

• [OOZIE-803] - Remove HadoopAccessorService createFileSystem() method that takes config only
• [OOZIE-786] - tomcat should stop if oozie does not start correctly
• [OOZIE-274] - move callback HTTP endpoint outside of the versioned WS endpoints
• [OOZIE-742] - Return workflow external id in getJobInfo
• [OOZIE-750] - enhance ooziedb tool not to require manual upgrade steps and not to require the -sqlfile option
• [OOZIE-747] - HadoopAccessorService hadoop-configs should be loaded from Hadoop -site.xml files
• [OOZIE-723] - getting rid of the old Command classes
• [OOZIE-734] - Simplify Kerberos/HadoopAccessorService code and remove Kerberos/DoAs code
• [OOZIE-727] - make test timeout value configurable via -D
• [OOZIE-710] - add a timeout to the surefire configuration
• [OOZIE-703] - Improve/Consolidate Hadoop job ID log harvesting logic
• [OOZIE-698] - Make sharelib components version dependency configurable.
• [OOZIE-690] - use hadoop-client/hadoop-minicluster artifacts for Hadoop 0.23.1 & trunk
• [OOZIE-667] - Change the way Oozie brings in Hadoop JARs into the build
• [OOZIE-621] - Option to add timeunit for frequency for coordinator jobs filtering
• [OOZIE-628] - Add libraries which can be used by all actions in oozie
• [OOZIE-610] - Oozie system share lib should have jars per action type
• [OOZIE-565] - Make Oozie compile against Hadoop 0.23
• [OOZIE-557] - Simplify/normalize testing configuration when using different databases

New Feature

• [OOZIE-792] - add default action configs per cluster
• [OOZIE-736] - Add support for configurations per JT/NN
• [OOZIE-631] - Oozie DB create/upgrade tool
• [OOZIE-652] - Add proxyuser capabilities to Oozie HTTP API
• [OOZIE-588] - Oozie to allow drill down to hadoop job's details
• [OOZIE-633] - Create sharelib directory for oozie
• [OOZIE-578] - Support shell action in Oozie WF
• [OOZIE-589] - Make the command requeue interval configurable

Task

• [OOZIE-754] - remove use of group from HadoopAccessor
• [OOZIE-725] - Increase JVM MaxPermSize value for the testcases
• [OOZIE-721] - remove obsolete/not-used JobServlet and JobServlet code
• [OOZIE-700] - update hadooplibs versions
• [OOZIE-617] - Add back Ssh action as extension. (tucu)

Test

• [OOZIE-582] - Adding new test cases for the feature - viewing log for coordinator actions in a given date range