CDH 5.13.0 Release Notes

Changes Not In Apache Pig 0.12.0

Pig

Bug

• [PIG-3655] - BinStorage and InterStorage approach to record markers is broken
• [PIG-3567] - LogicalPlanPrinter throws OOM for large scripts
• [PIG-4052] - TestJobControlSleep, TestInvokerSpeed are unreliable
• [PIG-2499] - Pig TestGrunt.testShellCommand occasionally fails
• [PIG-3807] - Pig creates wrong schema after dereferencing nested tuple fields with sorts
• [PIG-2949] - JsonLoader only reads arrays of objects
• [PIG-3413] - JsonLoader fails the pig job in case of malformed json input
• [PIG-3730] - Performance issue in SelfSpillBag
• [PIG-4355] - Piggybank: XPath cant handle namespace in xpath, nor can it return more than one match
• [PIG-3664] - Piggy Bank XPath UDF can't be called
• [PIG-3670] - Fix assert in Pig script
• [PIG-3779] - Assert constructs ConstantExpression with null when no comment is given
• [PIG-3641] - Split "otherwise" producing incorrect output when combined with ColumnPruning
• [PIG-3100] - If a .pig_schema file is present, can get an index out of bounds error
• [PIG-4155] - Quitting grunt shell using CTRL-D character throws exception
• [PIG-2122] - Parameter Substitution doesn't work in the Grunt shell
• [PIG-3507] - Pig fails to run in local mode on a Kerberos enabled Hadoop cluster
• [PIG-4326] - AvroStorageSchemaConversionUtilities does not properly convert schema for maps of arrays of records
• [PIG-4338] - Fix test failures with JDK8
• [PIG-4053] - TestMRCompiler succeeded with sun jdk 1.6 while failed with sun jdk 1.7
• [PIG-3444] - CONCAT with 2+ input parameters fail
• [PIG-4219] - When parsing a schema, pig drops tuple inside of Bag if it contains only one field
• [PIG-3584] - AvroStorage does not correctly translate arrays of strings
• [PIG-3621] - Python Avro library can't read Avros made with builtin AvroStorage
• [PIG-4133] - Need to update the default $HCAT_HOME dir in the PIG script.
• [PIG-3661] - Piggybank AvroStorage fails if used in more than one load or store statement
• [PIG-3833] - Relation loaded by AvroStorage with schema is projected incorrectly in foreach statement.
• [PIG-3677] - ConfigurationUtil.getLocalFSProperties can return an inconsistent property set
• [PIG-3480] - TFile-based tmpfile compression crashes in some cases
• [PIG-3606] - Pig script throws error when searching for hcatalog jars in latest hive
• [PIG-3726] - Ranking empty records leads to NullPointerException
• [PIG-2629] - Wrong Usage of Scalar which is null causes high namenode operation
• [PIG-3639] - TestRegisteredJarVisibility is broken in trunk
• [PIG-3633] - AvroStorage tests are failing when running against Avro 1.7.5
• [PIG-3285] - Jobs using HBaseStorage fail to ship dependency jars
• [PIG-3496] - Propagate HBase 0.95 jars to the backend
• [PIG-3469] - Skewed join can cause unrecoverable NullPointerException when one of its inputs is missing.
• [PIG-3510] - New filter extractor fails with more than one filter statement
• [PIG-3497] - JobControlCompiler should only do reducer estimation when the job has a reduce phase

Improvement

• [PIG-5025] - Fix flaky test failures in TestLoad.java
• [PIG-4787] - Log JSONLoader exception while parsing records
• [PIG-3619] - Provide XPath function
• [PIG-3569] - SUM function for BigDecimal and BigInteger
• [PIG-4024] - TestPigStreamingUDF and TestPigStreaming fail on IBM JDK.
• [PIG-4160] - Provide a way to pass local jars in pig.additional.jars when using a remote url for a script
• [PIG-3865] - Remodel the XMLLoader to work to be faster and more maintainable

New Feature

• [PIG-5110] - Removing schema alias and :: coming from parent relation
• [PIG-3970] - Merge Tez branch into trunk

Task

• [PIG-3483] - Document ASSERT keyword
• [PIG-3519] - Remove dependency on uber avro-tools jar