CDH 4 Release Notes

Changes Not In Apache Pig 0.10.0

Pig

Bug

• [PIG-2940] - HBaseStorage store fails in secure cluster
• [PIG-2963] - Illustrate command and POPackageLite
• [PIG-2918] - Avoid Spillable bag overhead where possible
• [PIG-2917] - SpillableMemoryManager memory leak for WeakReference
• [PIG-2890] - Revert PIG-2578
• [PIG-2821] - HBaseStorage should work with secure hbase
• [PIG-2785] - NoClassDefFoundError after upgrading to pig 0.10.0 from 0.9.0
• [PIG-2257] - AvroStorage doesn't recognize schema_file field when JSON isn't used in the constructor
• [PIG-2782] - Specifying sorting field(s) at nightly.conf
• [PIG-2860] - TestAvroStorageUtils.testGetConcretePathFromGlob fails on some version of hadoop
• [PIG-2856] - AvroStorage doesn't load files in the directories when a glob pattern matches both files and directories.
• [PIG-2837] - AvroStorage throws StackOverFlowError
• [PIG-2777] - Docs are broken due to malformed xml after PIG-2673
• [PIG-2569] - Fix org.apache.pig.test.TestInvoker.testSpeed
• [PIG-2854] - AvroStorage doesn't work with Avro 1.7.1
• [PIG-2501] - Changes needed to contrib/piggybank/java/build.xml in order to build piggybank.jar with Hadoop 0.23
• [PIG-2766] - Pig-HCat Usability
• [PIG-2775] - Register jar does not goes to classpath in some cases
• [PIG-2761] - With hadoop23 importing modules inside python script does not work
• [PIG-2759] - Typo in document "Built In Functions"
• [PIG-2745] - Pig e2e test RubyUDFs fails in MR mode when running from tarball
• [PIG-2741] - Python script throws an NameError: name 'Configuration' is not defined in case cache dir is not created
• [PIG-2669] - Pig release should include pig-default.properties after rebuild
• [PIG-2739] - PyList should map to Bag automatically in Jython
• [PIG-2730] - TFileStorage getStatistics incorrectly throws an exception instead of returning null
• [PIG-2717] - Tuple field mangled during flattening
• [PIG-2721] - Wrong output generated while loading bags as input
• [PIG-2727] - PigStorage Source tagging does not need pig.splitCombination to be turned off
• [PIG-2666] - LoadFunc.setLocation() is not called when pig script only has Order By
• [PIG-2680] - TOBAG output schema reporting
• [PIG-2685] - error in EvalFunc ctor when implementing Algebraic UDF whose return type is parameterized
• [PIG-2671] - e2e harness: Reference local test path via :LOCALTESTPATH:
• [PIG-2670] - glitches on copyright years in documentation

Improvement

• [PIG-2923] - Lazily register bags with SpillableMemoryManager
• [PIG-2909] - Add a new option for ignoring corrupted files to AvroStorage load func
• [PIG-2875] - Add recursive record support to AvroStorage
• [PIG-2492] - AvroStorage should recognize globs and commas
• [PIG-2673] - Allow Merge join to follow an ORDER statement
• [PIG-2711] - e2e harness: cache benchmark results between test runs

New Feature

• [PIG-2650] - Convenience mock Loader and Storer to simplify unit testing of Pig scripts