CDH 4 Release Notes

Changes Not In Apache Hive 0.9.0

Hive

Bug

• [HIVE-3523] - Hive info logging is broken
• [HIVE-3465] - insert into statement overwrites if target table is prefixed with database name
• [HIVE-3505] - log4j template has logging threshold that hides all audit logs
• [HIVE-3443] - Hive Metatool should take serde_param_key from the user to allow for changes to avro serde's schema url key
• [HIVE-2957] - Hive JDBC doesn't support TIMESTAMP column
• [HIVE-3416] - Fix TestAvroSerdeUtils.determineSchemaCanReadSchemaFromHDFS when running Hive on hadoop23
• [HIVE-3424] - Error by upgrading a Hive 0.7.0 database to 0.8.0 (008-HIVE-2246.mysql.sql)
• [HIVE-3412] - Fix TestCliDriver.repair on Hadoop 0.23.3, 3.0.0, and 2.2.0-alpha
• [HIVE-3413] - Fix pdk.PluginTest on hadoop23
• [HIVE-3295] - HIVE-3128 introduced bug causing dynamic partitioning to fail
• [HIVE-3265] - HiveHistory.printRowCount() throws NPE
• [HIVE-3409] - Increase test.junit.timeout value
• [HIVE-3365] - Upgrade Hive's Avro dependency to version 1.7
• [HIVE-3240] - Fix non-deterministic results in newline.q and timestamp_lazy.q
• [HIVE-3242] - Fix cascade_dbdrop.q when building hive on hadoop0.23
• [HIVE-3013] - TestCliDriver cannot be debugged with eclipse since hadoop_home is set incorrectly
• [HIVE-3049] - setup classpath for templates correctly for eclipse
• [HIVE-3249] - Upgrade guava to 11.0.2
• [HIVE-3275] - Fix autolocal1.q testcase failure when building hive on hadoop0.23 MR2
• [HIVE-3301] - Fix quote printing bug in mapreduce_stack_trace.q testcase failure when running hive on hadoop23
• [HIVE-3303] - Fix error code inconsistency bug in mapreduce_stack_trace.q and mapreduce_stack_trace_turnoff.q when running hive on hadoop23
• [HIVE-3273] - Add avro jars into hive execution classpath
• [HIVE-3257] - Fix avro_joins.q testcase failure when building hive on hadoop0.23
• [HIVE-2990] - Remove hadoop-source Ivy resolvers and Ant targets
• [HIVE-3031] - hive docs target does not work
• [HIVE-3100] - Add HiveCLI that runs over JDBC
• [HIVE-3207] - FileUtils.tar does not close input files
• [HIVE-3206] - FileUtils.tar assumes wrong directory in some cases
• [HIVE-3127] - Pass hconf values as XML instead of command line arguments to child JVM
• [HIVE-3180] - Fix Eclipse classpath template broken in HIVE-3128
• [HIVE-3125] - sort_array doesn't work with LazyPrimitive
• [HIVE-3178] - retry not honored in RetryingRawMetastore
• [HIVE-3128] - use commons-compress instead of forking tar process
• [HIVE-3135] - add an option in ptest to run on a single machine
• [HIVE-2955] - Queries consists of metadata-only-query returns always empty value
• [HIVE-3161] - drop the temporary function at end of autogen_colalias.q
• [HIVE-3009] - do authorization for all metadata operations
• [HIVE-3019] - Add JUnit to list of test dependencies managed by Ivy
• [HIVE-3134] - Drop table/index/database can result in orphaned locations
• [HIVE-3149] - Dynamically generated paritions deleted by Block level merge
• [HIVE-3062] - Insert into table overwrites existing table if table name contains uppercase character
• [HIVE-3123] - Hadoop20Shim. CombineFileRecordReader does not report progress within files
• [HIVE-3090] - Timestamp type values not having nano-second part breaks row
• [HIVE-3081] - ROFL Moment. Numberator and denaminator typos
• [HIVE-3120] - make copyLocal work for parallel tests
• [HIVE-3085] - make parallel tests work
• [HIVE-3112] - clear hive.metastore.partition.inherit.table.properties till HIVE-3109 is fixed
• [HIVE-3101] - dropTable will all ways excute hook.rollbackDropTable whether drop table success or faild.
• [HIVE-2736] - Hive UDFs cannot emit binary constants
• [HIVE-2941] - Hive should expand nested structs when setting the table schema from thrift structs
• [HIVE-3076] - drop partition does not work for non-partition columns
• [HIVE-3063] - drop partition for non-string columns is failing
• [HIVE-3052] - TestHadoop20SAuthBridge always uses the same port
• [HIVE-3057] - metastore.HiveMetaStore$HMSHandler should set the thread local raw store to null in shutdown()
• [HIVE-3059] - revert HIVE-2703
• [HIVE-2540] - LATERAL VIEW with EXPLODE produces ConcurrentModificationException
• [HIVE-3058] - hive.transform.escape.input breaks tab delimited data
• [HIVE-2372] - java.io.IOException: error=7, Argument list too long
• [HIVE-3030] - escape more chars for script operator
• [HIVE-2975] - Filter parsing does not recognize '!=' as operator and silently ignores invalid tokens
• [HIVE-2542] - DROP DATABASE CASCADE does not drop non-native tables.
• [HIVE-2732] - Reduce Sink deduplication fails if the child reduce sink is followed by a join
• [HIVE-2971] - GET_JSON_OBJECT fails on some valid JSON keys
• [HIVE-3000] - Potential infinite loop / log spew in ZookeeperHiveLockManager
• [HIVE-2804] - Task log retrieval fails on Hadoop 0.23
• [HIVE-2878] - TestHBaseMinimrCliDriver hbase_bulk.m fails on 0.23
• [HIVE-3164] - Fix non-deterministic testcases failures when running Hive0.9.0 on MapReduce2
• [HIVE-2860] - TestNegativeCliDriver autolocal1.q fails on 0.23
• [HIVE-2789] - query_properties.q contains non-deterministic queries
• [HIVE-3025] - Fix Hive ARCHIVE command on 0.22 and 0.23
• [HIVE-3014] - Fix metastore test failures caused by HIVE-2757
• [HIVE-2757] - hive can't find hadoop executor scripts without HADOOP_HOME set
• [HIVE-3029] - Update ShimLoader to work with Hadoop 2.x
• [HIVE-2904] - ant gen-test failed
• [HIVE-2721] - ability to select a view qualified by the database / schema name
• [HIVE-2918] - Hive Dynamic Partition Insert - move task not considering 'hive.exec.max.dynamic.partitions' from CLI
• [HIVE-2976] - Fix maven-build Ant target
• [HIVE-2803] - utc_from_timestamp and utc_to_timestamp returns incorrect results.
• [HIVE-2703] - ResultSetMetaData.getColumnType() always returns VARCHAR(string) for partition columns irrespective of partition column type
• [HIVE-2646] - Hive Ivy dependencies on Hadoop should depend on jars directly, not tarballs
• [HIVE-2942] - substr on string containing UTF-8 characters produces StringIndexOutOfBoundsException
• [HIVE-2711] - Make the header of RCFile unique
• [HIVE-2907] - Hive error when dropping a table with large number of partitions
• [HIVE-2929] - race condition in DAG execute tasks for hive

Improvement

• [HIVE-3590] - TCP KeepAlive and connection timeout for the HiveServer
• [HIVE-3277] - Enable Metastore audit logging for non-secure connections
• [HIVE-3146] - Support external hive tables whose data are stored in Azure blob store/Azure Storage Volumes (ASV)
• [HIVE-3048] - Collect_set Aggregate does uneccesary check for value.
• [HIVE-2021] - Add a configuration property that sets the variable substitution max depth
• [HIVE-2585] - Collapse hive.metastore.uris and hive.metastore.local
• [HIVE-2796] - Support auto completion for hive configs in CliDriver
• [HIVE-3106] - Add option to make multi inserts more atomic
• [HIVE-2911] - Move global .hiverc file
• [HIVE-3051] - JDBC cannot find metadata for tables/columns containing uppercase character
• [HIVE-3018] - Make the new header for RC Files introduced in HIVE-2711 optional
• [HIVE-3012] - hive custom scripts do not work well if the data contains new lines
• [HIVE-2956] - [hive] Provide error message when using UDAF in the place of UDF instead of throwing NPE
• [HIVE-2994] - pass a environment context to metastore thrift APIs
• [HIVE-2529] - metastore 0.8 upgrade script for PostgreSQL

New Feature

• [HIVE-2935] - Implement HiveServer2
• [HIVE-3056] - Create a new metastore tool to bulk update location field in Db/Table/Partition records
• [HIVE-2694] - Add FORMAT UDF
• [HIVE-2418] - replace or translate function in hive
• [HIVE-895] - Add SerDe for Avro serialized data
• [HIVE-3066] - Add the option -database DATABASE in hive cli to specify a default database to use for the cli session.
• [HIVE-3001] - Returning Meaningful Error Codes & Messages
• [HIVE-2986] - Create hooks for new test framework
• [HIVE-2928] - Support for Oracle-backed Hive-Metastore ("longvarchar" to "clob" in package.jdo)
• [HIVE-2530] - Implement SHOW TBLPROPERTIES
• [HIVE-2767] - Optionally use framed transport with metastore

Task

• [HIVE-3061] - hive.binary.record.max.length is a magic string
• [HIVE-1719] - Move RegexSerDe out of hive-contrib and over to hive-serde
• [HIVE-3022] - Add hive.exec.rcfile.use.explicit.header to hive-default.xml.template
• [HIVE-3002] - Revert HIVE-2986
• [HIVE-2979] - Implement INCLUDE_HADOOP_MAJOR_VERSION test macro
• [HIVE-2965] - Revert HIVE-2612

Test

• [HIVE-2960] - Stop testing concat of partitions containing control characters.
• [HIVE-2959] - TestRemoteHiveMetaStoreIpAddress always uses the same port
• [HIVE-2937] - TestHiveServerSessions hangs when executed directly

Wish

• [HIVE-2969] - Log Time To Submit metric with PerfLogger