CDH 5.3.6 Release Notes
The following lists all Apache Hive Jiras included in CDH 5.3.6
that are not included in the Apache Hive base version 0.13.1. The
hive-0.13.1-cdh5.3.6.CHANGES.txt
file lists all changes included in CDH 5.3.6. The patch for each
change can be found in the cloudera/patches directory in the release tarball.
Changes Not In Apache Hive 0.13.1
Common
Improvement
- [HADOOP-10448] - Support pluggable mechanism to specify proxy user settings
Hive
Bug
- [HIVE-7202] - DbTxnManager deadlocks in hcatalog.cli.TestSematicAnalysis.testAlterTblFFpart()
- [HIVE-9625] - Delegation tokens for HMS are not renewed
- [HIVE-10571] - HiveMetaStoreClient should close existing thrift connection before its reconnect
- [HIVE-10801] - 'drop view' fails throwing java.lang.NullPointerException
- [HIVE-11150] - Remove wrong warning message related to chgrp
- [HIVE-8318] - Null Scan optimizer throws exception when no partitions are selected
- [HIVE-7385] - Optimize for empty relation scans
- [HIVE-10808] - Inner join on Null throwing Cast Exception
- [HIVE-10841] - [WHERE col is not null] does not work sometimes for queries with many JOIN statements
- [HIVE-9620] - Cannot retrieve column statistics using HMS API if column name contains uppercase characters
- [HIVE-8863] - Cannot drop table with uppercase name after "compute statistics for columns"
- [HIVE-10956] - HS2 leaks HMS connections
- [HIVE-8298] - Incorrect results for n-way join when join expressions are not in same order across joins
- [HIVE-8895] - bugs in mergejoin
- [HIVE-10771] - "separatorChar" has no effect in "CREATE TABLE AS SELECT" statement
- [HIVE-6679] - HiveServer2 should support configurable the server side socket timeout and keepalive for various transports types where applicable
- [HIVE-10732] - Hive JDBC driver does not close operation for metadata queries
- [HIVE-8411] - Support partial partition spec for certain ALTER PARTITION statements
- [HIVE-7971] - Support alter table change/replace/add columns for existing partitions
- [HIVE-7027] - Hive job fails when referencing a view that explodes an array
- [HIVE-10646] - ColumnValue does not handle NULL_TYPE
- [HIVE-10453] - HS2 leaking open file descriptors when using UDFs
- [HIVE-9655] - Dynamic partition table insertion error
- [HIVE-10202] - Beeline outputs prompt+query on standard output when used in non-interactive mode
- [HIVE-10312] - SASL.QOP in JDBC URL is ignored for Delegation token Authentication
- [HIVE-10087] - Beeline's --silent option should suppress query from being echoed when running with -f option
- [HIVE-10098] - HS2 local task for map join fails in KMS encrypted cluster
- [HIVE-10085] - Lateral view on top of a view throws RuntimeException
- [HIVE-10086] - Hive throws error when accessing Parquet file schema using field name match
- [HIVE-7426] - ClassCastException: ...IntWritable cannot be cast to ...Text involving ql.udf.generic.GenericUDFBasePad.evaluate
- [HIVE-9934] - Vulnerability in LdapAuthenticationProviderImpl enables HiveServer2 client to degrade the authentication mechanism to "none", allowing authentication without password
- [HIVE-9828] - Semantic analyzer does not capture view parent entity for tables referred in view with union all
- [HIVE-9749] - ObjectStore schema verification logic is incorrect
- [HIVE-9755] - Hive built-in "ngram" UDAF fails when a mapper has no matches.
- [HIVE-9770] - Beeline ignores --showHeader for non-tablular output formats i.e csv,tsv,dsv
- [HIVE-8688] - serialized plan OutputStream is not being closed
- [HIVE-9716] - Map job fails when table's LOCATION does not have scheme
- [HIVE-5857] - Reduce tasks do not work in uber mode in YARN
- [HIVE-6978] - beeline always exits with 0 status, should exit with non-zero status on error
- [HIVE-8938] - Compiler should save the transform URI as input entity
- [HIVE-9023] - HiveHistoryImpl relies on removed counters to print num rows
- [HIVE-7733] - Ambiguous column reference error on query
- [HIVE-9303] - Parquet files are written with incorrect definition levels
- [HIVE-8444] - update pom to junit 4.11
- [HIVE-9474] - truncate table changes permissions on the target
- [HIVE-9234] - HiveServer2 leaks FileSystem objects in FileSystem.CACHE
- [HIVE-9462] - HIVE-8577 - breaks type evolution
- [HIVE-8577] - Cannot deserialize Avro schema with a map<string,string> with null values
- [HIVE-9482] - Hive parquet timestamp compatibility
- [HIVE-6308] - COLUMNS_V2 Metastore table not populated for tables created without an explicit column list.
- [HIVE-9502] - Parquet cannot read Map types from files written with Hive <= 0.12
- [HIVE-9393] - reduce noisy log level of ColumnarSerDe.java:116 from INFO to DEBUG
- [HIVE-7800] - Parquet Column Index Access Schema Size Checking
- [HIVE-9330] - DummyTxnManager will throw NPE if WriteEntity writeType has not been set
- [HIVE-9199] - Excessive exclusive lock used in some DDLs with DummyTxnManager
- [HIVE-8891] - Another possible cause to NucleusObjectNotFoundException from drops/rollback
- [HIVE-8889] - JDBC Driver ResultSet.getXXXXXX(String columnLabel) methods Broken
- [HIVE-8909] - Hive doesn't correctly read Parquet nested types
- [HIVE-8916] - Handle user@domain username under LDAP authentication
- [HIVE-8359] - Map containing null values are not correctly written in Parquet files
- [HIVE-8808] - HiveInputFormat caching cannot work with all input formats
- [HIVE-7011] - HiveInputFormat's split generation isn't thread safe
- [HIVE-5454] - HCatalog runs a partition listing with an empty filter
- [HIVE-8745] - Joins on decimal keys return different results whether they are run as reduce join or map join
- [HIVE-6853] - show create table for hbase tables should exclude LOCATION
- [HIVE-8784] - Querying partition does not work with JDO enabled against PostgreSQL
- [HIVE-8791] - Hive permission inheritance throws exception S3
- [HIVE-8827] - Remove SSLv2Hello from list of disabled protocols
- [HIVE-8704] - HivePassThroughOutputFormat cannot proxy more than one kind of OF (in one process)
- [HIVE-8680] - Set Max Message for Binary Thrift endpoints
- [HIVE-8713] - Unit test TestParquetTimestampUtils.testTimezone failing
- [HIVE-8757] - YARN dep in scheduler shim should be optional
- [HIVE-8708] - Add query id to explain log option
- [HIVE-8693] - Separate out fair scheduler dependency from hadoop 0.23 shim
- [HIVE-8634] - HiveServer2 fair scheduler queue mapping doesn't handle the secondary groups rules correctly
- [HIVE-8615] - beeline csv,tsv outputformat needs backward compatibility mode
- [HIVE-6847] - Improve / fix bugs in Hive scratch dir setup
- [HIVE-8627] - Compute stats on a table from impala caused the table to be corrupted
- [HIVE-8600] - Add option to log explain output for query
- [HIVE-7764] - Support all JDBC-HiveServer2 authentication modes on a secure cluster
- [HIVE-8448] - Union All might not work due to the type conversion issue
- [HIVE-7107] - Fix HiveServer1 JDBC Driver spec compliancy issue
- [HIVE-5865] - AvroDeserializer incorrectly assumes keys to Maps will always be of type 'org.apache.avro.util.Utf8'
- [HIVE-7618] - TestDDLWithRemoteMetastoreSecondNamenode unit test failure
- [HIVE-7902] - Cleanup hbase-handler/pom.xml dependency list
- [HIVE-6374] - Hive job submitted with non-default name node (fs.default.name) doesn't process locations properly
- [HIVE-7932] - It may cause NP exception when add accessed columns to ReadEntity
- [HIVE-8182] - beeline fails when executing multiple-line queries with trailing spaces
- [HIVE-8380] - NanoTime class serializes and deserializes Timestamp incorrectly
- [HIVE-7263] - Missing fixes from review of parquet-timestamp
- [HIVE-7433] - ColumnMappins.ColumnMapping should expose public accessors for its fields
- [HIVE-7109] - Resource leak in HBaseStorageHandler
- [HIVE-8345] - q-test for Avro date support
- [HIVE-7154] - TestMetrics fails intermittently on the trunk
- [HIVE-8266] - create function using <resource> statement compilation should include resource URI entity
- [HIVE-8317] - WebHCat pom should explicitly depend on jersey-core
- [HIVE-7200] - Beeline output displays column heading even if --showHeader=false is set
- [HIVE-8205] - Using strings in group type fails in ParquetSerDe
- [HIVE-7017] - Insertion into Parquet tables fails under Tez
- [HIVE-7201] - Fix TestHiveConf#testConfProperties test case
- [HIVE-8138] - Global Init file should allow specifying file name not only directory
- [HIVE-7647] - Beeline does not honor --headerInterval and --color when executing with "-e"
- [HIVE-7441] - Custom partition scheme gets rewritten with hive scheme upon concatenate
- [HIVE-8083] - Authorization DDLs should not enforce hive identifier syntax for user or group
- [HIVE-7777] - Add CSV Serde based on OpenCSV
- [HIVE-8139] - Upgrade commons-lang from 2.4 to 2.6
- [HIVE-7353] - HiveServer2 using embedded MetaStore leaks JDOPersistanceManager
- [HIVE-7373] - Hive should not remove trailing zeros for decimal numbers
- [HIVE-8034] - Don't add colon when no port is specified
- [HIVE-3756] - "LOAD DATA" does not honor permission inheritence
- [HIVE-6785] - query fails when partitioned table's table level serde is ParquetHiveSerDe and partition level serde is of different SerDe
- [HIVE-1363] - 'SHOW TABLE EXTENDED LIKE' command does not strip single/double quotes
- [HIVE-7629] - Problem in SMB Joins between two Parquet tables
- [HIVE-7620] - Hive metastore fails to start in secure mode due to "java.lang.NoSuchFieldError: SASL_PROPS" error
- [HIVE-6820] - HiveServer(2) ignores HIVE_OPTS
- [HIVE-7220] - Empty dir in external table causes issue (root_dir_external_table.q failure)
- [HIVE-8008] - NPE while reading null decimal value
- [HIVE-6741] - HiveServer2 startup fails in secure (kerberos) mode due to backward incompatible hadoop change
- [HIVE-7682] - HadoopThriftAuthBridge20S should not reset configuration unless required
- [HIVE-7890] - SessionState creates HMS Client while not impersonating
- [HIVE-6245] - HS2 creates DBs/Tables with wrong ownership when HMS setugi is true
- [HIVE-7730] - Extend ReadEntity to add accessed columns from query
- [HIVE-7889] - Query fails with char partition column
- [HIVE-6745] - HCat MultiOutputFormat hardcodes DistributedCache keynames
- [HIVE-7298] - desc database extended does not show properties of the database
- [HIVE-7824] - CLIServer.getOperationStatus eats ExceutionException
- [HIVE-6185] - DDLTask is inconsistent in creating a table and adding a partition when dealing with location
- [HIVE-7829] - Entity.getLocation can throw an NPE
- [HIVE-7635] - Query having same aggregate functions but different case throws IndexOutOfBoundsException
- [HIVE-7279] - UDF format_number() does not work on DECIMAL types
- [HIVE-7257] - UDF format_number() does not work on FLOAT types
- [HIVE-5456] - Queries fail on avro backed table with empty partition
- [HIVE-7345] - Beeline changes its prompt to reflect successful database connection even after failing to connect
- [HIVE-6622] - UDF translate doesn't take either CHAR or VARCHAR as any of its arguments
- [HIVE-7459] - Fix NPE when an empty file is included in a Hive query that uses CombineHiveInputFormat
- [HIVE-7042] - Fix stats_partscan_1_23.q and orc_createas1.q for hadoop-2
- [HIVE-7004] - Fix more unit test failures on hadoop-2
- [HIVE-6947] - More fixes for tests on hadoop-2
- [HIVE-6960] - Set Hive pom to use Hadoop-2.4
- [HIVE-7149] - Parquet not able to handle negative decimal numbers
- [HIVE-6684] - Beeline does not accept comments that are preceded by spaces
- [HIVE-6979] - Hadoop-2 test failures related to quick stats not being populated correctly
- [HIVE-7450] - Database should inherit perms of warehouse dir
- [HIVE-6858] - Unit tests decimal_udf.q, vectorization_div0.q fail with jdk-7.
- [HIVE-6602] - Multi-user HiveServer2 throws error
- [HIVE-6322] - Fix file_with_header_footer_negative.q
- [HIVE-6880] - TestHWISessionManager fails with -Phadoop-2
- [HIVE-7075] - JsonSerde raises NullPointerException when object key is not lower case
- [HIVE-7053] - Unable to fetch column stats from decimal columns
- [HIVE-7170] - Fix display_colstats_tbllvl.q in trunk
- [HIVE-7050] - Display table level column stats in DESCRIBE EXTENDED/FORMATTED TABLE
- [HIVE-6928] - Beeline should not chop off "describe extended" results by default
- [HIVE-4561] - Column stats : LOW_VALUE (or HIGH_VALUE) will always be 0.0000 ,if all the column values larger than 0.0 (or if all column values smaller than 0.0)
- [HIVE-7117] - Partitions not inheriting table permissions after alter rename partition
- [HIVE-7119] - Extended ACL's should be inherited if warehouse perm inheritance enabled
- [HIVE-7092] - Insert overwrite should not delete the original directory
- [HIVE-6913] - Hive unable to find the hashtable file during complex multi-staged map join
- [HIVE-7099] - Add Decimal datatype support for Windowing
- [HIVE-7015] - Failing to inherit group/permission should not fail the operation
- [HIVE-6916] - Export/import inherit permissions from parent directory
- [HIVE-6891] - Alter rename partition Perm inheritance and general partition/table group inheritance
- [HIVE-6648] - Permissions are not inherited correctly when tables have multiple partition columns
- [HIVE-6893] - out of sequence error in HiveMetastore server
- [HIVE-5847] - DatabaseMetadata.getColumns() doesn't show correct column size for char/varchar/decimal
- [HIVE-6792] - hive.warehouse.subdir.inherit.perms doesn't work correctly in CTAS
- [HIVE-6793] - DDLSemanticAnalyzer.analyzeShowRoles() should use HiveAuthorizationTaskFactory
- [HIVE-6612] - Misspelling "schemaTool completeted"
- [HIVE-6817] - Some hadoop2-only tests need diffs to be updated
- [HIVE-6822] - TestAvroSerdeUtils fails with -Phadoop-2
- [HIVE-6861] - more hadoop2 only golden files to fix
- [HIVE-5536] - Incorrect Operation Name is passed to hookcontext
Improvement
- [HIVE-5771] - Constant propagation optimizer for Hive
- [HIVE-9328] - Tests cannot move files due to change on HIVE-9325
- [HIVE-6024] - Load data local inpath unnecessarily creates a copy task
- [HIVE-10629] - Dropping table in an encrypted zone does not drop warehouse directory
- [HIVE-10630] - Renaming tables across encryption zones renames table even though the operation throws error
- [HIVE-5193] - Columnar Pushdown for RC/ORC File not happening in HCatLoader
- [HIVE-2828] - make timestamp accessible in the hbase KeyValue
- [HIVE-7737] - Hive logs full exception for table not found
- [HIVE-9788] - Make double quote optional in tsv/csv/dsv output
- [HIVE-5700] - enforce single date format for partition column storage
- [HIVE-9265] - Hive with encryption throws NPE to fs path without schema
- [HIVE-7858] - Parquet compression should be configurable via table property
- [HIVE-8904] - Hive should support multiple Key provider modes
- [HIVE-8864] - Fix permission inheritance with HDFS encryption
- [HIVE-8245] - Collect table read entities at same time as view read entities
- [HIVE-6430] - MapJoin hash table has large memory overhead
- [HIVE-6758] - Beeline doesn't work with -e option when started in background
- [HIVE-8471] - Cleanup TestSSL
- [HIVE-6657] - Add test coverage for Kerberos authentication implementation using Hadoop's miniKdc
- [HIVE-7432] - Remove deprecated Avro's Schema.parse usages
- [HIVE-6148] - Support arbitrary structs stored in HBase
- [HIVE-6147] - Support avro data stored in HBase columns
- [HIVE-3635] - allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
- [HIVE-6584] - Add HiveHBaseTableSnapshotInputFormat
- [HIVE-7211] - Throws exception if the name of conf var starts with "hive." does not exists in HiveConf
- [HIVE-6473] - Allow writing HFiles via HBaseStorageHandler table
- [HIVE-7048] - CompositeKeyHBaseFactory should not use FamilyFilter
- [HIVE-6411] - Support more generic way of using composite key for HBaseHandler
- [HIVE-6677] - HBaseSerDe needs to be refactored
- [HIVE-8130] - Support Date in Avro
- [HIVE-6561] - Beeline should accept -i option to Initializing a SQL file
- [HIVE-5871] - Use multiple-characters as field delimiter
- [HIVE-5160] - HS2 should support .hiverc
- [HIVE-5799] - session/operation timeout for hiveserver2
- [HIVE-7554] - Parquet Hive should resolve column names in case insensitive manner
- [HIVE-6438] - Sort query result for test, removing order by clause
- [HIVE-6938] - Add Support for Parquet Column Rename
- [HIVE-7445] - Improve LOGS for Hive when a query is not able to acquire locks
- [HIVE-7196] - Configure session by single open session call
- [HIVE-5946] - DDL authorization task factory should be better tested
- [HIVE-7140] - Bump default hive.metastore.client.socket.timeout to 5 minutes
New Feature
- [HIVE-7299] - Enable metadata only optimization on Tez
- [HIVE-7390] - Make quote character optional and configurable in BeeLine CSV/TSV output
- [HIVE-6806] - CREATE TABLE should support STORED AS AVRO
- [HIVE-6380] - Specify jars/files when creating permanent UDFs
- [HIVE-5823] - Support for DECIMAL primitive type in AvroSerDe
Task
- [HIVE-8465] - Fix some minor test fails on trunk
- [HIVE-8707] - Fix ordering differences due to Java 8 HashMap function
- [HIVE-8935] - Add debug logging around token stores
- [HIVE-8862] - Fix ordering diferences on TestParse tests due to Java8
- [HIVE-7108] - Cleanup HBaseStorageHandler
- [HIVE-8224] - Support Char, Varchar in AvroSerDe
- [HIVE-5976] - Decouple input formats from STORED as keywords
- [HIVE-7034] - Explain result of TezWork is not deterministic
- [HIVE-7095] - Fix test fails for both hadoop-1 and hadoop-2
- [HIVE-7735] - Implement Char, Varchar in ParquetSerDe
- [HIVE-6394] - Implement Timestmap in ParquetSerde
- [HIVE-6367] - Implement Decimal in ParquetSerde
Test
- [HIVE-4937] - Create description annotations for vectorized UDF
- [HIVE-7085] - TestOrcHCatPigStorer.testWriteDecimal tests are failing on trunk