CDH 5.3.2 Release Notes

Changes Not In Search engine server 4.4.0

Solr

Bug

• [SOLR-5961] - Solr gets crazy on /overseer/queue state change
• [SOLR-6640] - Replication can cause index corruption.
• [SOLR-5875] - QueryComponent.mergeIds() unmarshals all docs' sort field values once per doc instead of once per shard
• [SOLR-6969] - When opening an HDFSTransactionLog for append we must first attempt to recover it's lease to prevent data loss.
• [SOLR-5515] - NPE when getting stats on date field with empty result on solrcloud
• [SOLR-6776] - Data lost when use SoftCommit and TLog
• [SOLR-6738] - Admin UI - Escape Data on Plugins-View
• [SOLR-3774] - /admin/mbean returning duplicate search handlers with names that map to their classes?
• [SOLR-3852] - Admin UI - Cloud Tree ArrayIndexOutOfBoundsException if binary files anywhere in ZK tree
• [SOLR-5777] - JsonLoader: field values get out of order when fieldname key is repeated
• [SOLR-6698] - Solr is not consistent wrt ZkCredentialsProvider / ZkCredentialProvider
• [SOLR-6624] - Spelling mistakes in the Java source
• [SOLR-6609] - queryParams not respected for single-stream UpdateRequests
• [SOLR-6360] - Unnecessary Content-Charset header in HttpSolrServer
• [SOLR-5770] - All attempts to match a SolrCore with it's state in clusterstate.json should be done with the NodeName rather than the baseUrl.
• [SOLR-6426] - SolrZkClient clean can fail due to a race with children nodes.
• [SOLR-6264] - Distributed commit and optimize are executed serially across all replicas.
• [SOLR-6402] - OverseerCollectionProcessor should not exit for ZK ConnectionLoss
• [SOLR-6405] - ZooKeeper calls can easily not be retried enough on ConnectionLoss.
• [SOLR-5042] - MoreLikeThis doesn't return a score when mlt.count is set to 10
• [SOLR-6268] - HdfsUpdateLog has a race condition that can expose a closed HDFS FileSystem instance and should close it's FileSystem instance if either inherited close method is called.
• [SOLR-6136] - ConcurrentUpdateSolrServer includes a Spin Lock
• [SOLR-6159] - cancelElection fails on uninitialized ElectionContext
• [SOLR-6314] - Facet counts duplicated in the response if specified more than once on the request.
• [SOLR-5354] - Distributed sort is broken with CUSTOM FieldType
• [SOLR-6372] - HdfsDirectoryFactory should use supplied Configuration for communicating with secure kerberos
• [SOLR-6137] - Managed Schema / Schemaless and SolrCloud concurrency issues
• [SOLR-5593] - shard leader loss due to ZK session expiry
• [SOLR-5915] - Cannot set parserImpl=... with PreAnalyzedField
• [SOLR-6180] - Callers of ManagedIndexSchema mutators should hold the schemaUpdateLock
• [SOLR-6161] - OutOfMemoryError Not Thrown in sendError
• [SOLR-6056] - Zookeeper crash JVM stack OOM because of recover strategy
• [SOLR-6164] - Copy Fields Schema additions are not distributed to other nodes
• [SOLR-6145] - Concurrent Schema API field additions can result in endless loop
• [SOLR-5119] - Managed schema problems after adding fields via Schema Rest API
• [SOLR-5608] - Frequently reproducible failures in CollectionsAPIDistributedZkTest#testDistribSearch
• [SOLR-5993] - ZkController can warn about shard leader conflict after conflict resolved
• [SOLR-5796] - With many collections, leader re-election takes too long when a node dies or is rebooted, leading to some shards getting into a "conflicting" state about who is the leader.
• [SOLR-5727] - LBHttpSolrServer should only retry on Connection exceptions when sending updates.
• [SOLR-5811] - The Overseer will retry work items until success.
• [SOLR-5325] - zk connection loss causes overseer leader loss
• [SOLR-5615] - Deadlock while trying to recover after a ZK session expiry
• [SOLR-5693] - Running on HDFS does work correctly with NRT search.
• [SOLR-5657] - When a SolrCore starts on HDFS, it should gracefully handle HDFS being in safe mode.
• [SOLR-5667] - Performance problem when not using hdfs block cache.
• [SOLR-5666] - Using the hdfs write cache can result in appearance of corrupted index.
• [SOLR-5577] - indexing delay due to zookeeper election
• [SOLR-5636] - SolrRequestParsers does some xpath lookups on every request.
• [SOLR-5645] - A SolrCore reload via the CoreContainer will try and register in zk again with the new SolrCore.
• [SOLR-4992] - Solr queries don't propagate Java OutOfMemoryError back to the JVM
• [SOLR-5417] - The ChaosMonkey tests are not causing any disruption.
• [SOLR-5540] - HdfsLockFactory should create the lock parent directory if necessary like SimpleLockFactory does.
• [SOLR-4709] - dir lock error if reopening cores to fast?
• [SOLR-5568] - A SolrCore cannot decide to be the leader just because the cluster state says no other SolrCore's are active.
• [SOLR-5380] - Using cloudSolrServer.setDefaultCollection(collectionId) does not work as intended for an alias spanning more than 1 collection.
• [SOLR-5243] - killing a shard in one collection can result in leader election in a different collection
• [SOLR-5240] - SolrCloud node doesn't (quickly) come all the way back
• [SOLR-5150] - HdfsIndexInput may not fully read requested bytes.
• [SOLR-5187] - We are shipping with the wrong license for NetCDF.
• [SOLR-5133] - HdfsUpdateLog can fail to close a FileSystem instance if init is called more than once.
• [SOLR-5135] - Deleting a collection should be extra aggressive in the face of failures.
• [SOLR-5087] - CoreAdminHandler.handleMergeAction generating NullPointerException
• [SOLR-3633] - web UI reports an error if CoreAdminHandler says there are no SolrCores
• [SOLR-5018] - The Overseer should avoid publishing the state for collections that do not exist under the /collections zk node.

Improvement

• [SOLR-6919] - Log REST info before executing
• [SOLR-6586] - JmxMonitoredMap#getAttribute is not very efficient.
• [SOLR-6313] - Improve SolrCloud cloud-dev scripts.
• [SOLR-5570] - Generalize solrcloud-start.sh solrcloud dev script.
• [SOLR-5436] - Eliminate the 1500ms wait in overseer loop
• [SOLR-6565] - SolrRequest support for query params
• [SOLR-6543] - Give HttpSolrServer the ability to send PUT requests
• [SOLR-6393] - Improve transaction log replay speed on HDFS.
• [SOLR-2894] - Implement distributed pivot faceting
• [SOLR-6252] - A couple of small improvements to UnInvertedField class.
• [SOLR-3862] - add "remove" as update option for atomically removing a value from a multivalued field
• [SOLR-5395] - allow some update processors to run on forwarded update
• [SOLR-5865] - Provide a MiniSolrCloudCluster to enable easier testing
• [SOLR-5714] - You should be able to use one pool of memory for multiple collection's HDFS block caches.
• [SOLR-3530] - better error messages / Content-Type validation in solrJ
• [SOLR-5700] - Improve error handling of remote queries (proxied requests)
• [SOLR-5156] - Provide a way to move the contents of a file to ZooKeeper with ZkCLI
• [SOLR-5581] - Give ZkCLI the ability to get files
• [SOLR-5576] - ZkController.java registerAllCoresAsDown multiple cores logic
• [SOLR-5548] - Give DistributedSearchTestCase / JettySolrRunner the ability to specify extra filters
• [SOLR-5134] - Have HdfsIndexOutput extend BufferedIndexOutput

New Feature

• [SOLR-3076] - Solr(Cloud) should support block joins
• [SOLR-4580] - Support for protecting content in ZK
• [SOLR-5795] - Option to periodically delete docs based on an expiration field -- or ttl specified when indexed.
• [SOLR-5466] - Add List Collections functionality to Collections API
• [SOLR-5656] - Add autoAddReplicas feature for shared file systems.
• [SOLR-5463] - Provide cursor/token based "searchAfter" support that works with arbitrary sorting (ie: "deep paging")
• [SOLR-5556] - Allow class of CollectionsHandler and InfoHandler to be specified in solr.xml
• [SOLR-4943] - Add a new info admin handler.

Lucene

Bug

• [LUCENE-5672] - Addindexes does not call maybeMerge
• [LUCENE-5574] - NRT Reader close can wipe index it doesn't own
• [LUCENE-5347] - Update to forbidden-apis 1.4

Improvement

• [LUCENE-5209] - Allow the license checker to optionally avoid check sum comparisons on SNAPSHOT dependencies.
• [LUCENE-4661] - Reduce default maxMerge/ThreadCount for ConcurrentMergeScheduler

New Feature

• [LUCENE-5360] - Add support for developing in netbeans IDE