CDH 5.14.4 Release Notes

Changes Not In Search engine server 4.10.3

Solr

Bug

• [SOLR-12293] - Updates need to use their own connection pool to maintain connection reuse and prevent spurious recoveries.
• [SOLR-10904] - Unnecessary waiting during failover in case of failed core creation
• [SOLR-9527] - Solr RESTORE api doesn't distribute the replicas uniformly
• [SOLR-9153] - Update beanutils version to 1.9.2
• [SOLR-9507] - CoreContainer threads aren't setting MDC values
• [SOLR-8836] - /update should return BAD REQUEST when invalid JSON provided
• [SOLR-10360] - Solr HDFS snapshot export fails due to FileNotFoundException error
• [SOLR-9836] - Add more graceful recovery steps when failing to create SolrCore
• [SOLR-9389] - HDFS Transaction logs stay open for writes which leaks Xceivers
• [SOLR-4668] - Request for core status during core reload causes 500 error due to IndexWriter AlreadyClosedException
• [SOLR-9859] - replication.properties cannot be updated after being written and neither replication.properties or index.properties are durable in the face of a crash
• [SOLR-9284] - The HDFS BlockDirectoryCache should not let it's keysToRelease or names maps grow indefinitely.
• [SOLR-10031] - ReplicationHandler path traversal vulnerability
• [SOLR-9330] - Race condition between core reload and statistics request
• [SOLR-9524] - SolrIndexSearcher.getIndexFingerprint uses dubious sunchronization
• [SOLR-7866] - getMaxVersionFromIndex is throwing an NPE if some segments are not current
• [SOLR-9441] - Solr collection backup on HDFS can only be manipulated by the Solr process owner
• [SOLR-7374] - Backup/Restore should provide a param for specifying the directory implementation it should use
• [SOLR-7280] - Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts
• [SOLR-8449] - Multiple restores on the same core does not work
• [SOLR-9236] - AutoAddReplicas feature with one replica loses some documents not committed during failover
• [SOLR-7294] - Migrate API fails with: Invalid status request: notfoundretried 6times
• [SOLR-9091] - Solr index restore silently copies the corrupt segments in the backup
• [SOLR-6295] - AssertionError in ToChildBlockJoinScorer.advance
• [SOLR-8451] - We should not call method.abort in HttpSolrClient and HttpSolrCall#remoteQuery should not close streams.
• [SOLR-7178] - OverseerAutoReplicaFailoverThread compares Integer objects using ==
• [SOLR-8497] - Merge index does not mark the Directories it creates as 'done' and they are retained in the Directory cache.
• [SOLR-8771] - Multi-threaded core shutdown creates executor per core
• [SOLR-8933] - SolrDispatchFilter::consumeInput logs "Stream Closed" IOException
• [SOLR-6631] - DistributedQueue spinning on calling zookeeper getChildren()
• [SOLR-8892] - Allow SolrInfoMBeans to return different statistics for /jmx vs web ui calls
• [SOLR-7493] - Requests aren't distributed evenly if the collection isn't present locally
• [SOLR-8857] - HdfsUpdateLog does not use configured or new default number of version buckets and is hard coded to 256.
• [SOLR-8303] - CustomBufferedIndexInput should include resource description when throwing EOFException
• [SOLR-8578] - Successful or not, requests are not always fully consumed by Solrj clients and we count on HttpClient or the JVM.
• [SOLR-7625] - Version bucket seed not updated after new index is installed on a replica
• [SOLR-7587] - TestSpellCheckResponse stalled and never timed out -- possible VersionBucket bug? (5.2 branch)
• [SOLR-8450] - Our HttpClient retry policy is too permissive.
• [SOLR-7281] - Add an overseer action to publish an entire node as 'down'
• [SOLR-8215] - SolrCloud can select a core not in active state for querying
• [SOLR-8633] - DistributedUpdateProcess processCommit/deleteByQuery call finish on DUP and SolrCmdDistributor, which violates the lifecycle and can cause bugs.
• [SOLR-8683] - Always consume the full request on the server, not just in the case of an error.
• [SOLR-8575] - Fix HDFSLogReader replay status numbers and a performance bug where we can reopen FSDataInputStream too often.
• [SOLR-8407] - We log an error during normal recovery PeerSync now.
• [SOLR-8453] - Jetty update from 9.2 to 9.3 causes the server to reset formerly legitimate client connections.
• [SOLR-7989] - After a new leader is elected it, it should ensure it's state is ACTIVE if it has already registered with ZK.
• [SOLR-8372] - Canceled recovery can lead to data loss
• [SOLR-8367] - The new LIR 'all replicas participate' failsafe code needs to be improved.
• [SOLR-8203] - Stop processing updates more quickly on shutdown
• [SOLR-6406] - ConcurrentUpdateSolrServer hang in blockUntilFinished.
• [SOLR-6443] - TestManagedResourceStorage fails on Jenkins with SolrCore.getOpenCount()==2
• [SOLR-8288] - DistributedUpdateProcessor#doFinish should explicitly check and ensure it does not try to put itself into LIR.
• [SOLR-8075] - Leader Initiated Recovery should not stop a leader that participated in an election with all of it's replicas from becoming a valid leader.
• [SOLR-8223] - Take care not to accidentally swallow OOMErrors
• [SOLR-8152] - Overseer Task Processor/Queue can miss responses, leading to timeouts
• [SOLR-7967] - AddSchemaFieldsUpdateProcessorFactory does not check if the ConfigSet is immutable
• [SOLR-8095] - Allow disabling HDFS Locality Metrics
• [SOLR-8085] - Fix a variety of issues that can result in replicas getting out of sync.
• [SOLR-8094] - HdfsUpdateLog should not replay buffered documents as a replacement to dropping them.
• [SOLR-8077] - Replication can still cause index corruption.
• [SOLR-8069] - Ensure that only the valid ZooKeeper registered leader can put a replica into Leader Initiated Recovery.
• [SOLR-7844] - Zookeeper session expiry during shard leader election can cause multiple leaders.
• [SOLR-7109] - Indexing threads stuck during network partition can put leader into down state
• [SOLR-7950] - Invalid auth scheme configuration of Http client when using Kerberos (SPNEGO)
• [SOLR-7956] - There are interrupts on shutdown in places that can cause ChannelAlreadyClosed exceptions which prevents proper closing of transaction logs.
• [SOLR-7966] - Solr Admin pages should set X-Frame-Options to DENY
• [SOLR-7855] - OverseerCollectionProcessor: separate general task management from collection message handling
• [SOLR-7503] - Recovery after ZK session expiration happens in a single thread for all cores in a node
• [SOLR-7478] - UpdateLog#close shutdown it's executor with interrupts before running close, preventing a clean close.
• [SOLR-6583] - Resuming connection with ZooKeeper causes log replay
• [SOLR-7092] - Stop the HDFS lease recovery retries on HdfsTransactionLog on close and try to avoid lease recovery on closed files.
• [SOLR-7370] - FSHDFSUtils#recoverFileLease tries to recover the lease every one second after the first four second wait.
• [SOLR-7134] - Replication can still cause index corruption.
• [SOLR-6766] - Switch o.a.s.store.blockcache.Metrics to use JMX
• [SOLR-6752] - Buffer Cache allocate/lost metrics should be exposed
• [SOLR-7113] - Multiple calls to UpdateLog#init is not thread safe with respect to the HDFS FileSystem client object usage.
• [SOLR-6839] - Direct routing with CloudSolrServer will ignore the Overwrite document option.
• [SOLR-5961] - Solr gets crazy on /overseer/queue state change
• [SOLR-5596] - OverseerTest.testOverseerFailure - leader node already exists.
• [SOLR-7101] - JmxMonitoredMap can throw an exception in clear when queryNames fails.
• [SOLR-6640] - Replication can cause index corruption.
• [SOLR-6920] - During replication use checksums to verify if files are the same
• [SOLR-7066] - autoAddReplicas feature has bug when selecting replacement nodes.
• [SOLR-6698] - Solr is not consistent wrt ZkCredentialsProvider / ZkCredentialProvider
• [SOLR-6969] - When opening an HDFSTransactionLog for append we must first attempt to recover it's lease to prevent data loss.
• [SOLR-6987] - SSL support for MiniSolrCloudCluster
• [SOLR-3774] - /admin/mbean returning duplicate search handlers with names that map to their classes?
• [SOLR-6609] - queryParams not respected for single-stream UpdateRequests
• [SOLR-6426] - SolrZkClient clean can fail due to a race with children nodes.
• [SOLR-6137] - Managed Schema / Schemaless and SolrCloud concurrency issues
• [SOLR-6056] - Zookeeper crash JVM stack OOM because of recover strategy
• [SOLR-4992] - Solr queries don't propagate Java OutOfMemoryError back to the JVM
• [SOLR-5187] - We are shipping with the wrong license for NetCDF.

Improvement

• [SOLR-7391] - Use a time based expiration cache for one off hdfs FileSystem instances.
• [SOLR-10076] - Hiding keystore and truststore passwords from /admin/info/* outputs
• [SOLR-10430] - Add ls command to ZkCLI for listing only sub-directories
• [SOLR-9848] - Lower solr.cloud.wait-for-updates-with-stale-state-pause back down from 7 seconds.
• [SOLR-9819] - Upgrade commons-fileupload to 1.3.2
• [SOLR-10116] - BlockCache test and documentation improvement
• [SOLR-9941] - log replay redundently (pre-)applies DBQs as if they were out of order
• [SOLR-9899] - StandardDirectoryFactory should use optimizations for all FilterDirectorys not just NRTCachingDirectory.
• [SOLR-9506] - cache IndexFingerprint for each segment
• [SOLR-9242] - Collection level backup/restore should provide a param for specifying the repository implementation it should use
• [SOLR-5922] - Add Properties and other parameters to SolrJ Collection Admin Request calls
• [SOLR-6637] - Solr should have a way to restore a core
• [SOLR-9053] - Upgrade fileupload-commons to 1.3.1
• [SOLR-8551] - Make collection deletion more robust.
• [SOLR-6879] - Have an option to disable autoAddReplicas temporarily for all collections
• [SOLR-9047] - zkcli should allow alternative locations for log4j configuration
• [SOLR-8855] - The HDFS BlockDirectory should not clean up it's cache on shutdown.
• [SOLR-8856] - Do not cache merge or 'read once' contexts in the hdfs block cache.
• [SOLR-8869] - Optionally disable printing field cache entries in SolrFieldCacheMBean
• [SOLR-8720] - ZkController#publishAndWaitForDownStates should use #publishNodeAsDown.
• [SOLR-8351] - Improve HdfsDirectory and HdfsLock toString representation
• [SOLR-8691] - cache index fingerprint
• [SOLR-8690] - make fingerprinting optional
• [SOLR-8586] - Implement hash over all documents to check for shard synchronization
• [SOLR-8371] - Try and prevent too many recovery requests from stacking up and clean up some faulty logic.
• [SOLR-8615] - Just like creating cores, we should use multiple threads when closing cores.
• [SOLR-8415] - Provide command to switch between non/secure mode in ZK
• [SOLR-8535] - Support forcing define-lucene-javadoc-url to be local
• [SOLR-8353] - Support regex for skipping license checksums
• [SOLR-7049] - Move work done by the LIST Collections API call to the Collections Handler
• [SOLR-7955] - Auto create .system collection on first request if it does not exist
• [SOLR-5884] - When recovery is cancelled, any call to the leader to wait to see the replica in the right state for recovery should be aborted.
• [SOLR-7135] - Allow the server build.xml 'sync-hack' target to by skipped by specifying a system property.
• [SOLR-7458] - Expose HDFS Block Locality Metrics
• [SOLR-7457] - Make DirectoryFactory publishing MBeanInfo extensible
• [SOLR-7552] - Support using ZkCredentialsProvider/ZkACLProvider in custom filter
• [SOLR-7437] - Make HDFS transaction log replication factor configurable.
• [SOLR-7147] - Introduce new TrackingShardHandlerFactory for monitoring what requests are sent to shards during tests
• [SOLR-7141] - RecoveryStrategy: Raise time that we wait for any updates from the leader before they saw the recovery state to have finished.
• [SOLR-6915] - SaslZkACLProvider and Kerberos Test Using MiniKdc
• [SOLR-6896] - SolrJettyRunner blocks for 5 seconds on shutdown
• [SOLR-6919] - Log REST info before executing
• [SOLR-6586] - JmxMonitoredMap#getAttribute is not very efficient.
• [SOLR-6565] - SolrRequest support for query params
• [SOLR-6543] - Give HttpSolrServer the ability to send PUT requests
• [SOLR-6393] - Improve transaction log replay speed on HDFS.
• [SOLR-5700] - Improve error handling of remote queries (proxied requests)
• [SOLR-5714] - You should be able to use one pool of memory for multiple collection's HDFS block caches.

New Feature

• [SOLR-9326] - Ability to create/delete/list snapshots for a solr collection
• [SOLR-9269] - Ability to create/delete/list snapshots for a solr core
• [SOLR-5750] - Backup/Restore API for SolrCloud
• [SOLR-7766] - support creation of a coreless collection
• [SOLR-6761] - Ability to ignore commit and optimize requests from clients when running in SolrCloud mode.
• [SOLR-9038] - Ability to create/delete/list snapshots for a solr collection
• [SOLR-7928] - Improve CheckIndex to work against HdfsDirectory
• [SOLR-7789] - Introduce a ConfigSet management API
• [SOLR-7742] - Support for Immutable ConfigSets
• [SOLR-7176] - allow zkcli to modify JSON
• [SOLR-4580] - Support for protecting content in ZK
• [SOLR-5556] - Allow class of CollectionsHandler and InfoHandler to be specified in solr.xml

Task

• [SOLR-9055] - Make collection backup/restore extensible
• [SOLR-7332] - Seed version buckets with max version from index
• [SOLR-6820] - The sync on the VersionInfo bucket in DistributedUpdateProcesser#addDocument appears to be a large bottleneck when using replication.

Test

• [SOLR-10338] - Configure SecureRandom non blocking for tests.
• [SOLR-5776] - Look at speeding up using SSL with tests.
• [SOLR-10119] - TestReplicationHandler has always been too good at finding too many annoying bugs.
• [SOLR-8121] - It looks like ChaosMonkeySafeLeader test can fail with replica inconsistency because waitForThingsToLevelOut can pass while state is still changing.
• [SOLR-8046] - HdfsCollectionsAPIDistributedZkTest checks that no transaction logs failed to be opened during the test but does not isolate this to the test and could fail due to other tests.
• [SOLR-7999] - SolrRequetParserTest#testStreamURL started failing.

Sentry

Bug

• [SENTRY-989] - RealTimeGet with explicit ids can bypass document level authorization

Improvement

• [SENTRY-574] - Add Sentry solr handler

Lucene

Bug

• [LUCENE-5965] - Make CorruptIndexException require 'resource' like TooOld/TooNew do
• [LUCENE-7592] - EOFException while opening index should be rethrown as CorruptIndexException
• [LUCENE-5833] - Suggestor Version 2 doesn't support multiValued fields
• [LUCENE-6719] - NumericUtils.getMinLong and NumericUtils.getMaxLong have undefined behavior when no docs have value - throw NPE

Improvement

• [LUCENE-7710] - BlockPackedReader to throw better exception
• [LUCENE-5925] - Use rename instead of segments_N fallback / segments.gen etc
• [LUCENE-5895] - Add per-segment and per-commit id to help replication
• [LUCENE-7564] - AnalyzingInfixSuggester should close its IndexWriter by default at the end of build()
• [LUCENE-5889] - AnalyzingInfixSuggester should expose commit()
• [LUCENE-7115] - Speed up FieldCache.CacheEntry toString by setting initial StringBuilder capacity
• [LUCENE-6277] - Allow Ivy resolutionCacheDir to be overriden by system property.
• [LUCENE-4661] - Reduce default maxMerge/ThreadCount for ConcurrentMergeScheduler

Task

• [LUCENE-7543] - Make changes-to-html target an offline operation

Test

• [LUCENE-6210] - Unit tests failures in TestLucene40DocValuesFormat/TestDocValuesFormat