CDH 5.11.1 Release Notes
The following lists all Search engine server Jiras included in CDH 5.11.1
that are not included in the Search engine server base version 4.10.3. The
file lists all changes included in CDH 5.11.1. The patch for each
change can be found in the cloudera/patches directory in the release tarball.
Changes Not In Search engine server 4.10.3
- [SOLR-9836] - Add more graceful recovery steps when failing to create SolrCore
- [SOLR-10360] - Solr HDFS snapshot export fails due to FileNotFoundException error
- [SOLR-9389] - HDFS Transaction logs stay open for writes which leaks Xceivers
- [SOLR-4668] - Request for core status during core reload causes 500 error due to IndexWriter AlreadyClosedException
- [SOLR-9859] - replication.properties cannot be updated after being written and neither replication.properties or index.properties are durable in the face of a crash
- [SOLR-9284] - The HDFS BlockDirectoryCache should not let it's keysToRelease or names maps grow indefinitely.
- [SOLR-10031] - ReplicationHandler path traversal vulnerability
- [SOLR-9330] - Race condition between core reload and statistics request
- [SOLR-9524] - SolrIndexSearcher.getIndexFingerprint uses dubious sunchronization
- [SOLR-7866] - getMaxVersionFromIndex is throwing an NPE if some segments are not current
- [SOLR-9441] - Solr collection backup on HDFS can only be manipulated by the Solr process owner
- [SOLR-7374] - Backup/Restore should provide a param for specifying the directory implementation it should use
- [SOLR-7280] - Load cores in sorted order and tweak coreLoadThread counts to improve cluster stability on restarts
- [SOLR-8449] - Multiple restores on the same core does not work
- [SOLR-9236] - AutoAddReplicas feature with one replica loses some documents not committed during failover
- [SOLR-7294] - Migrate API fails with: Invalid status request: notfoundretried 6times
- [SOLR-9091] - Solr index restore silently copies the corrupt segments in the backup
- [SOLR-6295] - AssertionError in ToChildBlockJoinScorer.advance
- [SOLR-8451] - We should not call method.abort in HttpSolrClient and HttpSolrCall#remoteQuery should not close streams.
- [SOLR-7178] - OverseerAutoReplicaFailoverThread compares Integer objects using ==
- [SOLR-8497] - Merge index does not mark the Directories it creates as 'done' and they are retained in the Directory cache.
- [SOLR-8771] - Multi-threaded core shutdown creates executor per core
- [SOLR-8933] - SolrDispatchFilter::consumeInput logs "Stream Closed" IOException
- [SOLR-6631] - DistributedQueue spinning on calling zookeeper getChildren()
- [SOLR-8892] - Allow SolrInfoMBeans to return different statistics for /jmx vs web ui calls
- [SOLR-7493] - Requests aren't distributed evenly if the collection isn't present locally
- [SOLR-8857] - HdfsUpdateLog does not use configured or new default number of version buckets and is hard coded to 256.
- [SOLR-8303] - CustomBufferedIndexInput should include resource description when throwing EOFException
- [SOLR-8578] - Successful or not, requests are not always fully consumed by Solrj clients and we count on HttpClient or the JVM.
- [SOLR-7625] - Version bucket seed not updated after new index is installed on a replica
- [SOLR-7587] - TestSpellCheckResponse stalled and never timed out -- possible VersionBucket bug? (5.2 branch)
- [SOLR-8450] - Our HttpClient retry policy is too permissive.
- [SOLR-7281] - Add an overseer action to publish an entire node as 'down'
- [SOLR-8215] - SolrCloud can select a core not in active state for querying
- [SOLR-8633] - DistributedUpdateProcess processCommit/deleteByQuery call finish on DUP and SolrCmdDistributor, which violates the lifecycle and can cause bugs.
- [SOLR-8683] - Always consume the full request on the server, not just in the case of an error.
- [SOLR-8575] - Fix HDFSLogReader replay status numbers and a performance bug where we can reopen FSDataInputStream too often.
- [SOLR-8407] - We log an error during normal recovery PeerSync now.
- [SOLR-8453] - Jetty update from 9.2 to 9.3 causes the server to reset formerly legitimate client connections.
- [SOLR-7989] - After a new leader is elected it, it should ensure it's state is ACTIVE if it has already registered with ZK.
- [SOLR-8372] - Canceled recovery can lead to data loss
- [SOLR-8367] - The new LIR 'all replicas participate' failsafe code needs to be improved.
- [SOLR-8203] - Stop processing updates more quickly on shutdown
- [SOLR-6406] - ConcurrentUpdateSolrServer hang in blockUntilFinished.
- [SOLR-6443] - TestManagedResourceStorage fails on Jenkins with SolrCore.getOpenCount()==2
- [SOLR-8288] - DistributedUpdateProcessor#doFinish should explicitly check and ensure it does not try to put itself into LIR.
- [SOLR-8075] - Leader Initiated Recovery should not stop a leader that participated in an election with all of it's replicas from becoming a valid leader.
- [SOLR-8223] - Take care not to accidentally swallow OOMErrors
- [SOLR-8152] - Overseer Task Processor/Queue can miss responses, leading to timeouts
- [SOLR-7967] - AddSchemaFieldsUpdateProcessorFactory does not check if the ConfigSet is immutable
- [SOLR-8095] - Allow disabling HDFS Locality Metrics
- [SOLR-8085] - Fix a variety of issues that can result in replicas getting out of sync.
- [SOLR-8094] - HdfsUpdateLog should not replay buffered documents as a replacement to dropping them.
- [SOLR-8077] - Replication can still cause index corruption.
- [SOLR-8069] - Ensure that only the valid ZooKeeper registered leader can put a replica into Leader Initiated Recovery.
- [SOLR-7844] - Zookeeper session expiry during shard leader election can cause multiple leaders.
- [SOLR-7109] - Indexing threads stuck during network partition can put leader into down state
- [SOLR-7950] - Invalid auth scheme configuration of Http client when using Kerberos (SPNEGO)
- [SOLR-7956] - There are interrupts on shutdown in places that can cause ChannelAlreadyClosed exceptions which prevents proper closing of transaction logs.
- [SOLR-7966] - Solr Admin pages should set X-Frame-Options to DENY
- [SOLR-7855] - OverseerCollectionProcessor: separate general task management from collection message handling
- [SOLR-7503] - Recovery after ZK session expiration happens in a single thread for all cores in a node
- [SOLR-7478] - UpdateLog#close shutdown it's executor with interrupts before running close, preventing a clean close.
- [SOLR-6583] - Resuming connection with ZooKeeper causes log replay
- [SOLR-7092] - Stop the HDFS lease recovery retries on HdfsTransactionLog on close and try to avoid lease recovery on closed files.
- [SOLR-7370] - FSHDFSUtils#recoverFileLease tries to recover the lease every one second after the first four second wait.
- [SOLR-7134] - Replication can still cause index corruption.
- [SOLR-6766] - Switch o.a.s.store.blockcache.Metrics to use JMX
- [SOLR-6752] - Buffer Cache allocate/lost metrics should be exposed
- [SOLR-7113] - Multiple calls to UpdateLog#init is not thread safe with respect to the HDFS FileSystem client object usage.
- [SOLR-6839] - Direct routing with CloudSolrServer will ignore the Overwrite document option.
- [SOLR-5961] - Solr gets crazy on /overseer/queue state change
- [SOLR-5596] - OverseerTest.testOverseerFailure - leader node already exists.
- [SOLR-7101] - JmxMonitoredMap can throw an exception in clear when queryNames fails.
- [SOLR-6640] - Replication can cause index corruption.
- [SOLR-6920] - During replication use checksums to verify if files are the same
- [SOLR-7066] - autoAddReplicas feature has bug when selecting replacement nodes.
- [SOLR-6698] - Solr is not consistent wrt ZkCredentialsProvider / ZkCredentialProvider
- [SOLR-6969] - When opening an HDFSTransactionLog for append we must first attempt to recover it's lease to prevent data loss.
- [SOLR-6987] - SSL support for MiniSolrCloudCluster
- [SOLR-3774] - /admin/mbean returning duplicate search handlers with names that map to their classes?
- [SOLR-6609] - queryParams not respected for single-stream UpdateRequests
- [SOLR-6426] - SolrZkClient clean can fail due to a race with children nodes.
- [SOLR-6137] - Managed Schema / Schemaless and SolrCloud concurrency issues
- [SOLR-6056] - Zookeeper crash JVM stack OOM because of recover strategy
- [SOLR-4992] - Solr queries don't propagate Java OutOfMemoryError back to the JVM
- [SOLR-5187] - We are shipping with the wrong license for NetCDF.
- [SOLR-10430] - Add ls command to ZkCLI for listing only sub-directories
- [SOLR-9848] - Lower solr.cloud.wait-for-updates-with-stale-state-pause back down from 7 seconds.
- [SOLR-9819] - Upgrade commons-fileupload to 1.3.2
- [SOLR-10116] - BlockCache test and documentation improvement
- [SOLR-9941] - log replay redundently (pre-)applies DBQs as if they were out of order
- [SOLR-9899] - StandardDirectoryFactory should use optimizations for all FilterDirectorys not just NRTCachingDirectory.
- [SOLR-9506] - cache IndexFingerprint for each segment
- [SOLR-9242] - Collection level backup/restore should provide a param for specifying the repository implementation it should use
- [SOLR-5922] - Add Properties and other parameters to SolrJ Collection Admin Request calls
- [SOLR-6637] - Solr should have a way to restore a core
- [SOLR-9053] - Upgrade fileupload-commons to 1.3.1
- [SOLR-8551] - Make collection deletion more robust.
- [SOLR-6879] - Have an option to disable autoAddReplicas temporarily for all collections
- [SOLR-9047] - zkcli should allow alternative locations for log4j configuration
- [SOLR-8855] - The HDFS BlockDirectory should not clean up it's cache on shutdown.
- [SOLR-8856] - Do not cache merge or 'read once' contexts in the hdfs block cache.
- [SOLR-8869] - Optionally disable printing field cache entries in SolrFieldCacheMBean
- [SOLR-8720] - ZkController#publishAndWaitForDownStates should use #publishNodeAsDown.
- [SOLR-8351] - Improve HdfsDirectory and HdfsLock toString representation
- [SOLR-8691] - cache index fingerprint
- [SOLR-8690] - make fingerprinting optional
- [SOLR-8586] - Implement hash over all documents to check for shard synchronization
- [SOLR-8371] - Try and prevent too many recovery requests from stacking up and clean up some faulty logic.
- [SOLR-8615] - Just like creating cores, we should use multiple threads when closing cores.
- [SOLR-8415] - Provide command to switch between non/secure mode in ZK
- [SOLR-8535] - Support forcing define-lucene-javadoc-url to be local
- [SOLR-8353] - Support regex for skipping license checksums
- [SOLR-7049] - Move work done by the LIST Collections API call to the Collections Handler
- [SOLR-7955] - Auto create .system collection on first request if it does not exist
- [SOLR-5884] - When recovery is cancelled, any call to the leader to wait to see the replica in the right state for recovery should be aborted.
- [SOLR-7135] - Allow the server build.xml 'sync-hack' target to by skipped by specifying a system property.
- [SOLR-7458] - Expose HDFS Block Locality Metrics
- [SOLR-7457] - Make DirectoryFactory publishing MBeanInfo extensible
- [SOLR-7552] - Support using ZkCredentialsProvider/ZkACLProvider in custom filter
- [SOLR-7437] - Make HDFS transaction log replication factor configurable.
- [SOLR-7147] - Introduce new TrackingShardHandlerFactory for monitoring what requests are sent to shards during tests
- [SOLR-7141] - RecoveryStrategy: Raise time that we wait for any updates from the leader before they saw the recovery state to have finished.
- [SOLR-6915] - SaslZkACLProvider and Kerberos Test Using MiniKdc
- [SOLR-6896] - SolrJettyRunner blocks for 5 seconds on shutdown
- [SOLR-6919] - Log REST info before executing
- [SOLR-6586] - JmxMonitoredMap#getAttribute is not very efficient.
- [SOLR-6565] - SolrRequest support for query params
- [SOLR-6543] - Give HttpSolrServer the ability to send PUT requests
- [SOLR-6393] - Improve transaction log replay speed on HDFS.
- [SOLR-5700] - Improve error handling of remote queries (proxied requests)
- [SOLR-5714] - You should be able to use one pool of memory for multiple collection's HDFS block caches.
- [SOLR-9326] - Ability to create/delete/list snapshots for a solr collection
- [SOLR-9269] - Ability to create/delete/list snapshots for a solr core
- [SOLR-5750] - Backup/Restore API for SolrCloud
- [SOLR-7766] - support creation of a coreless collection
- [SOLR-6761] - Ability to ignore commit and optimize requests from clients when running in SolrCloud mode.
- [SOLR-9038] - Ability to create/delete/list snapshots for a solr collection
- [SOLR-7928] - Improve CheckIndex to work against HdfsDirectory
- [SOLR-7789] - Introduce a ConfigSet management API
- [SOLR-7742] - Support for Immutable ConfigSets
- [SOLR-7176] - allow zkcli to modify JSON
- [SOLR-4580] - Support for protecting content in ZK
- [SOLR-5556] - Allow class of CollectionsHandler and InfoHandler to be specified in solr.xml
- [SOLR-9055] - Make collection backup/restore extensible
- [SOLR-7332] - Seed version buckets with max version from index
- [SOLR-6820] - The sync on the VersionInfo bucket in DistributedUpdateProcesser#addDocument appears to be a large bottleneck when using replication.
- [SOLR-10338] - Configure SecureRandom non blocking for tests.
- [SOLR-5776] - Look at speeding up using SSL with tests.
- [SOLR-10119] - TestReplicationHandler has always been too good at finding too many annoying bugs.
- [SOLR-8121] - It looks like ChaosMonkeySafeLeader test can fail with replica inconsistency because waitForThingsToLevelOut can pass while state is still changing.
- [SOLR-8046] - HdfsCollectionsAPIDistributedZkTest checks that no transaction logs failed to be opened during the test but does not isolate this to the test and could fail due to other tests.
- [SOLR-7999] - SolrRequetParserTest#testStreamURL started failing.
- [SENTRY-989] - RealTimeGet with explicit ids can bypass document level authorization
- [LUCENE-5965] - Make CorruptIndexException require 'resource' like TooOld/TooNew do
- [LUCENE-7592] - EOFException while opening index should be rethrown as CorruptIndexException
- [LUCENE-5833] - Suggestor Version 2 doesn't support multiValued fields
- [LUCENE-6719] - NumericUtils.getMinLong and NumericUtils.getMaxLong have undefined behavior when no docs have value - throw NPE
- [LUCENE-7710] - BlockPackedReader to throw better exception
- [LUCENE-5925] - Use rename instead of segments_N fallback / segments.gen etc
- [LUCENE-5895] - Add per-segment and per-commit id to help replication
- [LUCENE-7564] - AnalyzingInfixSuggester should close its IndexWriter by default at the end of build()
- [LUCENE-5889] - AnalyzingInfixSuggester should expose commit()
- [LUCENE-7115] - Speed up FieldCache.CacheEntry toString by setting initial StringBuilder capacity
- [LUCENE-6277] - Allow Ivy resolutionCacheDir to be overriden by system property.
- [LUCENE-4661] - Reduce default maxMerge/ThreadCount for ConcurrentMergeScheduler
- [LUCENE-7543] - Make changes-to-html target an offline operation
- [LUCENE-6210] - Unit tests failures in TestLucene40DocValuesFormat/TestDocValuesFormat