Search commit ingestion and cache-backed reads

SearchEngine.onCommitNotification updates latest uploaded BCC information, starts prefetch if enabled, then queues the newest notification for serialized processing.

searchDirectory.updateLatestUploadedBcc(...);
searchDirectory.updateLatestCommitInfo(ccTermAndGen, notification.nodeId());
SubscribableListener
    .newForked(l -> maybePreFetchLatestCommit(notification, l))
    .andThen(...);

The prefetcher skips idle shards, starts at the current term to avoid downloading long history, computes pending BCC ranges, and fills shared cache regions. It can prefetch uploaded BCCs from object store or small non-uploaded ranges from indexing nodes.

Map<BlobFile, ByteRange> bccRangesToPrefetch =
    getPendingRangesToPrefetch(...);
cacheService.fetchRange(cacheKey, region, adjustedRangeToPrefetch, ...);

Commit notifications are collapsed to the latest term/generation. The search directory is updated with the new commit metadata, then Lucene reader refresh runs while preserving retained file sets for open readers.

final boolean commitUpdated =
    searchDirectory.updateCommit(latestCommit, blobFileRangesMap);
if (commitUpdated) {
    updateInternalState(latestCommit, current);
}

When Lucene opens a file, BlobStoreCacheDirectory resolves the file to BlobFileRanges, builds a CacheFileReader, and returns BlobCacheIndexInput. Cache misses are filled by a component-specific CacheBlobReader.

var reader = new CacheFileReader(
    getCacheFile(blobFileRanges),
    getCacheBlobReader(name, blobFile),
    blobFileRanges,
    ...);
return new BlobCacheIndexInput(...);