Elasticsearch — Architecture

3. Main runtime architecture

Runtime components

Component	Class	Responsibility
Node shell	`Node`	Lifecycle of all services; holds Guice `Injector`
Cluster service	`ClusterService`	Facade over master + applier threads
Master path	`MasterService`	Serializes cluster-state update tasks on elected master
Follower path	`ClusterApplierService`	Applies published cluster state locally
Election & publish	`Coordinator`	Leader election, state publication, commit
Transport	`TransportService`	Inter-node RPC; action routing
HTTP	`HttpServerTransport`	REST listener (Netty implementation in module)
Indices	`IndicesService`	Creates/removes `IndexService` / `IndexShard`
Shard	`IndexShard`	Per-shard indexing, search, recovery, engine
Engine	`InternalEngine`	Lucene + translog + seq_no logic
Allocation	`AllocationService`	Decides shard placement; invoked via reroute tasks
Gateway	`GatewayMetaState`	Persists/recovers cluster metadata on disk
Thread pools	`ThreadPool`	`write`, `search`, `management`, `generic`, etc.

Core abstractions

ClusterState — immutable snapshot: Metadata, RoutingTable, ClusterBlocks, CoordinationMetadata. Documented as conceptually immutable in the class javadoc.
ShardRouting — assignment lifecycle: UNASSIGNED → INITIALIZING → STARTED (or RELOCATING).
ActionType<Response> — named action string (e.g. indices:data/write/bulk) bound to a TransportAction.
Plugin — extension point; sub-interfaces (ActionPlugin, SearchPlugin, …) register capabilities.
Engine.Operation — index/delete/noop at Lucene layer with seq_no / version semantics.

Startup lifecycle

bin/elasticsearch → ServerLauncher (CLI JVM) → ServerCli preparer (settings, keystore, JVM args) → Elasticsearch.main (server JVM) initPhase1: Environment, logging, ServerArgs initPhase2: natives, JarHell, plugins, entitlements initPhase3: new Node(...); node.start() plugin LifecycleComponents.start() IndicesService.start() TransportService.start() GatewayMetaState.start() ← load persisted metadata BootstrapChecks (if HTTP about to open) Coordinator.start() ClusterService.start() Coordinator.startInitialJoin() [wait for master if configured] HttpServerTransport.start() sendCliMarker(SERVER_READY) main() returns; keep-alive thread holds JVM

Node.start() ordering (evidence)

Node.start() deliberately starts transport before HTTP, runs gateway metadata recovery, validates bootstrap checks, then joins the cluster and only then binds REST. HTTP starts last so clients cannot hit a half-initialized node.

Node.java — start() Elasticsearch.java — initPhase3

NodeConstruction — the wiring diagram

NodeConstruction.prepareConstruction() is the composition root: loads plugins, builds SettingsModule, NetworkModule, ClusterModule, IndicesModule, ActionModule, creates Guice injector, returns a NodeConstruction handle consumed by Node's constructor.

Key side effect: every TransportAction and REST handler is registered here; missing registration → action not found at runtime.

NodeConstruction.java

Control flow vs data flow

CONTROL (cluster metadata): REST/admin API → TransportMasterNodeAction → MasterService task queue → executor mutates ClusterState → Coordinator.publish → all nodes ClusterApplierService.applyChanges → IndicesClusterStateService creates/drops shards DATA (documents): REST bulk/index → TransportBulkAction → per-shard TransportShardBulkAction → IndexShard → InternalEngine → Lucene IndexWriter + Translog → replication to replica shards (TransportReplicationAction) DATA (search): REST search → TransportSearchAction (coordinator) → SearchTransportService → SearchService.executeQueryPhase → QueryPhase / FetchPhase → merge SearchResponse

External systems

System	Integration point
Local filesystem	`NodeEnvironment` — data, logs, shard stores
Remote object stores	`RepositoryPlugin` — S3/GCS/Azure modules
LDAP/OIDC/SAML	x-pack `security` realm plugins
Kibana	HTTP consumer; system indices (`kibana` module)
Apache Kafka / Beats	Ingest pipelines, Elastic Agent (outside this repo)
Cloud orchestration	Autoscaling x-pack plugin; ESS docker exports in distribution
ML models	x-pack ML plugin; inference ingest processors

Plugin and module system

Built-in modules/ are plugins loaded from modules/ directory in the distribution. Optional plugins install under plugins/. PluginsLoader scans JARs, validates entitlements, instantiates Plugin subclasses.

Plugin hooks used everywhere:

createComponents() — register lifecycle services
getActions() — extra transport actions
getRestHandlers() — REST routes
onIndexModule() — per-index listeners, custom engines
getSettings() — declare Setting<T> keys

Plugin.java

← Overview · Execution Flows →