Create and query a product index
Create an index with text, keyword, and date fields. Bulk insert 20 products, then run one match query and one term filter. Capture response time and explain why scores differ.
Data / Knowledge library
Elasticsearch
distributed search and analytics engine
Why learn Elasticsearch?
Elasticsearch enables fast full-text search, filtering, aggregations, and near real-time analytics over large document datasets, with horizontal scaling and high availability.
Where it came from
2010: First public release, built on Apache Lucene. 2012: Elastic (company) formed to support Elasticsearch and ecosystem growth. 2015: ELK evolved toward the Elastic Stack (Elasticsearch, Logstash, Kibana, Beats). 2021: Licensing changed from Apache 2.0 distribution to Elastic License/SSPL model for new versions. 2022: Elasticsearch 8.0 enabled security features by default (TLS and authentication).
- 2010: First public release, built on Apache Lucene.
- 2012: Elastic (company) formed to support Elasticsearch and ecosystem growth.
- 2015: ELK evolved toward the Elastic Stack (Elasticsearch, Logstash, Kibana, Beats).
- 2021: Licensing changed from Apache 2.0 distribution to Elastic License/SSPL model for new versions.
- 2022: Elasticsearch 8.0 enabled security features by default (TLS and authentication).
Build your understanding
Work through these exercises in order. Try the task before opening a reference.
- Create an index with text, keyword, and date fields. Bulk insert 20 products, then run one match query and one term filter. Capture response time and explain why scores differ.
- Write one _search request that returns results plus category and price-range aggregations. Keep query under 25 lines and make it mobile-readable with clear indentation.
- For daily log data, define index template + ILM policy (hot to delete) and justify shard/replica choices for 100 GB/day ingest.
- Combine BM25 query text relevance with vector kNN retrieval for a help-center dataset. Describe when hybrid beats keyword-only search using one concrete example.
- Simulate one node loss in a 3-node cluster. Observe cluster health, reroute behavior, and explain recovery steps and key APIs used.
More ways to practice
Build a faceted search endpoint
Write one _search request that returns results plus category and price-range aggregations. Keep query under 25 lines and make it mobile-readable with clear indentation.
Design log retention with ILM
For daily log data, define index template + ILM policy (hot to delete) and justify shard/replica choices for 100 GB/day ingest.
Prototype hybrid retrieval
Combine BM25 query text relevance with vector kNN retrieval for a help-center dataset. Describe when hybrid beats keyword-only search using one concrete example.
Failure drill: shard recovery
Simulate one node loss in a 3-node cluster. Observe cluster health, reroute behavior, and explain recovery steps and key APIs used.
Documentation and sources
- Elasticsearch reference (www.elastic.co)
- Query DSL (www.elastic.co)
- Aggregations (www.elastic.co)
- REST APIs (www.elastic.co)
- JavaScript client (www.elastic.co)
Catalog research notes
- 8.x introduced ES|QL for piped search/analysis workflows across observability and security data.
- Vector search in 8.x improved with better kNN tuning and stronger hybrid (keyword + vector) retrieval patterns.
- Search relevance tooling expanded with semantic-oriented field types and simpler retrieval pipelines.
- Elastic Cloud Serverless options expanded, reducing ops overhead for small teams and bursty workloads.
Connect the ideas
Alternatives
Explore next
Real World Example
Global Logistics Inventory Visibility
A global logistics provider managing millions of shipping containers across thousands of international ports.
The business problem
The company struggles with fragmented data silos, where tracking information for containers is stored in disparate legacy databases, leading to high latency when generating real-time inventory reports for regional managers.
How this topic solves it
Elasticsearch acts as a centralized, distributed search and analytics engine that ingests container status updates from various sources into a unified index, allowing for near real-time querying and complex aggregations across the entire global fleet.
- Define a consistent index mapping to normalize container status fields across different regional data sources.
- Configure ingest pipelines to transform and enrich raw incoming container telemetry data before indexing.
- Utilize data streams to manage time-series container tracking events for efficient storage and lifecycle management.
- Implement Query DSL to execute complex filtering and aggregation requests for regional inventory dashboards.
- Monitor cluster health and shard distribution to ensure high availability and consistent search performance.
This diagram illustrates the flow of container data from regional sources into the centralized Elasticsearch cluster.
Legacy databases and IoT sensors
Normalization and enrichment logic
Unified container data store
Query DSL interface for managers
- Regional Sources → raw telemetry → Ingest Pipeline
- Ingest Pipeline → indexed documents → Elasticsearch Index
- Elasticsearch Index → search results → Dashboard API
Expected business outcome
Regional managers gain immediate, unified visibility into container locations and statuses, significantly reducing the time required to generate operational reports.
Limitations and checks
A critical limitation is the potential for shard imbalance during heavy ingestion spikes, which can lead to uneven resource utilization across nodes; this is checked by monitoring node-level heap usage and shard distribution statistics.