Elasticsearch works out the time boundaries and creates the past backing indices as the documents land, so a historical data migration runs on your normal ingest path. Check out the different ways to ingest data into Elasticsearch and dive into practical examples to try something new. Elasticsearch is packed with new features to help you build the best search solutions for your use case. Start a free cloud trial or try Elastic on your local machine now. You can now write documents with past timestamps straight into Elasticsearch time series data streams (TSDB) .
Send months of historical metrics through the bulk API , the OpenTelemetry Protocol (OTLP) endpoint , or the Prometheus remote write endpoint . Elasticsearch creates the past backing indices as the documents arrive, computing each index's time boundaries and attaching it to the data stream . Backfilled documents are stored exactly like live ones, with columnar storage and write-time deduplication, along with up to 70% storage savings . Time series data backfill ships in Elasticsearch 9. 5, disabled by default, and turns on with one cluster setting.
How far back you can write depends on your lifecycle configuration, since backfill doesn’t apply to indices that are already read-only as a result of downsampling or a searchable snapshot . Even if loading historical metrics isn’t a very common use case, it’s an important step when teams are adopting TSDB. Two scenarios have been the most prominent: bootstrapping a new time series data stream and migrating data from a different system or data stream to a time series one. You want to start a new time series data stream with a week of historical data so you have something meaningful to query from the start.
With existing tooling, you had to set index. look_back_time to the seven-day maximum in the index template, and all historical data would land in a single backing index. For anything beyond seven days, you needed to create past backing indices manually. You have months worth of metrics stored on a different system and want to move your full dataset to TSDB. You need to load months of metrics history alongside live ingestion. The workaround was to manually create all the necessary past backing indices with the right time_series. start_time and time_series.