Skip to content

Latest commit

 

History

History
48 lines (34 loc) · 2.52 KB

README.md

File metadata and controls

48 lines (34 loc) · 2.52 KB
Raw

Elasticsearch Snapshot Index Orchestrator

RESTful API for orchestrating the restoration of time-based Elasticsearch snapshots.

This API gives you a simple interface for restoring snapshots based on a given time range.

For example, if snapshots are stored in the following daily (repo/snap/index) pattern: logs-%Y/logs-%Y-%m-%d/logs-v1-%Y-%m-%d then requests made to the POST /{start}/{end} route would trigger a restore of indices in the range that maps to the pattern.

High level of what the API does:

  • Accepts requests to restore indices from snapshot repository.
  • Incoming requests are queued for restore and queue is processed every 2 seconds.
  • Requests for additional index restorations are queued while other recoveries are in progress.
  • Available index snapshots are listed via the ES Snapshot/Restore API.
  • Ongoing recoveries and online indices are obtained via the ES Index API.

OpenAPI Spec

The OpenAPI 2.0 spec is here: ./swagger.yml

Configuration

ESIO Flags:
      --es-host=         Elasticsearch Host [$ES_HOST]
      --max-restore=     Maximum number of indices allowed to restore at once, default is 1 [$MAX_RESTORE]
      --resolution=      Resolution of indices being restored (day, month, year) [$INDEX_RESOLUTION]
      --repo-pattern=    Snapshot repo pattern (repo/snap/index), ex: logs-%y/logs-%y-%m-%d/logs-v1-%y-%m-%d,
                         [$REPO_PATTERN]

Development

Use the make targets to build and test the esio-server.

  • make: validate the swagger spec, fetch dependencies, compile the source and run the server in the foreground.
  • make test: Start Elasticsearch via Docker in background, start server in background and run all tests
  • make gen: Regenerate swagger framework from swagger.yml
  • make init-test-data: Creates indices with faux timeseries data and snapshot repo in Elasticsearch for testing.
  • make start-elastic: start Elasticsearch and Kibana
  • make stop-elastic: stop Elasticsearch and Kibana

Resources