Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support writing to elastic data streams #2996

Closed
jeremyross opened this issue Feb 6, 2022 · 2 comments · Fixed by #2997
Closed

Support writing to elastic data streams #2996

jeremyross opened this issue Feb 6, 2022 · 2 comments · Fixed by #2997
Labels
enhancement A general enhancement registry: elastic An ElasticSearch Registry related issue
Milestone

Comments

@jeremyross
Copy link
Contributor

jeremyross commented Feb 6, 2022

Currently micrometer cannot write to elastic data streams because micrometer uses the index operation.

From the docs:

A data stream lets you store append-only time series data across multiple indices while giving you a single named resource for requests. Data streams are well-suited for logs, events, metrics, and other continuously generated data.

Data streams have been out for a while now and are the preferred mechanism for write-only, time series data because they support Index Lifecycle Management, which provides automation around index management.

The current barrier to micrometer writing to data streams is that requests must use an op_type of create

Luckily this is a simple change. Currently micrometer uses the index operation, which is analogous to a PUT or upsert. Semantically speaking, the create operation is a better fit since micrometer should always be creating new documents. So, the solution is to use the create operation, which is a better semantic fit and works with data streams.

@jeremyross
Copy link
Contributor Author

This also solves the root issue behind #2255.

@shakuzen shakuzen added the registry: elastic An ElasticSearch Registry related issue label Feb 7, 2022
@shakuzen shakuzen added this to the 1.9 backlog milestone Feb 7, 2022
@jeremyross
Copy link
Contributor Author

This also solves the inevitable roadblock users would hit after #2705 is solved. See this comment.

@shakuzen shakuzen modified the milestones: 1.9 backlog, 1.9.0-M3 Feb 16, 2022
@shakuzen shakuzen linked a pull request Feb 16, 2022 that will close this issue
shakuzen added a commit that referenced this issue Feb 16, 2022
Allows data to be written to indexes or data streams.

Resolves gh-2996

Co-authored-by: Tommy Ludwig <8924140+shakuzen@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement A general enhancement registry: elastic An ElasticSearch Registry related issue
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants