Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prometheus should store data to a persistent storage backend #99

Closed
jfchevrette opened this issue Nov 16, 2018 · 4 comments
Closed

Prometheus should store data to a persistent storage backend #99

jfchevrette opened this issue Nov 16, 2018 · 4 comments

Comments

@jfchevrette
Copy link
Member

Following our conversation last week, I'm opening this issue to discuss adding persistent storage to Prometheus for telemeter.

I see two options:

  • Add a spec.storage.volumeClaimTemplate on the Prometheus object and use EC2 backed PVs to store data.
  • Deploy Thanos and use that as a storage backend

I think we can also do both where EC2 would be short term data and Thanos/S3 would have mid-long term data?

We need to keep in mind EC2 PV resizing is not possible at the moment on the target platform so if we're going to use that we'll need to think through sizing the volumes adequately for the amount of data we expect to store.

@squat
Copy link
Contributor

squat commented Nov 29, 2018

If we really want some kind of persistence, I think we should add Thanos to our architecture. This will do automatic backing up (to S3) and compaction of all collected metrics, while still allowing them to be queried.

cc @brancz

@brancz
Copy link
Contributor

brancz commented Nov 29, 2018

@squat I think that's a good idea.

@jfchevrette
Copy link
Member Author

@brancz @squat what is the status on this?

@jfchevrette
Copy link
Member Author

Addressed by #126

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants