Skip to content

/prometheus

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce timestamp monotonicity #481

Closed
beorn7 opened this Issue Jan 27, 2015 · 2 comments

Comments

Assignees

@beorn7 beorn7

Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@beorn7 @juliusv
@beorn7
Copy link
Member

beorn7 commented Jan 27, 2015

The storage requires that samples are sorted by timestamp, i.e. a timestamp that is saved to the storage layer must have a timestamp at least as high as the previous sample. This constraint is just by contract so far. It is not checked or enforced.

Unfortunately, that's not trivial as a time series might already be archived when a sample for it comes in again. It's pretty expensive to unarchive a series just to check the latest timestamp.

Ideas:

  • Never accept timestamps from the future (+/- a certain tolerated skew).
  • Then the mtime of a time series file is an upper bound for the last timestamp.
  • Save the last timestamp in the series object.

@beorn7 beorn7 self-assigned this Jan 27, 2015

@beorn7 beorn7 added the enhancement label Feb 2, 2015

@juliusv juliusv referenced this issue Feb 17, 2015

Open

Add mechanism to perform bulk imports #535

@beorn7 beorn7 changed the title Enforce timestamp monotony Enforce timestamp monotonicity Mar 14, 2015

@juliusv juliusv referenced this issue Jul 12, 2015

Merged

Ensure timestamp monotonicity within a series. #889

@juliusv

This comment has been minimized.

Sign in to view
Copy link
Member

juliusv commented Jul 12, 2015

When we append a new sample to an archived series, we unarchive the series anyways and get the last timestamp of the series from the unarchiving. So even that case seems to be cheap to check.

Implementation: #889

juliusv added a commit that referenced this issue Jul 13, 2015

@juliusv
storage: ensure timestamp monotonicity within series. 
Fixes #481
Loading status checks…
02bec64

beorn7 added a commit that referenced this issue Jul 13, 2015

@beorn7
storage: ensure timestamp monotonicity within series. 
Fixes #481

While doing so, clean up and fix a few other things:
- TODO
Loading status checks…
cfc6573

beorn7 added a commit that referenced this issue Jul 15, 2015

@beorn7
storage: ensure timestamp monotonicity within series. 
Fixes #481

While doing so, clean up and fix a few other things:

- Fix `go vet` warnings (@fabxc to blame ;).

- Fix a racey problem with unarchiving: Whenever we unarchive a
  series, we essentially want to do something with it. However, until
  we have done something with it, it appears like a series that is
  ready to be archived or even purged. So e.g. it would be ignored
  during checkpointing. With this fix, we always load the chunkDescs
  upon unarchiving. This is wasteful if we only want to add a new
  sample to an archived time series, but the (presumably more common)
  case where we access an archived time series in a query doesn't
  become more expensive.

- The change above streamlined the getOrCreateSeries ond
  newMemorySeries flow. Also, the modTime is now always set correctly.

- Fix the leveldb-backed implementation of KeyValueStore.Delete. It
  had the wrong behavior of still returning true, nil if a
  non-existing key has been passed in.
Loading status checks…
ff08f0b

@beorn7 beorn7 closed this in #889 Jul 16, 2015

@lock

This comment has been minimized.

Sign in to view
Copy link

lock bot commented Mar 24, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Mar 24, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.