Make a DB aware of the first timestamp stored in it #134
Conversation
Seems like you'd probably want to acquire a read-lock on |
There's no requirement from prometheus/tsdb's side for stored timestamps to
map to wall time in any way and esp. the chosen precision.
So returning `time.Time` would not make sense here.
…On Wed, Aug 30, 2017 at 4:53 PM Matt Layher ***@***.***> wrote:
Seems like you'd probably want to acquire a read-lock on mtx since it
guards blocks? Also, would it make sense to return a time.Time instead of
a UNIX timestamp disguised as an int64?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#134 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AEuA8jdMcSCzMN6JWTiRzRaC6MuaxwQxks5sdXd8gaJpZM4PHGd2>
.
|
db.go
Outdated
|
||
indexr := b.Index() | ||
joblabel := "job" | ||
tpls, err := indexr.LabelValues(joblabel) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's leaking Prometheus semantics into TSDB, which we generally want to avoid.
I think it's somewhat questionable how well the overall query has something to do with TSDB.
It's only accessing public interfaces so it doesn't have to be in here.
Anything speaking against implementing this in prometheus/storage/tsdb
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could work.
Would you see another way to get the first time without having to do this hack with a label ? Maybe iterating on the chunks directly ? (I'm not super familiar with the interfaces yet)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@fabxc It doesn't only access public interfacess. It need access to blocks list which are not public yet (and I thought it was private on purpose)
@iksaif I didn't find an other way. Especially I couldn't access to a chunks list i could iterate on it.
I only see two possible solutions :
(1) make blocks public adding a BD.Blocks() func (and move the logic to prometheus/storage/tsdb
)
(2) iterate on existing labels (using block.Index().LabelIndices()) instead of using "job" labels. It means keeping the same logic without prometheus specific labelname.
What do you think ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Making certain methods public if there's a use case for it is perfectly valid. The exported interfaces so far are largely on a best-guess basis :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
Hmm, anything speaking against just returning the MinTime in the oldest block? |
@gouthamve the MinTime in the oldest block seems to be not very accurate (could be several hours from the MinTime of the first chunk AFAIK) |
This enable computing the first timestamp in the DB only accessing public interfaces. Signed-off-by: Thibault Chataigner <t.chataigner@criteo.com>
@Thib17 Sorry for the late reply, yes, but it is not possible that there exist a sample in remote-storage that doesn't exist locally. For example if This doesn't hold in the case of a manual series delete, but I don't think we need to optimise for that. PS: I have nothing against a public method |
@gouthamve , see prometheus/prometheus#10 (comment) See the remote storage as something that's really durable, and local disk as something that can easily fail and get lost. This is even more true when you start running prometheus on marathon/mesos/kube/borg. |
Yes, I agree to that. Hmm, it will be very rare that data is corrupted in just the last chunk of a particular series. In that case, this makes sense, but in the general case, I think just looking at the I am just saying instead of trying to lookup the In case there is corruption and you lost a block, |
@gouthamve I think we agree and move the discussion on Prometheus' side : @fabxc Is there anything missing before merging this PR ? |
Fabian is on a vacation now but this PR looks good (: Looking forward to see what comes up on prometheus/prometheus#3129 |
This is needed to fix prometheus/prometheus#3041