-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sql: align the behavior of 'infinity' timestamp with Postgres #127141
Conversation
Thank you for contributing to CockroachDB. Please ensure you have followed the guidelines for creating a PR. My owl senses detect your PR is good for review. Please keep an eye out for any test failures in CI. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
8816411
to
9fb7b4d
Compare
Thank you for updating your pull request. My owl senses detect your PR is good for review. Please keep an eye out for any test failures in CI. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
ecba23d
to
f2de34b
Compare
hi, @rafiss |
Thank you for updating your pull request. Before a member of our team reviews your PR, I have some potential action items for you:
🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
f2de34b
to
f7c9283
Compare
b122b3c
to
79a3639
Compare
Thanks for the contribution, @XiaochenCui! I'll let @rafiss take a look at most of this, but please ping me when you address the TODO in the stats code. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @herkolategan, @nameisbhaskar, @rafiss, and @rytaft)
pkg/sql/logictest/testdata/logic_test/lookup_join
line 1343 at r8 (raw file):
Previously, XiaochenCui (Xiaochen Cui) wrote…
don't worry, the behaviour of 'infinity'::TIME and 'infinity'::TIME has been keep intact carefully, see here for details:
// 24 hours after "MaxSupportedTime". (294277-01-01 23:59:59.999999 +0000 UTC) // We keep the "time" part as "23:59:59.999999" to keep the behavior of 'Infinity'::time // intact. (In the previous implementation, 'Infinity'::time becomes '23:59:59.999999') TimeInfinity = timeutil.Unix(9224318102399, 999999000) TimeInfinitySec = float64(TimeInfinity.Unix()) // 24 hours before "MinSupportedTime". (-4714-11-23 00:00:00 +0000 UTC) // We keep the "time" part as "00:00:00" to keep the behavior of '-Infinity'::time // intact. (In the previous implementation, '-Infinity'::time becomes '00:00:00') TimeNegativeInfinity = timeutil.Unix(-210898425600, 0) TimeNegativeInfinitySec = float64(TimeNegativeInfinity.Unix())
typo: the behaviour of 'infinity'::TIME and '-infinity'::TIME
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @herkolategan, @nameisbhaskar, @rafiss, and @rytaft)
pkg/sql/logictest/testdata/logic_test/lookup_join
line 1343 at r8 (raw file):
Previously, XiaochenCui (Xiaochen Cui) wrote…
typo: the behaviour of 'infinity'::TIME and '-infinity'::TIME
I have moved the discussion of 'infinity'::TIME to a separate issue:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 2 of 2 files at r10, 1 of 2 files at r11, 2 of 2 files at r12, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @herkolategan, @nameisbhaskar, @rafiss, and @XiaochenCui)
pkg/sql/stats/quantile.go
line 366 at r12 (raw file):
// support round-trip conversions. // // TODO(michae2): Add support for DECIMAL and INTERVAL.
why did you remove these values from the comment? Looks like they still aren't supported
pkg/sql/stats/quantile.go
line 418 at r12 (raw file):
// rather than failing the conversion (and thus the entire histogram). // // TODO(michae2): Add support for DECIMAL, TIME, TIMETZ, and INTERVAL.
ditto
pkg/util/timeutil/pgdate/parsing.go
line 72 at r9 (raw file):
Previously, XiaochenCui (Xiaochen Cui) wrote…
It's the args used to construct "294277-01-01 23:59:59.999999 +0000 UTC", we have to click "EXPAND" to see the full comments.
I see the comment, but it's not obvious to me how you got this number from "294277-01-01 23:59:59.999999 +0000 UTC"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @herkolategan, @nameisbhaskar, @rafiss, and @rytaft)
pkg/sql/stats/quantile.go
line 366 at r12 (raw file):
Previously, rytaft (Rebecca Taft) wrote…
why did you remove these values from the comment? Looks like they still aren't supported
Is this the support for TIME and TIMETZ?
code in the master branch:
cockroach/pkg/sql/stats/quantile.go
Lines 382 to 392 in 446bc36
case *tree.DTimestamp: | |
if v.Equal(pgdate.TimeInfinity) || v.Equal(pgdate.TimeNegativeInfinity) { | |
return 0, tree.ErrFloatOutOfRange | |
} | |
return float64(v.Unix()) + float64(v.Nanosecond())*1e-9, nil | |
case *tree.DTimestampTZ: | |
// TIMESTAMPTZ doesn't store a timezone, so this is the same as TIMESTAMP. | |
if v.Equal(pgdate.TimeInfinity) || v.Equal(pgdate.TimeNegativeInfinity) { | |
return 0, tree.ErrFloatOutOfRange | |
} | |
return float64(v.Unix()) + float64(v.Nanosecond())*1e-9, nil |
code in this pr:
case *tree.DTimestamp:
if v.Equal(pgdate.TimeInfinity) {
return pgdate.TimeInfinitySec, nil
}
if v.Equal(pgdate.TimeNegativeInfinity) {
return pgdate.TimeNegativeInfinitySec, nil
}
return float64(v.Unix()) + float64(v.Nanosecond())*1e-9, nil
case *tree.DTimestampTZ:
// TIMESTAMPTZ doesn't store a timezone, so this is the same as TIMESTAMP.
if v.Equal(pgdate.TimeInfinity) {
return pgdate.TimeInfinitySec, nil
}
if v.Equal(pgdate.TimeNegativeInfinity) {
return pgdate.TimeNegativeInfinitySec, nil
}
return float64(v.Unix()) + float64(v.Nanosecond())*1e-9, nil
pkg/sql/stats/quantile.go
line 418 at r12 (raw file):
Previously, rytaft (Rebecca Taft) wrote…
ditto
Is this the support for TIME and TIMETZ?
code in the master branch:
cockroach/pkg/sql/stats/quantile.go
Lines 471 to 486 in 446bc36
case types.TimestampFamily, types.TimestampTZFamily: | |
sec, frac := math.Modf(val) | |
var t time.Time | |
// Clamp to (our alternative finite) DTimestamp bounds. | |
if sec <= quantileMinTimestampSec { | |
t = quantileMinTimestamp | |
} else if sec >= quantileMaxTimestampSec { | |
t = quantileMaxTimestamp | |
} else { | |
t = timeutil.Unix(int64(sec), int64(frac*1e9)) | |
} | |
roundTo := tree.TimeFamilyPrecisionToRoundDuration(colType.Precision()) | |
if colType.Family() == types.TimestampFamily { | |
return tree.MakeDTimestamp(t, roundTo) | |
} | |
return tree.MakeDTimestampTZ(t, roundTo) |
code in this pr:
case types.TimestampFamily, types.TimestampTZFamily:
sec, frac := math.Modf(val)
var t time.Time
// Clamp to DTimestamp bounds.
if sec <= pgdate.TimeNegativeInfinitySec {
t = pgdate.TimeNegativeInfinity
} else if sec >= pgdate.TimeInfinitySec {
t = pgdate.TimeInfinity
} else {
t = timeutil.Unix(int64(sec), int64(frac*1e9))
}
roundTo := tree.TimeFamilyPrecisionToRoundDuration(colType.Precision())
if colType.Family() == types.TimestampFamily {
return tree.MakeDTimestamp(t, roundTo)
}
return tree.MakeDTimestampTZ(t, roundTo)
pkg/util/timeutil/pgdate/parsing.go
line 72 at r9 (raw file):
Previously, rytaft (Rebecca Taft) wrote…
I see the comment, but it's not obvious to me how you got this number from "294277-01-01 23:59:59.999999 +0000 UTC"
Yeah, it's not easy, I wrote a script to convert a time literal to the args of "timeuitl.Unix":
We can prove the result is right after that:
https://go.dev/play/p/z5S_Z4R1q4L
Do you have any ideas for this code? It will be great if we can make if clearer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is getting really close! Once you fix up the comments I think it's ready to merge.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @herkolategan, @nameisbhaskar, @rafiss, and @XiaochenCui)
pkg/sql/stats/quantile.go
line 366 at r12 (raw file):
Previously, XiaochenCui (Xiaochen Cui) wrote…
Is this the support for TIME and TIMETZ?
code in the master branch:
cockroach/pkg/sql/stats/quantile.go
Lines 382 to 392 in 446bc36
case *tree.DTimestamp: if v.Equal(pgdate.TimeInfinity) || v.Equal(pgdate.TimeNegativeInfinity) { return 0, tree.ErrFloatOutOfRange } return float64(v.Unix()) + float64(v.Nanosecond())*1e-9, nil case *tree.DTimestampTZ: // TIMESTAMPTZ doesn't store a timezone, so this is the same as TIMESTAMP. if v.Equal(pgdate.TimeInfinity) || v.Equal(pgdate.TimeNegativeInfinity) { return 0, tree.ErrFloatOutOfRange } return float64(v.Unix()) + float64(v.Nanosecond())*1e-9, nil code in this pr:
case *tree.DTimestamp: if v.Equal(pgdate.TimeInfinity) { return pgdate.TimeInfinitySec, nil } if v.Equal(pgdate.TimeNegativeInfinity) { return pgdate.TimeNegativeInfinitySec, nil } return float64(v.Unix()) + float64(v.Nanosecond())*1e-9, nil case *tree.DTimestampTZ: // TIMESTAMPTZ doesn't store a timezone, so this is the same as TIMESTAMP. if v.Equal(pgdate.TimeInfinity) { return pgdate.TimeInfinitySec, nil } if v.Equal(pgdate.TimeNegativeInfinity) { return pgdate.TimeNegativeInfinitySec, nil } return float64(v.Unix()) + float64(v.Nanosecond())*1e-9, nil
No, those are for Timestamp and TimesampTZ, which are different data types from Time and TimeTZ
pkg/sql/stats/quantile.go
line 418 at r12 (raw file):
Previously, XiaochenCui (Xiaochen Cui) wrote…
Is this the support for TIME and TIMETZ?
code in the master branch:
cockroach/pkg/sql/stats/quantile.go
Lines 471 to 486 in 446bc36
case types.TimestampFamily, types.TimestampTZFamily: sec, frac := math.Modf(val) var t time.Time // Clamp to (our alternative finite) DTimestamp bounds. if sec <= quantileMinTimestampSec { t = quantileMinTimestamp } else if sec >= quantileMaxTimestampSec { t = quantileMaxTimestamp } else { t = timeutil.Unix(int64(sec), int64(frac*1e9)) } roundTo := tree.TimeFamilyPrecisionToRoundDuration(colType.Precision()) if colType.Family() == types.TimestampFamily { return tree.MakeDTimestamp(t, roundTo) } return tree.MakeDTimestampTZ(t, roundTo) code in this pr:
case types.TimestampFamily, types.TimestampTZFamily: sec, frac := math.Modf(val) var t time.Time // Clamp to DTimestamp bounds. if sec <= pgdate.TimeNegativeInfinitySec { t = pgdate.TimeNegativeInfinity } else if sec >= pgdate.TimeInfinitySec { t = pgdate.TimeInfinity } else { t = timeutil.Unix(int64(sec), int64(frac*1e9)) } roundTo := tree.TimeFamilyPrecisionToRoundDuration(colType.Precision()) if colType.Family() == types.TimestampFamily { return tree.MakeDTimestamp(t, roundTo) } return tree.MakeDTimestampTZ(t, roundTo)
Same response as above -- those are different datatypes.
pkg/util/timeutil/pgdate/parsing.go
line 72 at r9 (raw file):
Previously, XiaochenCui (Xiaochen Cui) wrote…
Yeah, it's not easy, I wrote a script to convert a time literal to the args of "timeuitl.Unix":
We can prove the result is right after that:
https://go.dev/play/p/z5S_Z4R1q4L
Do you have any ideas for this code? It will be great if we can make if clearer.
I think if you could put the key pieces of that code into the comment it would help. i.e. describe what the getBinary
function does in your script, and include the proof that conversion back to human-readable format works.
It would also help to write this as timeutil.Unix(9224318102399 /* sec */, 999999000 /* nsec */)
(ditto below).
I'd also keep some of the previous comment, which includes info about the Postgres behavior and how we differ. Someone reading this code without the context of this PR won't know what you mean by "the previous implementation".
You might also want to add a reference to the issue you opened here.
6df392c
to
e2f56dd
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @herkolategan, @nameisbhaskar, @rafiss, and @rytaft)
pkg/sql/stats/quantile.go
line 366 at r12 (raw file):
Previously, rytaft (Rebecca Taft) wrote…
No, those are for Timestamp and TimesampTZ, which are different data types from Time and TimeTZ
Done.
pkg/sql/stats/quantile.go
line 418 at r12 (raw file):
Previously, rytaft (Rebecca Taft) wrote…
Same response as above -- those are different datatypes.
Done.
pkg/util/timeutil/pgdate/parsing.go
line 72 at r9 (raw file):
I've updated the comments for "TimeInfinity" and "TimeNegativeInfinity", the comments follow a structure of:
- what the value is
- the reason behind the date part
- the reason behind the time part
- the implementation of postgres
- the process to derive the arguments "sec" and "nsec"
I tried to clarify 'the reason behind the time part' and the explanation still feels a bit unclear. Do you have any suggestions?
For 'the the process to derive the arguments "sec" and "nsec"', I documented the process in the doc comments of function "timeutil.Unix" and referenced it. I didn't write it at "TimeInfinity" since there are at least 4 special timestamps in crdb which need this comment:
- TimeInfinity
- TimeNegativeInfinity
- MaxSupportedTime
- MinSupportedTime
describe what the
getBinary
function does in your script
the getBinary
function has nothing to do with this logic, it's used to verify updates in the file "pkg/sql/pgwire/testdata/encodings.json"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the contribution, @XiaochenCui ! Sorry this took so long for us to get merged.
bors r+
Reviewed 4 of 4 files at r13, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @herkolategan, @nameisbhaskar, @rafiss, and @XiaochenCui)
pkg/util/timeutil/pgdate/parsing.go
line 72 at r9 (raw file):
Previously, XiaochenCui (Xiaochen Cui) wrote…
I've updated the comments for "TimeInfinity" and "TimeNegativeInfinity", the comments follow a structure of:
- what the value is
- the reason behind the date part
- the reason behind the time part
- the implementation of postgres
- the process to derive the arguments "sec" and "nsec"
I tried to clarify 'the reason behind the time part' and the explanation still feels a bit unclear. Do you have any suggestions?
For 'the the process to derive the arguments "sec" and "nsec"', I documented the process in the doc comments of function "timeutil.Unix" and referenced it. I didn't write it at "TimeInfinity" since there are at least 4 special timestamps in crdb which need this comment:
- TimeInfinity
- TimeNegativeInfinity
- MaxSupportedTime
- MinSupportedTime
describe what the
getBinary
function does in your scriptthe
getBinary
function has nothing to do with this logic, it's used to verify updates in the file "pkg/sql/pgwire/testdata/encodings.json"
Thanks so much, this helps a lot! I think this is ready to go.
Build failed (retrying...): |
Build failed (retrying...): |
Build failed (retrying...): |
bors r- This PR is failing on the |
Canceled. |
Previously, "SELECT 'infinity'::timestamp" would return a concrete timestamp instead of "infinity", which is incompatible with Postgres, causing issue cockroachdb#41564. This commit addresses this issue by: - Returning "infinity" for 'infinity'::timestamp, supporting both text and binary formats. This commit also updates the cmp/eval behavior of "infinity" timestamps. The updated behavior is the same as Postgres: - "infinity" is always larger than other timestamps, and "infinity" equals itself. - "-infinity" is always smaller than other timestamps, and "-infinity" equals itself. - "infinity"/"-infinity" add or subtract any duration results in itself. Fixes: cockroachdb#41564 Release note (bug fix): Fixed a bug where 'infinity'::timestamp returned a different result than Postgres.
e2f56dd
to
63f8a13
Compare
lint fixed |
Thanks! bors r+ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale)
pkg/sql/pgwire/pgwirebase/encoding.go
line 929 at r14 (raw file):
if i == math.MinInt64 { return pgdate.TimeNegativeInfinity }
hi, @rytaft , I would like to add comments to explain the meaning of these two special numbers, what do you think? should we open a new PR to add tiny comments?
Previously, XiaochenCui (Xiaochen Cui) wrote…
Hi @XiaochenCui -- you're welcome and encouraged to open a PR to improve comments! Please feel free to tag me as a reviewer. |
Description
Previously,
SELECT 'infinity'::timestamp
will return a concrete timestamp instead of 'infinity', which is incompatible with Postgres, causing issue #41564.This commit addresses this issue by:
This commit also updates the cmp/eval behavior of "infinity" timestamps. The updated behavior is the same as Postgres:
Fixes: #41564
Release note (bug fix): Fixed a bug where 'infinity'::timestamp returns different result than postgres.
How to Review
TimeInfinity
andTimeNegativeInfinity
in pkg/util/timeutil/pgdate/parsing.go.Tasks
TODO (sql behavior):
'infinity'::TIME
and'-infinity'::TIME
intact.TODO (implementation):
/pkg/sql/pgwire/testdata/encodings.json
.to_timestamp
function align with type cast.MakeDTimestamp
andMakeDTimestampTZ
.cockroach/pkg/sql/sem/tree/datum.go
Lines 2592 to 2599 in 1ebd9e1
cockroach/pkg/sql/sem/tree/datum.go
Lines 2877 to 2883 in 1ebd9e1
TODO (review):