Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

compact: data corruption during downsapmle, test and fix. #6598

Merged
merged 10 commits into from
Sep 12, 2023
Merged

compact: data corruption during downsapmle, test and fix. #6598

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
7d40d81
Samples to reproduce data corruption during downsapmle, tests and fix.
xBazilio Aug 9, 2023
26cee2c
Samples to reproduce data corruption during downsapmle, tests and fix.
xBazilio Aug 9, 2023
053e5e5
added test for chunk with NaN values only
xBazilio Aug 11, 2023
9666d26
Merge branch 'main' into fix_downsample_data_corruption
xBazilio Aug 11, 2023
33e413f
CHANGELOG.md
xBazilio Aug 11, 2023
c8b6eac
Merge branch 'main' into fix_downsample_data_corruption
xBazilio Aug 18, 2023
088628b
Merge branch 'main' into fix_downsample_data_corruption
xBazilio Aug 21, 2023
58aa26e
added check for math.NaN
xBazilio Aug 21, 2023
e61bac0
optimized NaN checking
xBazilio Aug 24, 2023
6c70731
Merge branch 'main' into fix_downsample_data_corruption
xBazilio Aug 24, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion pkg/compact/downsample/downsample.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -374,7 +374,7 @@ func downsampleRawLoop(data []sample, resolution int64, numChunks int) []chunks.

batch := make([]sample, 0, j)
for _, s := range data[:j] {
if value.IsStaleNaN(s.v) || math.Float64bits(s.v) == value.NormalNaN {
if value.IsStaleNaN(s.v) || math.Float64bits(s.v) == value.NormalNaN || math.IsNaN(s.v) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need this? I thought math.IsNaN includes both StaleNaN and NormalNaN

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed. and added math.NaN() to test

continue
}
batch = append(batch, s)
Expand Down