New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stats: fix bug when convert time to scalar #4760

Merged
merged 2 commits into from Oct 12, 2017

Conversation

Projects
None yet
4 participants
@lamxTyler
Member

lamxTyler commented Oct 12, 2017

When we do sub on zero Timestamp, original code will raise an error.
Fix #4758
PTAL @coocood @hanfei1991 @winoros

@lamxTyler lamxTyler closed this Oct 12, 2017

@coocood coocood reopened this Oct 12, 2017

@coocood

This comment has been minimized.

Show comment
Hide comment
@coocood

coocood Oct 12, 2017

Member

This is more accurate than packedUint.

Member

coocood commented Oct 12, 2017

This is more accurate than packedUint.

@coocood

This comment has been minimized.

Show comment
Hide comment
@coocood

coocood Oct 12, 2017

Member

LGTM

Member

coocood commented Oct 12, 2017

LGTM

@zz-jason

LGTM

@coocood coocood merged commit bd89428 into master Oct 12, 2017

3 of 4 checks passed

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
ci/circleci Your tests passed on CircleCI!
Details
jenkins-ci-tidb/build Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details

@coocood coocood deleted the xhb/fix branch Oct 12, 2017

Fsp: types.DefaultFsp,
}
}
return float64(valueTime.Sub(&minTime).Duration)

This comment has been minimized.

@Darren

Darren Oct 12, 2017

Contributor

if valueTime.isZero(), then it will log error again.

@Darren

Darren Oct 12, 2017

Contributor

if valueTime.isZero(), then it will log error again.

This comment has been minimized.

@lamxTyler

lamxTyler Oct 12, 2017

Member

This will not happen because this function will be called only when the value is not less than the bucket boundaries, and the bucket boundaries are at least as large as minTime.

@lamxTyler

lamxTyler Oct 12, 2017

Member

This will not happen because this function will be called only when the value is not less than the bucket boundaries, and the bucket boundaries are at least as large as minTime.

@Darren

This comment has been minimized.

Show comment
Hide comment
@Darren

Darren Oct 12, 2017

Contributor

@lamxTyler

I tried commit bd89428, But it still reports error in our cluster, I added panic in util/types/mytime.go, here's the stacktrace, all the line numbers are correct as in bd89428 execept the first one.

Is it because there are bad boundaries in our mysql.stats_buckets?

panic: tm: -0001-11-30 00:00:00 +0800 CST, t:{year:0 month:0 day:0 hour:0 minute:0 second:0 microsecond:0} 


goroutine 1 [running]:
github.com/pingcap/tidb/util/types.mysqlTime.GoTime(0x0, 0x0, 0x0, 0x1afede0, 0x0, 0xd8, 0xc42097f778, 0x440fa0, 0xc42069ebf8)
        /go/src/github.com/pingcap/tidb/util/types/mytime.go:104 +0x53d
github.com/pingcap/tidb/util/types.(*mysqlTime).GoTime(0xc424abdaa0, 0x1afede0, 0xc42097f738, 0x7ffe787fea63, 0xc42097f778, 0x40568c, 0xffb620)
        <autogenerated>:63 +0x60
github.com/pingcap/tidb/util/types.(*Time).Sub(0xc42097f7f8, 0xc42097f828, 0x1, 0x0)
        /go/src/github.com/pingcap/tidb/util/types/time.go:505 +0xf4
github.com/pingcap/tidb/statistics.convertDatumToScalar(0xc42097fa48, 0x0, 0x1140f00)
        /go/src/github.com/pingcap/tidb/statistics/scalar.go:95 +0x409
github.com/pingcap/tidb/statistics.preCalculateDatumScalar(0xc42097fa48, 0xc42097fa10, 0xc420aa1b90, 0xd, 0x0)
        /go/src/github.com/pingcap/tidb/statistics/scalar.go:51 +0x8e
github.com/pingcap/tidb/statistics.histogramFromStorage(0x18e47c0, 0xc42038a500, 0x35e, 0x2, 0xc420aa1b90, 0x1, 0x0, 0x57c3d569ed80002, 0x0, 0xc424b3e940, ...)
        /go/src/github.com/pingcap/tidb/statistics/histogram.go:146 +0x4b9
github.com/pingcap/tidb/statistics.(*Handle).tableStatsFromStorage(0xc420397500, 0xc42029eea0, 0x18fa5a0, 0xc42073bd40, 0x1)
        /go/src/github.com/pingcap/tidb/statistics/table.go:123 +0x8c7
github.com/pingcap/tidb/statistics.(*Handle).Update(0xc420397500, 0x7fdbd809c208, 0xc4202c4750, 0xc420397500, 0xc4205b1bf0)
        /go/src/github.com/pingcap/tidb/statistics/handle.go:110 +0x344
github.com/pingcap/tidb/domain.(*Domain).UpdateTableStatsLoop(0xc4204603c0, 0x18e47c0, 0xc42038a500, 0x0, 0x0)
        /go/src/github.com/pingcap/tidb/domain/domain.go:554 +0xe2
github.com/pingcap/tidb.BootstrapSession(0x18e2b60, 0xc4204141c0, 0x18e2b60, 0xc4204141c0, 0x0)
        /go/src/github.com/pingcap/tidb/session.go:996 +0x126
main.createStoreAndDomain()
        /go/src/github.com/pingcap/tidb/tidb-server/main.go:154 +0x162
main.main()
        /go/src/github.com/pingcap/tidb/tidb-server/main.go:129 +0x6c 
Contributor

Darren commented Oct 12, 2017

@lamxTyler

I tried commit bd89428, But it still reports error in our cluster, I added panic in util/types/mytime.go, here's the stacktrace, all the line numbers are correct as in bd89428 execept the first one.

Is it because there are bad boundaries in our mysql.stats_buckets?

panic: tm: -0001-11-30 00:00:00 +0800 CST, t:{year:0 month:0 day:0 hour:0 minute:0 second:0 microsecond:0} 


goroutine 1 [running]:
github.com/pingcap/tidb/util/types.mysqlTime.GoTime(0x0, 0x0, 0x0, 0x1afede0, 0x0, 0xd8, 0xc42097f778, 0x440fa0, 0xc42069ebf8)
        /go/src/github.com/pingcap/tidb/util/types/mytime.go:104 +0x53d
github.com/pingcap/tidb/util/types.(*mysqlTime).GoTime(0xc424abdaa0, 0x1afede0, 0xc42097f738, 0x7ffe787fea63, 0xc42097f778, 0x40568c, 0xffb620)
        <autogenerated>:63 +0x60
github.com/pingcap/tidb/util/types.(*Time).Sub(0xc42097f7f8, 0xc42097f828, 0x1, 0x0)
        /go/src/github.com/pingcap/tidb/util/types/time.go:505 +0xf4
github.com/pingcap/tidb/statistics.convertDatumToScalar(0xc42097fa48, 0x0, 0x1140f00)
        /go/src/github.com/pingcap/tidb/statistics/scalar.go:95 +0x409
github.com/pingcap/tidb/statistics.preCalculateDatumScalar(0xc42097fa48, 0xc42097fa10, 0xc420aa1b90, 0xd, 0x0)
        /go/src/github.com/pingcap/tidb/statistics/scalar.go:51 +0x8e
github.com/pingcap/tidb/statistics.histogramFromStorage(0x18e47c0, 0xc42038a500, 0x35e, 0x2, 0xc420aa1b90, 0x1, 0x0, 0x57c3d569ed80002, 0x0, 0xc424b3e940, ...)
        /go/src/github.com/pingcap/tidb/statistics/histogram.go:146 +0x4b9
github.com/pingcap/tidb/statistics.(*Handle).tableStatsFromStorage(0xc420397500, 0xc42029eea0, 0x18fa5a0, 0xc42073bd40, 0x1)
        /go/src/github.com/pingcap/tidb/statistics/table.go:123 +0x8c7
github.com/pingcap/tidb/statistics.(*Handle).Update(0xc420397500, 0x7fdbd809c208, 0xc4202c4750, 0xc420397500, 0xc4205b1bf0)
        /go/src/github.com/pingcap/tidb/statistics/handle.go:110 +0x344
github.com/pingcap/tidb/domain.(*Domain).UpdateTableStatsLoop(0xc4204603c0, 0x18e47c0, 0xc42038a500, 0x0, 0x0)
        /go/src/github.com/pingcap/tidb/domain/domain.go:554 +0xe2
github.com/pingcap/tidb.BootstrapSession(0x18e2b60, 0xc4204141c0, 0x18e2b60, 0xc4204141c0, 0x0)
        /go/src/github.com/pingcap/tidb/session.go:996 +0x126
main.createStoreAndDomain()
        /go/src/github.com/pingcap/tidb/tidb-server/main.go:154 +0x162
main.main()
        /go/src/github.com/pingcap/tidb/tidb-server/main.go:129 +0x6c 
@lamxTyler

This comment has been minimized.

Show comment
Hide comment
@lamxTyler

lamxTyler Oct 12, 2017

Member

@Darren
It seems that when we are in the strict mode, we could make the time zero, while in MySQL this can not happen, this is another problem as described in this issue.
But when we are in the non-strict mode, this is perfectly legal to make the time zero, but since this will only happen when the stats get updated, maybe just log it will not cause a problem.

Member

lamxTyler commented Oct 12, 2017

@Darren
It seems that when we are in the strict mode, we could make the time zero, while in MySQL this can not happen, this is another problem as described in this issue.
But when we are in the non-strict mode, this is perfectly legal to make the time zero, but since this will only happen when the stats get updated, maybe just log it will not cause a problem.

@Darren

This comment has been minimized.

Show comment
Hide comment
@Darren

Darren Oct 12, 2017

Contributor

@lamxTyler
Got it. I retstarted our tidb-server and no longer see the error again!

Contributor

Darren commented Oct 12, 2017

@lamxTyler
Got it. I retstarted our tidb-server and no longer see the error again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment