test leak in TestInfo #14817

wjhuang2016 · 2020-02-17T05:34:25Z

FAIL:TestInfo

Bug Report

Please answer these questions before submitting your issue. Thanks!

What did you do?
If possible, provide a recipe for reproducing the error.
https://internal.pingcap.net/idc-jenkins/blue/rest/organizations/jenkins/pipelines/tidb_ghpr_unit_test/runs/24373/nodes/84/log/?start=0

[2020-02-17T03:08:06.587Z] === RUN   TestInfo
[2020-02-17T03:08:06.587Z] [2020/02/17 11:08:01.308 +08:00] [ERROR] [syncer.go:199] ["close session failed"] [] [stack="github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Done\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/ddl/util/syncer.go:199\ngithub.com/pingcap/tidb/domain.TestInfo\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/domain/domain_test.go:150\ntesting.tRunner\n\t/usr/local/go/src/testing/testing.go:909"]
[2020-02-17T03:08:06.587Z] [2020/02/17 11:08:01.311 +08:00] [ERROR] [syncer.go:199] ["close session failed"] [] [stack="github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Done\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/ddl/util/syncer.go:199\ngithub.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/domain/domain.go:485"]
[2020-02-17T03:08:06.587Z] [2020/02/17 11:08:01.314 +08:00] [ERROR] [syncer.go:199] ["close session failed"] [] [stack="github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Done\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/ddl/util/syncer.go:199\ngithub.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/domain/domain.go:485"]
[2020-02-17T03:08:06.587Z] [2020/02/17 11:08:01.318 +08:00] [ERROR] [syncer.go:199] ["close session failed"] [] [stack="github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Done\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/ddl/util/syncer.go:199\ngithub.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/domain/domain.go:485"]
[2020-02-17T03:08:06.587Z] [2020/02/17 11:08:01.321 +08:00] [ERROR] [syncer.go:199] ["close session failed"] [] [stack="github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Done\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/ddl/util/syncer.go:199\ngithub.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/domain/domain.go:485"]
[2020-02-17T03:08:06.587Z] [2020/02/17 11:08:01.324 +08:00] [ERROR] [syncer.go:199] ["close session failed"] [] [stack="github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Done\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/ddl/util/syncer.go:199\ngithub.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/domain/domain.go:485"]
[2020-02-17T03:08:06.587Z] [2020/02/17 11:08:01.327 +08:00] [ERROR] [syncer.go:199] ["close session failed"] [] [stack="github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Done\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/ddl/util/syncer.go:199\ngithub.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/domain/domain.go:485"]
[2020-02-17T03:08:06.587Z] [2020/02/17 11:08:01.330 +08:00] [ERROR] [syncer.go:199] ["close session failed"] [] [stack="github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Done\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/ddl/util/syncer.go:199\ngithub.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/domain/domain.go:485"]
[2020-02-17T03:08:06.588Z] [2020/02/17 11:08:01.334 +08:00] [ERROR] [syncer.go:199] ["close session failed"] [] [stack="github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Done\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/ddl/util/syncer.go:199\ngithub.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/domain/domain.go:485"]
[2020-02-17T03:08:06.588Z] [2020/02/17 11:08:01.337 +08:00] [ERROR] [syncer.go:199] ["close session failed"] [] [stack="github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Done\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/ddl/util/syncer.go:199\ngithub.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/domain/domain.go:485"]
[2020-02-17T03:08:06.588Z] [2020/02/17 11:08:01.340 +08:00] [ERROR] [syncer.go:199] ["close session failed"] [] [stack="github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Done\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/ddl/util/syncer.go:199\ngithub.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/domain/domain.go:485"]
[2020-02-17T03:08:06.588Z] [2020/02/17 11:08:01.343 +08:00] [ERROR] [syncer.go:199] ["close session failed"] [] [stack="github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Done\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/ddl/util/syncer.go:199\ngithub.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/domain/domain.go:485"]
[2020-02-17T03:08:06.588Z] [2020/02/17 11:08:01.346 +08:00] [ERROR] [syncer.go:199] ["close session failed"] [] [stack="github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Done\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/ddl/util/syncer.go:199\ngithub.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/domain/domain.go:485"]
[2020-02-17T03:08:06.588Z] [2020/02/17 11:08:01.349 +08:00] [ERROR] [syncer.go:199] ["close session failed"] [] [stack="github.com/pingcap/tidb/ddl/util.(*schemaVersionSyncer).Done\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/ddl/util/syncer.go:199\ngithub.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/domain/domain.go:485"]
[2020-02-17T03:08:06.588Z] {"level":"warn","ts":"2020-02-17T11:08:01.350+0800","caller":"clientv3/retry_interceptor.go:61","msg":"retrying of unary invoker failed","target":"endpoint://client-516b5633-69d2-49d2-a3a5-8ed7da21c073/localhost:1525069566242647020","attempt":0,"error":"rpc error: code = NotFound desc = etcdserver: requested lease not found"}
[2020-02-17T03:08:06.588Z] --- FAIL: TestInfo (2.75s)
[2020-02-17T03:08:06.588Z]     leaktest.go:143: Test TestInfo check-count 50 appears to have leaked: github.com/pingcap/tidb/domain.(*Domain).loadSchemaInLoop(0xc0006026c0, 0x2faf080)
[2020-02-17T03:08:06.588Z]         	/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/domain/domain.go:469 +0x269
[2020-02-17T03:08:06.588Z]         created by github.com/pingcap/tidb/domain.(*Domain).Init
[2020-02-17T03:08:06.588Z]         	/home/jenkins/agent/workspace/tidb_ghpr_unit_test@2/go/src/github.com/pingcap/tidb/domain/domain.go:696 +0x59c
[2020-02-17T03:08:06.588Z] FAIL

What did you expect to see?
What did you see instead?
What version of TiDB are you using (tidb-server -V or run select tidb_version(); on TiDB)?

The text was updated successfully, but these errors were encountered:

SunRunAway · 2020-02-28T12:09:42Z

another occurence
https://internal.pingcap.net/idc-jenkins/blue/rest/organizations/jenkins/pipelines/tidb_ghpr_unit_test/runs/25629/nodes/84/steps/368/log/?start=0

SunRunAway · 2020-03-10T12:31:05Z

+1
https://internal.pingcap.net/idc-jenkins/blue/organizations/jenkins/tidb_ghpr_unit_test/detail/tidb_ghpr_unit_test/26968/pipeline

SunRunAway · 2020-03-10T12:31:18Z

@wjhuang2016 @bb7133 PTAL

SunRunAway · 2020-03-16T05:16:32Z

Another error message.

[2020-03-16T05:13:27.940Z] --- FAIL: TestInfo (2.74s)
[2020-03-16T05:13:27.940Z]     leaktest.go:143: Test TestInfo check-count 50 appears to have leaked: go.etcd.io/etcd/clientv3.waitRetryBackoff(0x2f8ff00, 0xc000511a40, 0x5d, 0xc000ae0660, 0x3, 0x3)
[2020-03-16T05:13:27.940Z]         	/home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/go.etcd.io/etcd@v0.5.0-alpha.5.0.20191023171146-3cf2f69b5738/clientv3/retry_interceptor.go:267 +0xfd
[2020-03-16T05:13:27.940Z]         go.etcd.io/etcd/clientv3.(*Client).unaryClientInterceptor.func1(0x2f8ff00, 0xc000511a40, 0x2bddb89, 0x1f, 0x2a127a0, 0xc001a331a0, 0x29ff480, 0xc00167c1e8, 0xc0004d7180, 0x2c64820, ...)
[2020-03-16T05:13:27.940Z]         	/home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/go.etcd.io/etcd@v0.5.0-alpha.5.0.20191023171146-3cf2f69b5738/clientv3/retry_interceptor.go:49 +0x14f
[2020-03-16T05:13:27.940Z]         google.golang.org/grpc.(*ClientConn).Invoke(0xc0004d7180, 0x2f8ff00, 0xc000511a40, 0x2bddb89, 0x1f, 0x2a127a0, 0xc001a331a0, 0x29ff480, 0xc00167c1e8, 0xc000511aa0, ...)
[2020-03-16T05:13:27.940Z]         	/home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/google.golang.org/grpc@v1.25.1/call.go:35 +0x109
[2020-03-16T05:13:27.940Z]         google.golang.org/grpc.Invoke(...)
[2020-03-16T05:13:27.940Z]         	/home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/google.golang.org/grpc@v1.25.1/call.go:60
[2020-03-16T05:13:27.940Z]         go.etcd.io/etcd/etcdserver/etcdserverpb.(*leaseClient).LeaseRevoke(0xc0002dcbd0, 0x2f8ff00, 0xc000511a40, 0xc001a331a0, 0xc000511aa0, 0x4, 0x6, 0x6, 0xf7b4c8, 0x10)
[2020-03-16T05:13:27.940Z]         	/home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/go.etcd.io/etcd@v0.5.0-alpha.5.0.20191023171146-3cf2f69b5738/etcdserver/etcdserverpb/rpc.pb.go:3801 +0xcf
[2020-03-16T05:13:27.940Z]         go.etcd.io/etcd/clientv3.(*retryLeaseClient).LeaseRevoke(0xc0020744c0, 0x2f8ff00, 0xc000511a40, 0xc001a331a0, 0x47dec00, 0x3, 0x3, 0x2f8ff00, 0xc000511a40, 0xc001949fe0)
[2020-03-16T05:13:27.940Z]         	/home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/go.etcd.io/etcd@v0.5.0-alpha.5.0.20191023171146-3cf2f69b5738/clientv3/retry.go:148 +0xeb
[2020-03-16T05:13:27.940Z]         go.etcd.io/etcd/clientv3.(*lessor).Revoke(0xc000c83cc0, 0x2f8ff00, 0xc000511a40, 0x3feb70e1c1edc705, 0xc000511a40, 0xc001949fe0, 0x19)
[2020-03-16T05:13:27.940Z]         	/home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/go.etcd.io/etcd@v0.5.0-alpha.5.0.20191023171146-3cf2f69b5738/clientv3/lease.go:231 +0x92
[2020-03-16T05:13:27.940Z]         github.com/pingcap/tidb/owner.(*ownerManager).revokeSession(0xc000c45c70, 0xc0002bb6d0, 0x4a, 0x3feb70e1c1edc705)
[2020-03-16T05:13:27.940Z]         	/home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/owner/manager.go:290 +0xaa
[2020-03-16T05:13:27.940Z]         github.com/pingcap/tidb/owner.(*ownerManager).campaignLoop(0xc000c45c70, 0x2f8fe80, 0xc001888a80, 0xc00120f020)
[2020-03-16T05:13:27.940Z]         	/home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/owner/manager.go:244 +0x10b7
[2020-03-16T05:13:27.940Z]         created by github.com/pingcap/tidb/owner.(*ownerManager).CampaignOwner
[2020-03-16T05:13:27.940Z]         	/home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/owner/manager.go:186 +0x1b0
[2020-03-16T05:13:27.940Z] FAIL
[2020-03-16T05:13:27.940Z] FAIL	github.com/pingcap/tidb/domain	3.185s

https://internal.pingcap.net/idc-jenkins/blue/organizations/jenkins/tidb_ghpr_unit_test/detail/tidb_ghpr_unit_test/27672/pipeline

Seen at #15194 (comment)