TiDB CI data race (once)

[LittleFall]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

see:
https://internal.pingcap.net/idc-jenkins/blue/rest/organizations/jenkins/pipelines/tidb_ghpr_unit_test/runs/67419/nodes/82/steps/323/log/?start=0

2. What did you expect to see? (Required)

test success

3. What did you see instead (Required)

(*ParallelNestedLoopApplyExec).innerWorker\n\t/home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/parallel_apply.go:238"]
[2021-03-12T07:05:37.307Z] ==================
[2021-03-12T07:05:37.307Z] WARNING: DATA RACE
[2021-03-12T07:05:37.308Z] Write at 0x00c08f9bf300 by goroutine 332:
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/statistics.(*QueryFeedback).Update()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/statistics/feedback.go:256 +0x8e4
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/distsql.(*selectResult).fetchResp()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/distsql/select_result.go:153 +0xaae
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/distsql.(*selectResult).Next()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/distsql/select_result.go:180 +0x2b3
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/executor.(*tableResultHandler).nextChunk()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/table_reader.go:329 +0xaa
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/executor.(*TableReaderExecutor).Next()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/table_reader.go:206 +0x21e
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/executor.Next()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/executor.go:277 +0x27d
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/executor.(*ParallelNestedLoopApplyExec).outerWorker()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/parallel_apply.go:202 +0x5a9
[2021-03-12T07:05:37.308Z] 
[2021-03-12T07:05:37.308Z] Previous read at 0x00c08f9bf300 by goroutine 407:
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/session.(*session).StoreQueryFeedback()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:372 +0x92e
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/executor.(*TableReaderExecutor).Close()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/table_reader.go:226 +0x121
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/executor.(*ParallelNestedLoopApplyExec).Close()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/parallel_apply.go:162 +0xa8
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/executor.(*baseExecutor).Close()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/executor.go:172 +0xad
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/executor.(*ProjectionExec).Close()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/projection.go:320 +0x255
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/executor.(*recordSet).Close()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/adapter.go:166 +0x59
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/session.(*execStmtResult).Close()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/session/session.go:1565 +0x7b
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/util/testkit.(*TestKit).QueryToErr()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/util/testkit/testkit.go:306 +0x56a
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/executor_test.(*testSuite).TestApplyGoroutinePanic()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/parallel_apply_test.go:587 +0x546
[2021-03-12T07:05:37.308Z]   runtime.call32()
[2021-03-12T07:05:37.308Z]       /usr/local/go/src/runtime/asm_amd64.s:539 +0x3a
[2021-03-12T07:05:37.308Z]   reflect.Value.Call()
[2021-03-12T07:05:37.308Z]       /usr/local/go/src/reflect/value.go:321 +0xd3
[2021-03-12T07:05:37.308Z]   github.com/pingcap/check.(*suiteRunner).forkTest.func1()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:850 +0x9aa
[2021-03-12T07:05:37.308Z]   github.com/pingcap/check.(*suiteRunner).forkCall.func1()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:739 +0x113
[2021-03-12T07:05:37.308Z] 
[2021-03-12T07:05:37.308Z] Goroutine 332 (running) created at:
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/executor.(*ParallelNestedLoopApplyExec).Next()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/parallel_apply.go:137 +0xdb
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/executor.Next()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/executor.go:277 +0x27d
[2021-03-12T07:05:37.308Z]   github.com/pingcap/tidb/executor.(*projectionInputFetcher).run()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/src/github.com/pingcap/tidb/executor/projection.go:373 +0x48f
[2021-03-12T07:05:37.308Z] 
[2021-03-12T07:05:37.308Z] Goroutine 407 (running) created at:
[2021-03-12T07:05:37.308Z]   github.com/pingcap/check.(*suiteRunner).forkCall()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:734 +0x4a3
[2021-03-12T07:05:37.308Z]   github.com/pingcap/check.(*suiteRunner).forkTest()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:832 +0x1b9
[2021-03-12T07:05:37.308Z]   github.com/pingcap/check.(*suiteRunner).doRun()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:666 +0x13a
[2021-03-12T07:05:37.308Z]   github.com/pingcap/check.(*suiteRunner).asyncRun.func1()
[2021-03-12T07:05:37.308Z]       /home/jenkins/agent/workspace/tidb_ghpr_unit_test/go/pkg/mod/github.com/pingcap/check@v0.0.0-20200212061837-5e12011dc712/check.go:650 +0xf7
[2021-03-12T07:05:37.308Z] ==================

4. What is your TiDB version? (Required)

2021.3.12 master

[guo-shaoge]

/assign

[guo-shaoge]

As @eurekaka suggested, this data race happens when Apply operator panic and its' Close() is called, which will read feedback. At the same time, its outer worker(TableReaderExecutor) runs normally, which will write feedback.

So we may need to wait Apply operator's outer worker to finish, then read feedback.

[ti-srebot]

Please edit this comment or add a new comment to complete the following information

Not a bug

1. Remove the 'type/bug' label
2. Add notes to indicate why it is not a bug

Duplicate bug

1. Add the 'type/duplicate' label
2. Add the link to the original bug

Bug

Note: Make Sure that 'component', and 'severity' labels are added
Example for how to fill out the template: #20100

1. Root Cause Analysis (RCA) (optional)

2. Symptom (optional)

3. All Trigger Conditions (optional)

4. Workaround (optional)

5. Affected versions

6. Fixed versions