[8.19](backport #49980) Fix race condition in multiline reader shutdown and other tests

[mergify]

Proposed commit message

The following tests had race conditions in the test code:
- TestFileWatcher
- TestProspectorHarvesterUpdateIgnoredFiles

TestParsersRabbitMQMultilineLog was failing with the race detector due
to an race condition during shutdown. The race condition is fixed by
adding a mutex and a benchmark (BenchmarkPatternReaderRabbitMQLike) is
added to ensure there is no performance regression.

Benchmark results from BenchmarkPatternReaderRabbitMQLike
Benchstat output:
goos: linux
goarch: amd64
pkg: github.com/elastic/beats/v7/libbeat/reader/multiline
cpu: Intel(R) Core(TM) Ultra 9 285H
                             │ before-10.txt │            after-10.txt            │
                             │    sec/op     │   sec/op     vs base               │
PatternReaderRabbitMQLike-16     750.5µ ± 7%   714.8µ ± 5%  -4.75% (p=0.035 n=10)

                             │ before-10.txt │            after-10.txt             │
                             │      B/s      │     B/s       vs base               │
PatternReaderRabbitMQLike-16    63.11Mi ± 7%   66.26Mi ± 5%  +4.99% (p=0.035 n=10)

                             │ before-10.txt │          after-10.txt          │
                             │     B/op      │     B/op      vs base          │
PatternReaderRabbitMQLike-16    557.6Ki ± 0%   557.8Ki ± 0%  ~ (p=0.579 n=10)

                             │ before-10.txt │          after-10.txt           │
                             │   allocs/op   │  allocs/op   vs base            │
PatternReaderRabbitMQLike-16     5.680k ± 0%   5.680k ± 0%  ~ (p=1.000 n=10) ¹
¹ all samples are equal

Before the changes:
goos: linux
goarch: amd64
pkg: github.com/elastic/beats/v7/libbeat/reader/multiline
cpu: Intel(R) Core(TM) Ultra 9 285H
BenchmarkPatternReaderRabbitMQLike-16    	    1590	    756970 ns/op	  65.61 MB/s	  570284 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    2132	    726478 ns/op	  68.36 MB/s	  570434 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    1486	    738715 ns/op	  67.23 MB/s	  570981 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    1982	    642365 ns/op	  77.31 MB/s	  571042 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    1344	    753941 ns/op	  65.87 MB/s	  570748 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    1504	    774597 ns/op	  64.12 MB/s	  571365 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    1844	    805264 ns/op	  61.67 MB/s	  571732 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    1579	    729805 ns/op	  68.05 MB/s	  570959 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    1574	    818257 ns/op	  60.69 MB/s	  571389 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    1419	    747003 ns/op	  66.48 MB/s	  571037 B/op	    5680 allocs/op
PASS
ok  	github.com/elastic/beats/v7/libbeat/reader/multiline	12.369s

After the changes:
goos: linux
goarch: amd64
pkg: github.com/elastic/beats/v7/libbeat/reader/multiline
cpu: Intel(R) Core(TM) Ultra 9 285H
BenchmarkPatternReaderRabbitMQLike-16    	    2268	    742420 ns/op	  66.89 MB/s	  571058 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    1586	    730727 ns/op	  67.97 MB/s	  571797 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    2067	    699545 ns/op	  70.99 MB/s	  570589 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    1742	    717458 ns/op	  69.22 MB/s	  570487 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    1569	    786419 ns/op	  63.15 MB/s	  571493 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    1429	    703584 ns/op	  70.59 MB/s	  570798 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    1802	    680524 ns/op	  72.98 MB/s	  571433 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    1714	    712191 ns/op	  69.73 MB/s	  571443 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    1783	    722538 ns/op	  68.74 MB/s	  570243 B/op	    5680 allocs/op
BenchmarkPatternReaderRabbitMQLike-16    	    1624	    679507 ns/op	  73.09 MB/s	  571222 B/op	    5680 allocs/op
PASS
ok  	github.com/elastic/beats/v7/libbeat/reader/multiline	12.648s

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

## Disruptive User Impact

How to test this PR locally

Runt the tests multiple times with the race detector

go test -race -count=5 -tags=integration -run="TestFileWatcher|TestProspectorHarvesterUpdateIgnoredFiles|TestParsersRabbitMQMultilineLog" .

## Related issues
## Use cases
## Screenshots
## Logs

---

This is an automatic backport of pull request #49980 done by Mergify.

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)
• /test : Run the Buildkite pipeline.

[infra-vault-gh-plugin-prod]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)