Add locking mechanism for tenant monitoring probabilistic approach

[gokhangulbiz]

This PR

• Addresses a concurrency issue in the probabilistic approach of tenant monitoring by acquiring a shared lock for tenant existence checks.
• Changes citus.stat_tenants_sample_rate_for_new_tenants type to double
• Renames citus.stat_tenants_sample_rate_for_new_tenants to citus.stat_tenants_untracked_sample_rate

[JelteF]

No need seed the random generator. It's already done by Postgres at process start.

[JelteF]

See the InitProcessGlobals function in case you're interested.

[JelteF]

Realised this did not have test yet.

[JelteF]

Name needs some PM input.

[JelteF]

nit: unnecessary newline

[codecov]

Codecov Report

Merging #7026 (32a3414) into main (ac24e11) will decrease coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main    #7026      +/-   ##
==========================================
- Coverage   93.34%   93.32%   -0.02%     
==========================================
  Files         273      273              
  Lines       58511    58514       +3     
==========================================
- Hits        54618    54611       -7     
- Misses       3893     3903      +10     

[JelteF]

I think this is good, with the exception that we still need a decision on the name of the GUC.

[JelteF]

LGTM, but somehow the test is still a little bit flaky. I also don't see obvious reasons why that is happening though. The only thing that might help is adding a SELECT pg_sleep(0.1); after the pg_reload_conf(). We needed that in a few other tests because pg_reload_conf() is asynchronous. We already sleep here until the next period, but that might be to short if the new period is almost starting.

If that still doesn't help let's remove the final part of the test where the sample rate is 1. It's already tested earlier in this test file implicitly anyway.