Support for unsetting environment variables

[crusaderky]

• Closes Unsetting MALLOC_TRIM_THRESHOLD_ is sensitive to import order #8143

[mrocklin]

cc @ntabris

[github-actions]

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

       21 files  ±  0         21 suites  ±0   10h 56m 23s ⏱️ - 11m 42s
  3 793 tests +  1    3 683 ✔️ +  2     107 💤 ±0  3 ❌  - 1 
36 667 runs  +22  34 865 ✔️ +28  1 799 💤  - 5  3 ❌  - 1 

For more details on these failures, see this check.

Results for commit 702f065. ± Comparison against base commit 9ba2eed.

♻️ This comment has been updated with latest results.

[hendrikmakait]

Thanks, @crusaderky!

[ntabris]

Just to be clear, this PR doesn't actually change the behavior of the code that (I think) was supposed to demonstrate the problem in #8143 (comment)

That's probably fine, but it's a little confusing.

In particular, dask.config.set({"distributed.nanny.pre-spawn-environ": {'OMP_NUM_THREADS': 2}}) won't prevent the default values of (eg) MALLOC_TRIM_THRESHOLD_, you'd need to explicitly say

dask.config.set({"distributed.nanny.pre-spawn-environ": {
  'OMP_NUM_THREADS': 2,
  'MALLOC_TRIM_THRESHOLD_': None
}})

[crusaderky]

Just to be clear, this PR doesn't actually change the behavior of the code that (I think) was supposed to demonstrate the problem in #8143 (comment)

That's probably fine, but it's a little confusing.

In particular, dask.config.set({"distributed.nanny.pre-spawn-environ": {'OMP_NUM_THREADS': 2}}) won't prevent the default values of (eg) MALLOC_TRIM_THRESHOLD_, you'd need to explicitly say

dask.config.set({"distributed.nanny.pre-spawn-environ": {
  'OMP_NUM_THREADS': 2,
  'MALLOC_TRIM_THRESHOLD_': None
}})

this is correct.

[crusaderky]

dask.config.set({"distributed.nanny.pre-spawn-environ": {
  'OMP_NUM_THREADS': 2,
  'MALLOC_TRIM_THRESHOLD_': None
}})

this is correct.

or just write

dask.config.set({"distributed.nanny.pre-spawn-environ.MALLOC_TRIM_THRESHOLD_': None})