Skip to content

Cluster dump now excludes run_spec by default#5803

Merged
crusaderky merged 1 commit intodask:mainfrom
fjetter:fix_cluster_dump_exclude
Feb 15, 2022
Merged

Cluster dump now excludes run_spec by default#5803
crusaderky merged 1 commit intodask:mainfrom
fjetter:fix_cluster_dump_exclude

Conversation

@fjetter
Copy link
Copy Markdown
Member

@fjetter fjetter commented Feb 14, 2022

  1. Fix a bug where the exclude flag was not respected for the scheduler
  2. Align usage of run_spec attribute name in worker to scheduler. I consider this private API and don't intend to implement a deprecation mechanism
  3. Ignore run_spec by default for dump_cluster since this includes pickled user functions which might be considered sensitive, is typically useless for debugging and increases dump size significantly.
  4. I added type annotations for worker.TaskState while I was at it

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Feb 14, 2022

Unit Test Results

       18 files  +       2         18 suites  +2   10h 38m 1s ⏱️ + 1h 9m 21s
  2 597 tests ±       0    2 517 ✔️ ±       0       80 💤 ±    0  0 ±0 
23 239 runs  +2 583  21 802 ✔️ +2 447  1 437 💤 +136  0 ±0 

Results for commit f1be76d. ± Comparison against base commit f25f1fc.

♻️ This comment has been updated with latest results.

@fjetter
Copy link
Copy Markdown
Member Author

fjetter commented Feb 15, 2022

Test failures are all unrelated

@fjetter fjetter requested a review from crusaderky February 15, 2022 13:41
@fjetter fjetter force-pushed the fix_cluster_dump_exclude branch from de4ed0b to f1be76d Compare February 15, 2022 13:41
@crusaderky crusaderky merged commit 4ad02a0 into dask:main Feb 15, 2022
@fjetter fjetter deleted the fix_cluster_dump_exclude branch February 15, 2022 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants