Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Submit collections metadata to scheduler #8612

Merged
merged 12 commits into from
May 22, 2024

Conversation

fjetter
Copy link
Member

@fjetter fjetter commented Apr 11, 2024

We're already collecting client local context, e.g. with code. This is some extention to this that enriches span information with the API being used. In the future, this could be expanded with other information but for now I just included the type of collection we're submitting

cc @phofl

Copy link
Contributor

github-actions bot commented Apr 11, 2024

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

    29 files  ±    0      29 suites  ±0   11h 11m 7s ⏱️ + 1h 14m 50s
 4 052 tests  -     4   3 947 ✅ +    8     97 💤  -   9  8 ❌ +1 
55 813 runs  +7 591  53 643 ✅ +7 359  2 161 💤 +248  9 ❌ +1 

For more details on these failures, see this check.

Results for commit ba2bb6f. ± Comparison against base commit 854a280.

This pull request removes 13 and adds 9 tests. Note that renamed tests count towards both.
distributed.protocol.tests.test_arrow
distributed.protocol.tests.test_collection
distributed.protocol.tests.test_highlevelgraph
distributed.protocol.tests.test_numpy
distributed.protocol.tests.test_pandas
distributed.shuffle.tests.test_graph
distributed.shuffle.tests.test_merge
distributed.shuffle.tests.test_merge_column_and_index
distributed.shuffle.tests.test_metrics
distributed.shuffle.tests.test_rechunk
…
distributed.diagnostics.tests.test_memray ‑ test_basic_integration_scheduler
distributed.diagnostics.tests.test_memray ‑ test_basic_integration_scheduler_report_args[False]
distributed.diagnostics.tests.test_memray ‑ test_basic_integration_scheduler_report_args[report_args0]
distributed.diagnostics.tests.test_memray ‑ test_basic_integration_workers[1]
distributed.diagnostics.tests.test_memray ‑ test_basic_integration_workers[False]
distributed.diagnostics.tests.test_memray ‑ test_basic_integration_workers[True]
distributed.diagnostics.tests.test_memray ‑ test_basic_integration_workers_report_args[False]
distributed.diagnostics.tests.test_memray ‑ test_basic_integration_workers_report_args[report_args0]
distributed.tests.test_spans ‑ test_collections_metadata

♻️ This comment has been updated with latest results.

Copy link
Collaborator

@phofl phofl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small comment, otherwise lgtm.

distributed/spans.py Outdated Show resolved Hide resolved
distributed/scheduler.py Outdated Show resolved Hide resolved
distributed/spans.py Outdated Show resolved Hide resolved
Copy link
Member

@hendrikmakait hendrikmakait left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @fjetter! Just one style nit.

distributed/client.py Outdated Show resolved Hide resolved
distributed/client.py Outdated Show resolved Hide resolved
distributed/client.py Outdated Show resolved Hide resolved
distributed/client.py Outdated Show resolved Hide resolved
distributed/client.py Outdated Show resolved Hide resolved
Co-authored-by: Hendrik Makait <hendrik.makait@gmail.com>
@hendrikmakait hendrikmakait merged commit b3f8fcd into dask:main May 22, 2024
26 of 34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants