Skip to content

Upgrade with Step Degradation Fix + Re-enable Goodput Monitoring#1341

Merged
changlan merged 1 commit into
apple:mainfrom
dipannita08:gp-integration-v15
Oct 3, 2025
Merged

Upgrade with Step Degradation Fix + Re-enable Goodput Monitoring#1341
changlan merged 1 commit into
apple:mainfrom
dipannita08:gp-integration-v15

Conversation

@dipannita08
Copy link
Copy Markdown
Contributor

This PR bumps the ml-goodput-measurement package version to 0.0.15 to integrate a critical performance fix for the GoodputMonitor.

Context
This release contains a major refactor GoodputMonitor to use multiprocessing throughout all APIs, instead of multithreading.

A step time degradation caused on a previous run caused by Python's Global Interpreter Lock (GIL) starving the main process due to the GoodputMonitor's tight-computation loops when there are multiple restarts is resolved with this fix.

Tests:

  • Example runs 1, 2
  • Dashboards for run 1:
image image

@dipannita08 dipannita08 requested a review from a team as a code owner September 16, 2025 01:55
@dipannita08 dipannita08 requested a review from a team as a code owner September 24, 2025 19:22
@dipannita08 dipannita08 changed the title Update Goodput Package to v15 Re-enable Goodput Monitoring + Upgrade with Step Degradation Fix Sep 24, 2025
@dipannita08 dipannita08 changed the title Re-enable Goodput Monitoring + Upgrade with Step Degradation Fix Upgrade with Step Degradation Fix + Re-enable Goodput Monitoring Sep 24, 2025
Comment thread axlearn/cloud/gcp/measurement.py
Comment thread axlearn/cloud/gcp/measurement.py
@Ethanlm Ethanlm added the ready-to-merge Ready to merge after clearing all the reviews. label Oct 3, 2025
@changlan changlan merged commit 75fbcb1 into apple:main Oct 3, 2025
6 checks passed
ds-hwang pushed a commit that referenced this pull request Oct 30, 2025
ORIGINAL_AUTHOR=Dipannita Shaw <dishaw@google.com>
COPYBARA_INTEGRATE_REVIEW=#1341 from dipannita08:gp-integration-v15 52064d4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Ready to merge after clearing all the reviews.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants