Skip to content

Fix config resolution for pip installed maxtext#3356

Merged
copybara-service[bot] merged 1 commit intomainfrom
fix-pypi-config-resolve
Mar 10, 2026
Merged

Fix config resolution for pip installed maxtext#3356
copybara-service[bot] merged 1 commit intomainfrom
fix-pypi-config-resolve

Conversation

@dipannita08
Copy link
Copy Markdown
Collaborator

@dipannita08 dipannita08 commented Mar 9, 2026

Description

When we run maxtext workload with pip-installed maxtext, the config path does not get resolved correctly resulting in the need to clone MaxText to pass config when running with pip.

The resolve_config_path now handles this case, by resolving the config file using MAXTEXT_CONFIGS_DIR.

The rest of the description includes relevant details and context, examples:

  • why is this change being made,
  • the problem being solved and any relevant context,
  • why this is a good solution,
  • some information about the specific implementation,
  • shortcomings of the solution and possible future improvements.

If the change fixes a bug or a Github issue, please include a link, e.g.,:
FIXES: b/491156065

Tests

  • CI
  • Build a test wheel and install in a tmp dir with no src/ folder:
uv pip install --python ~/maxtext/maxtext_venv_2/bin/python \
  "$(ls ~/maxtext/dist/maxtext-*.whl)[tpu]"
Using Python 3.12.13 environment at: /home/dishaw_google_com/maxtext/maxtext_venv_2
Resolved 263 packages in 783ms
      Built pylatexenc==2.10
Prepared 139 packages in 30.81s
Installed 262 packages in 1.23s

/tmp/maxtext_test$ ~/maxtext/maxtext_venv_2/bin/python3 -m maxtext.trainers.pre_train.train src/maxtext/configs/base.yml \
  run_name=dishaw_test_pip_install \
  base_output_directory=$OUTPUT_PATH \
  dataset_type=synthetic \
  steps=3

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

Copy link
Copy Markdown
Collaborator

@richjames0 richjames0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 9, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Copy link
Copy Markdown
Collaborator

@bvandermoon bvandermoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thank you @dipannita08

@copybara-service copybara-service Bot merged commit d2c172a into main Mar 10, 2026
49 of 50 checks passed
@copybara-service copybara-service Bot deleted the fix-pypi-config-resolve branch March 10, 2026 00:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants