Skip to content

Xpk merge from maxtext final#5

Merged
Obliviour merged 23 commits into
mainfrom
xpk-merge-from-maxtext-final
Nov 2, 2023
Merged

Xpk merge from maxtext final#5
Obliviour merged 23 commits into
mainfrom
xpk-merge-from-maxtext-final

Conversation

@Obliviour
Copy link
Copy Markdown
Collaborator

Fixes / Features

  • Merge final PRs from maxtext xpk into xpk.

Testing / Documentation

  • None done since changes are pulled from previously tested components.

gobbleturk and others added 23 commits November 2, 2023 00:13
Add Ahead of Time Compilation functionality to maxtext with train_compile.py
[xpk] Incorporate xpk into maxtext directory for open source access

Tested all commands listed in the README. Run commands from maxtext home directory.
Allow kueue setup to retry up to 5 times in cluster create
Update default docker image to python:3.10 since python:3.8 is deprec…
CacheImage now explicitly selects set cluster.
Correctly error if the project name or zone name is unset
[xpk] Use kueue github repo manifest file instead of local one
* initial commit

* add gke_accelerator type for v4

* remove unused import

* run pylint on xpk

* add command line flags

* fix nits

* add license

* fix pylint

* better function name

* run pylint on all python files

* support all v4 slices

* remove cluster type flag

* revert --cluster-type flag changes

* minor fixes
Modify workload name checking to not look for prefix but match names
* [xpk] Add deletion prompt and force flag on cluster create

TESTING=yes

[y] checked that cluster create with deletion returns the prmpt
[y] y / yes continues the prompt
[y] n / no stops the deletion
[y] other words stops the deletion
[y] force skips the prompt

* add a log message when delete is skipped.

* Add pylint changes
* [xpk] Support base-docker-image and script-dir

- Adds support to build local files into the docker image
- Adds readme example for how to do this
- Continues to support docker-image for not building local files into the image.

Tested
- Ran examples with local file changes
- Verified that directory path has to be valid
- Verified that invalid group of arguments don't work

* Improve documentation and add datetime to the dockertag

* Modify documentation

* Additional documentation nits

* docker documentation
…rom appearing when no nodepools are to be deleted. (#211)

* Update Readme with clearer organization

* Avoid asking delete prompt when no node pools are to be deleted
If python version is < 3.10, return error message.

Tested: Yes.
* Remove unnecessary pylints and fix code lint errors

* Add with commands
* Add large scale guide to xpk repository

* Readme details on large scale

* add details about vm size needed for different limits and spelling
Add Ahead of Time Compilation functionality to maxtext with train_compile.py
@google-cla
Copy link
Copy Markdown

google-cla Bot commented Nov 2, 2023

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@Obliviour
Copy link
Copy Markdown
Collaborator Author

a8f042e Author: @a-googler <no****ly​@google.com> is an automated googler. I don't think the CLA is needed.

@Obliviour Obliviour merged commit b2b7b57 into main Nov 2, 2023
@Obliviour Obliviour deleted the xpk-merge-from-maxtext-final branch November 2, 2023 00:41
jamOne- added a commit to jamOne-/xpk that referenced this pull request Feb 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants