Skip to content

Xpk system files#7

Closed
Obliviour wants to merge 27 commits into
mainfrom
xpk-system-files
Closed

Xpk system files#7
Obliviour wants to merge 27 commits into
mainfrom
xpk-system-files

Conversation

@Obliviour
Copy link
Copy Markdown
Collaborator

Fixes / Features

Testing / Documentation

Testing details.

  • [ y/n ] Tests pass
  • [ y/n ] Appropriate changes to documentation are included in the PR

gobbleturk and others added 27 commits November 2, 2023 00:13
Add Ahead of Time Compilation functionality to maxtext with train_compile.py
[xpk] Incorporate xpk into maxtext directory for open source access

Tested all commands listed in the README. Run commands from maxtext home directory.
Allow kueue setup to retry up to 5 times in cluster create
Update default docker image to python:3.10 since python:3.8 is deprec…
CacheImage now explicitly selects set cluster.
Correctly error if the project name or zone name is unset
[xpk] Use kueue github repo manifest file instead of local one
* initial commit

* add gke_accelerator type for v4

* remove unused import

* run pylint on xpk

* add command line flags

* fix nits

* add license

* fix pylint

* better function name

* run pylint on all python files

* support all v4 slices

* remove cluster type flag

* revert --cluster-type flag changes

* minor fixes
Modify workload name checking to not look for prefix but match names
* [xpk] Add deletion prompt and force flag on cluster create

TESTING=yes

[y] checked that cluster create with deletion returns the prmpt
[y] y / yes continues the prompt
[y] n / no stops the deletion
[y] other words stops the deletion
[y] force skips the prompt

* add a log message when delete is skipped.

* Add pylint changes
* [xpk] Support base-docker-image and script-dir

- Adds support to build local files into the docker image
- Adds readme example for how to do this
- Continues to support docker-image for not building local files into the image.

Tested
- Ran examples with local file changes
- Verified that directory path has to be valid
- Verified that invalid group of arguments don't work

* Improve documentation and add datetime to the dockertag

* Modify documentation

* Additional documentation nits

* docker documentation
…rom appearing when no nodepools are to be deleted. (#211)

* Update Readme with clearer organization

* Avoid asking delete prompt when no node pools are to be deleted
If python version is < 3.10, return error message.

Tested: Yes.
* Remove unnecessary pylints and fix code lint errors

* Add with commands
* Add large scale guide to xpk repository

* Readme details on large scale

* add details about vm size needed for different limits and spelling
Add Ahead of Time Compilation functionality to maxtext with train_compile.py
@google-cla
Copy link
Copy Markdown

google-cla Bot commented Nov 2, 2023

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@Obliviour Obliviour closed this Nov 2, 2023
@Obliviour Obliviour deleted the xpk-system-files branch November 2, 2023 20:38
ultrons added a commit to ultrons/xpk that referenced this pull request May 4, 2026
Round-3 review AI-Hypercomputer#6/AI-Hypercomputer#7. Match parser/common.py:71's pattern: each
add_argument(...).completer = X is now a single chain expression at
the call site instead of going through a helper that wrapped setattr.

Mypy stays clean by relaxing the parameter type on the two enclosing
parser-setup functions (set_workload_delete_parser and
set_workload_list_parser) to match this file's existing convention
in add_shared_workload_create_required_arguments — untyped parser
param so Any.add_argument().completer = X works for mypy.

Removes the _set_completer wrapper, its docstring, and the typing.Any
need at that boundary.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants