-
Notifications
You must be signed in to change notification settings - Fork 451
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TGI 1.0.3 Release #3285
TGI 1.0.3 Release #3285
Conversation
@haixiw is there a plan to merge this PR in the near future? Is there a possibility of accessing the image before the official release? I was checking the image I that the PR - awslabs/llm-hosting-container#30 got already approved |
Yeah we are gonna release this in a week or so. Accessing image before official image is not allowed by the process, But let me know if it's urgently needed Thanks |
Would be great to have EAP. I created the AWS case as well for the same, but generally speaking this would be helpful to have earlier access. We found a workaround, but maybe we will hit again some limitations and will be forced to use TGI 1.0.3. So I will contact again. |
…3304) * update torchserve to 0.8.2 * retry failed tests * add allowlist for torch CVE * Trigger Build * trigger x86 tests * set datetime_tag = true * fix allowlists and run only sanity test * fix testTorchdata and only run ec2 test * revert testrunner.py and run whole ec2 test * revert graviton buildspec and test config --------- Co-authored-by: Sally Seok <sallyseo@amazon.com> Co-authored-by: Tejas Chumbalkar <34728580+tejaschumbalkar@users.noreply.github.com>
… 0.8.2 (#3316) * update torchserve to 0.8.2 and add allowlists * revert test config --------- Co-authored-by: Sally Seok <sallyseo@amazon.com> Co-authored-by: arjkesh <33526713+arjkesh@users.noreply.github.com>
…for Neuron release 2.13.2 (#3314) * Add Tensorflow inference images for release 2.13.2 * Fix file not updated and vulns * Fix ignore_ids json * Whitelist more scipy and grpcio vulns * Whitelist grpcio * Add collectives lib to neuronx image * Revert develop config
Co-authored-by: Sally Seok <sallyseo@amazon.com>
new PR:#3323 |
GitHub Issue #, if available:
Note:
If merging this PR should also close the associated Issue, please also add that Issue # to the Linked Issues section on the right.
All PR's are checked weekly for staleness. This PR will be closed if not updated in 30 days.
Description
Modifying the release images for TGI 1.0.2 for preparation.
Tests run
NOTE: By default, docker builds are disabled. In order to build your container, please update dlc_developer_config.toml and specify the framework to build in "build_frameworks"
NOTE: If you are creating a PR for a new framework version, please ensure success of the standard, rc, and efa sagemaker remote tests by updating the dlc_developer_config.toml file:
sagemaker_remote_tests = "standard"
sagemaker_remote_tests = "rc"
sagemaker_remote_tests = "efa"
Additionally, please run the sagemaker local tests in at least one revision:
sagemaker_local_tests = true
Formatting
black -l 100
on my code (formatting tool: https://black.readthedocs.io/en/stable/getting_started.html)DLC image/dockerfile
Additional context
PR Checklist
Pytest Marker Checklist
@pytest.mark.model("<model-type>")
to the new tests which I have added, to specify the Deep Learning model that is used in the test (use"N/A"
if the test doesn't use a model)@pytest.mark.integration("<feature-being-tested>")
to the new tests which I have added, to specify the feature that will be tested@pytest.mark.multinode(<integer-num-nodes>)
to the new tests which I have added, to specify the number of nodes used on a multi-node test@pytest.mark.processor(<"cpu"/"gpu"/"eia"/"neuron">)
to the new tests which I have added, if a test is specifically applicable to only one processor typeEIA/NEURON/GRAVITON Testing Checklist
dlc_developer_config.toml
in my PR branch by settingei_mode = true
,neuron_mode = true
orgraviton_mode = true
Benchmark Testing Checklist
dlc_developer_config.toml
in my PR branch by settingbenchmark_mode = true
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.