fix failed to create model archive #1508

lxning · 2022-03-15T01:41:33Z

Description

Please include a summary of the feature or issue being fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes #(issue)

Type of change

#1498

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

Feature/Issue validation/testing

Please describe the tests [UT/IT] that you ran to verify your changes and relevant result summary. Provide instructions so it can be reproduced.
Please also list any relevant details for your test configuration.

Test A
Test B
Logs

config.properties

inference_address=http://0.0.0.0:8080
management_address=http://0.0.0.0:8081

number_of_netty_threads=32
job_queue_size=1000

vmargs=-Xmx4g -XX:+ExitOnOutOfMemoryError -XX:+HeapDumpOnOutOfMemoryError
prefer_direct_buffer=True

default_response_timeout=300
unregister_model_timeout=300
install_py_dep_per_model=true
default_service_handler=./model_store/noop_no_archive_no_version/service.py:handle

tree model_store
model_store
└── noop_no_archive_no_version
├── model.pt
└── service.py
torchserve --ncs --start --model-store model_store --models noop_no_archive_no_version --ts-config config.properties
server log:
Config file: config.properties
Inference address: http://0.0.0.0:8080
Management address: http://0.0.0.0:8081
Metrics address: http://127.0.0.1:8082
Model Store: /Volumes/workplace/python_env/serve/model_store
Initial Models: noop_no_archive_no_version
Log dir: /Volumes/workplace/python_env/serve/logs
Metrics dir: /Volumes/workplace/python_env/serve/logs
Netty threads: 32
Netty client threads: 0
Default workers per model: 12
Blacklist Regex: N/A
Maximum Response Size: 6553500
Maximum Request Size: 6553500
Limit Maximum Image Pixels: true
Prefer direct buffer: True
Allowed Urls: [file://.|http(s)?://.]
Custom python dependency for model allowed: true
Metrics report format: prometheus
Enable metrics API: true
Workflow Store: /Volumes/workplace/python_env/serve/model_store
Model config: N/A
2022-03-14T18:58:43,154 [INFO ] main org.pytorch.serve.servingsdk.impl.PluginsManager - Loading snapshot serializer plugin...
2022-03-14T18:58:43,186 [INFO ] main org.pytorch.serve.ModelServer - Loading initial models: noop_no_archive_no_version
2022-03-14T18:58:43,190 [WARN ] main org.pytorch.serve.archive.model.ModelArchive - Model archive version is not defined. Please upgrade to torch-model-archiver 0.2.0 or higher
2022-03-14T18:58:43,190 [WARN ] main org.pytorch.serve.archive.model.ModelArchive - Model archive createdOn is not defined. Please upgrade to torch-model-archiver 0.2.0 or higher
2022-03-14T18:58:43,193 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Adding new version 1.0 for model noop_no_archive_no_version
2022-03-14T18:58:43,193 [DEBUG] main org.pytorch.serve.wlm.ModelVersionedRefs - Setting default version to 1.0 for model noop_no_archive_no_version
2022-03-14T18:58:43,194 [INFO ] main org.pytorch.serve.wlm.ModelManager - Model noop_no_archive_no_version loaded.
2022-03-14T18:58:43,194 [DEBUG] main org.pytorch.serve.wlm.ModelManager - updateModel: noop_no_archive_no_version, count: 12
2022-03-14T18:58:43,216 [DEBUG] W-9002-noop_no_archive_no_version_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/Users/lninga/opt/anaconda3/envs/py38/bin/python, /Users/lninga/opt/anaconda3/envs/py38/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /var/folders/w6/s5gp9htn2pb9z87lwp6fzjg9hv4nys/T//.ts.sock.9002]
2022-03-14T18:58:43,216 [DEBUG] W-9005-noop_no_archive_no_version_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/Users/lninga/opt/anaconda3/envs/py38/bin/python, /Users/lninga/opt/anaconda3/envs/py38/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /var/folders/w6/s5gp9htn2pb9z87lwp6fzjg9hv4nys/T//.ts.sock.9005]
2022-03-14T18:58:43,216 [DEBUG] W-9006-noop_no_archive_no_version_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/Users/lninga/opt/anaconda3/envs/py38/bin/python, /Users/lninga/opt/anaconda3/envs/py38/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /var/folders/w6/s5gp9htn2pb9z87lwp6fzjg9hv4nys/T//.ts.sock.9006]
2022-03-14T18:58:43,216 [DEBUG] W-9008-noop_no_archive_no_version_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/Users/lninga/opt/anaconda3/envs/py38/bin/python, /Users/lninga/opt/anaconda3/envs/py38/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /var/folders/w6/s5gp9htn2pb9z87lwp6fzjg9hv4nys/T//.ts.sock.9008]
2022-03-14T18:58:43,216 [DEBUG] W-9001-noop_no_archive_no_version_1.0 org.pytorch.serve.wlm.WorkerLifeCycle - Worker cmdline: [/Users/lninga/opt/anaconda3/envs/py38/bin/python, /Users/lninga/opt/anaconda3/envs/py38/lib/python3.8/site-packages/ts/model_service_worker.py, --sock-type, unix, --sock-name, /var/folders/w6/s5gp9htn2pb9z87lwp6fzjg9hv4nys/T//.ts.sock.9001]

Checklist:

Have you added tests that prove your fix is effective or that this feature works?
Has code been commented, particularly in hard-to-understand areas?
Have you made corresponding changes to the documentation?

sagemaker-neo-ci-bot · 2022-03-15T02:08:12Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-win
Commit ID: 67e214b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-03-15T02:25:03Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-cpu
Commit ID: 67e214b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-03-15T02:30:13Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-gpu
Commit ID: 67e214b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

HamidShojanazeri · 2022-03-15T16:14:16Z

frontend/archive/src/main/java/org/pytorch/serve/archive/model/ModelArchive.java

@@ -70,6 +70,11 @@ public static ModelArchive downloadModel(

        if (new File(url).isDirectory()) {
            return load(url, new File(url), false);
+        } else if (modelLocation.exists()) {


@lxning how would we handle the case if there are multiple mar files in a directory? are we relying on user to have only one mar file in the /xxx/model_store/modelXXX directory? In this case we need to make sure we have documented it clearly.

Do you we have any documentation on it?

I understood this PR as no longer needing a model_store and just letting users link to various model files directy

In model_store dir, existing code ignores the mar files in the subdir (ie. /xxx/model_store/modelXXX).

maaquib · 2022-03-15T17:51:14Z

@lxning Can you add more details as to why this change is being made? What changed that this stopped working?

msaroufim · 2022-03-15T17:52:32Z

...tend/archive/src/test/resources/models/noop_no_archive_no_version/noop/MAR-INF/MANIFEST.json

+  "modelServerVersion": "1.0",
+  "implementationVersion": "1.0",
+  "specificationVersion": "1.0"
+}


Do we need to store a model.pt in version control? Is it just an empty file?

Here, noop is an example. model.pt is an empty file. Manifest requires it to pass validation.

msaroufim · 2022-03-15T17:56:15Z

frontend/archive/src/test/resources/models/noop_no_archive_no_version/noop/service.py

+import time
+
+
+class NoopService(object):


I'm not sure I follow why a NoopService is needed, it seems very similar to the base handler? Could you please explain at a high level in the github issue the design of this PR

The example noop_no_archive_no_version is copied from original noop_no_archive. I did slightly change (ie remove modelversion in manifest.json) for unit test the default manifest bug fixing.

msaroufim · 2022-03-15T17:58:17Z

frontend/archive/src/main/java/org/pytorch/serve/archive/model/ModelArchive.java

@@ -70,6 +70,11 @@ public static ModelArchive downloadModel(

        if (new File(url).isDirectory()) {
            return load(url, new File(url), false);
+        } else if (modelLocation.exists()) {


I understood this PR as no longer needing a model_store and just letting users link to various model files directy

sagemaker-neo-ci-bot · 2022-03-16T22:21:54Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-win
Commit ID: a513eaf
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-03-16T22:41:08Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-cpu
Commit ID: a513eaf
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-03-16T22:48:52Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-gpu
Commit ID: a513eaf
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-03-17T21:59:14Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-win
Commit ID: 338913e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-03-17T22:10:58Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-cpu
Commit ID: 338913e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-03-17T22:19:48Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-gpu
Commit ID: 338913e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-03-19T18:47:03Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-win
Commit ID: 338913e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

sagemaker-neo-ci-bot · 2022-03-19T19:10:42Z

AWS CodeBuild CI Report

CodeBuild project: torch-serve-build-gpu
Commit ID: 338913e
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

lxning added 3 commits March 14, 2022 17:45

fix no verion and no full path

ad961e2

unregister noop verison 1.0

85bd2a4

fmt

67e214b

lxning added the bug Something isn't working label Mar 15, 2022

lxning added this to the v0.6.0 milestone Mar 15, 2022

lxning requested review from maaquib and HamidShojanazeri March 15, 2022 01:41

lxning self-assigned this Mar 15, 2022

HamidShojanazeri reviewed Mar 15, 2022

View reviewed changes

msaroufim requested changes Mar 15, 2022

View reviewed changes

msaroufim self-requested a review March 16, 2022 18:41

msaroufim approved these changes Mar 16, 2022

View reviewed changes

add comments

a513eaf

HamidShojanazeri approved these changes Mar 17, 2022

View reviewed changes

Merge branch 'master' into issue_1498

338913e

lxning merged commit b8f19c4 into master Mar 19, 2022

msaroufim deleted the issue_1498 branch June 16, 2022 01:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix failed to create model archive #1508

fix failed to create model archive #1508

lxning commented Mar 15, 2022 •

edited

sagemaker-neo-ci-bot commented Mar 15, 2022

sagemaker-neo-ci-bot commented Mar 15, 2022

sagemaker-neo-ci-bot commented Mar 15, 2022

HamidShojanazeri Mar 15, 2022

msaroufim Mar 15, 2022

lxning Mar 16, 2022

maaquib commented Mar 15, 2022

msaroufim Mar 15, 2022

lxning Mar 16, 2022

msaroufim Mar 15, 2022

lxning Mar 16, 2022 •

edited

msaroufim Mar 15, 2022

sagemaker-neo-ci-bot commented Mar 16, 2022

sagemaker-neo-ci-bot commented Mar 16, 2022

sagemaker-neo-ci-bot commented Mar 16, 2022

sagemaker-neo-ci-bot commented Mar 17, 2022

sagemaker-neo-ci-bot commented Mar 17, 2022

sagemaker-neo-ci-bot commented Mar 17, 2022

sagemaker-neo-ci-bot commented Mar 19, 2022

sagemaker-neo-ci-bot commented Mar 19, 2022

		import time


		class NoopService(object):

fix failed to create model archive #1508

fix failed to create model archive #1508

Conversation

lxning commented Mar 15, 2022 • edited

Description

Type of change

Feature/Issue validation/testing

Checklist:

sagemaker-neo-ci-bot commented Mar 15, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Mar 15, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Mar 15, 2022

AWS CodeBuild CI Report

HamidShojanazeri Mar 15, 2022

Choose a reason for hiding this comment

msaroufim Mar 15, 2022

Choose a reason for hiding this comment

lxning Mar 16, 2022

Choose a reason for hiding this comment

maaquib commented Mar 15, 2022

msaroufim Mar 15, 2022

Choose a reason for hiding this comment

lxning Mar 16, 2022

Choose a reason for hiding this comment

msaroufim Mar 15, 2022

Choose a reason for hiding this comment

lxning Mar 16, 2022 • edited

Choose a reason for hiding this comment

msaroufim Mar 15, 2022

Choose a reason for hiding this comment

sagemaker-neo-ci-bot commented Mar 16, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Mar 16, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Mar 16, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Mar 17, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Mar 17, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Mar 17, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Mar 19, 2022

AWS CodeBuild CI Report

sagemaker-neo-ci-bot commented Mar 19, 2022

AWS CodeBuild CI Report

lxning commented Mar 15, 2022 •

edited

lxning Mar 16, 2022 •

edited