Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Package model hosted in curated registries failing #2757

Open
sqlshep opened this issue Oct 24, 2023 · 0 comments
Open

Package model hosted in curated registries failing #2757

sqlshep opened this issue Oct 24, 2023 · 0 comments
Labels

Comments

@sqlshep
Copy link

sqlshep commented Oct 24, 2023

Operating System

Linux

Version Information

I cloned the following repo 10/24/2023
The following notebook is failing i made no changes to it

sdk/python/endpoints/online/deploy-with-packages/mlflow-model/sdk-deploy-and-test.ipynb

At the following command
model_package = ml_client.models.package(model_name, model.version, pakage_config)

with the following error

Method package: This is an experimental method, and may change at any time. Please see https://aka.ms/azuremlexperimental for more information.
Creating package with name: heart-classifier-mlflow-pkg

OperationFailed Traceback (most recent call last)
File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/azure/core/polling/base_polling.py:466, in LROBasePolling.run(self)
465 try:
--> 466 self._poll()
468 except BadStatus as err:

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/azure/core/polling/base_polling.py:500, in LROBasePolling._poll(self)
499 if _failed(self.status()):
--> 500 raise OperationFailed("Operation failed or canceled")
502 final_get_url = self._operation.get_final_get_url(self._pipeline_response)

OperationFailed: Operation failed or canceled

During handling of the above exception, another exception occurred:

HttpResponseError Traceback (most recent call last)
Cell In[12], line 1
----> 1 model_package = ml_client.models.package(model_name, model.version, pakage_config)

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/azure/ai/ml/_utils/_experimental.py:69, in _add_method_docstring..wrapped(*args, **kwargs)
67 if not _should_skip_warning() and not _is_warning_cached(message):
68 module_logger.warning(message)
---> 69 return func(*args, **kwargs)

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/azure/ai/ml/_telemetry/activity.py:263, in monitor_with_activity..monitor..wrapper(*args, **kwargs)
260 @functools.wraps(f)
261 def wrapper(*args, **kwargs):
262 with log_activity(logger, activity_name or f.name, activity_type, custom_dimensions):
--> 263 return f(*args, **kwargs)

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/azure/ai/ml/operations/_model_operations.py:625, in ModelOperations.package(self, name, version, package_request, **kwargs)
615 package_request = package_request._to_rest_object()
617 module_logger.info("Creating package with name: %s", package_request.target_environment_name)
619 package_out = self._model_versions_operation.begin_package(
620 name=name,
621 version=version,
622 workspace_name=self._workspace_name,
623 body=package_request,
624 **self._scope_kwargs,
--> 625 ).result()
627 if package_out is not None and package_out.class.name == "PackageResponse":
628 environment_operation = self._all_operations.all_operations[AzureMLResourceType.ENVIRONMENT]

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/azure/core/polling/_poller.py:230, in LROPoller.result(self, timeout)
222 def result(self, timeout: Optional[float] = None) -> PollingReturnType:
223 """Return the result of the long running operation, or
224 the result available after the specified timeout.
225
(...)
228 :raises ~azure.core.exceptions.HttpResponseError: Server problem with the query.
229 """
--> 230 self.wait(timeout)
231 return self._polling_method.resource()

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/azure/core/tracing/decorator.py:76, in distributed_trace..decorator..wrapper_use_tracer(*args, **kwargs)
74 span_impl_type = settings.tracing_implementation()
75 if span_impl_type is None:
---> 76 return func(*args, **kwargs)
78 # Merge span is parameter is set, but only if no explicit parent are passed
79 if merge_span and not passed_in_parent:

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/azure/core/polling/_poller.py:249, in LROPoller.wait(self, timeout)
245 self._thread.join(timeout=timeout)
246 try:
247 # Let's handle possible None in forgiveness here
248 # python/mypy#8165
--> 249 raise self._exception # type: ignore
250 except TypeError: # Was None
251 pass

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/azure/core/polling/_poller.py:169, in LROPoller._start(self)
162 """Start the long running operation.
163 On completion, runs any callbacks.
164
165 :param callable update_cmd: The API request to check the status of
166 the operation.
167 """
168 try:
--> 169 self._polling_method.run()
170 except AzureError as error:
171 if not error.continuation_token:

File /anaconda/envs/azureml_py310_sdkv2/lib/python3.10/site-packages/azure/core/polling/base_polling.py:481, in LROBasePolling.run(self)
474 raise HttpResponseError(
475 response=self._pipeline_response.http_response,
476 message=str(err),
477 error=err,
478 )
480 except OperationFailed as err:
--> 481 raise HttpResponseError(response=self._pipeline_response.http_response, error=err)

HttpResponseError: Operation returned an invalid status 'OK'
Content: {
"status": "Failed"
}

Steps to reproduce

  1. clone the repo
  2. Run the notebook

Expected behavior

Not fail

Actual behavior

Fail

Addition information

None

@sqlshep sqlshep added the bug label Oct 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant