[FEATURE] Check that all AzureML notebooks are tested #207

catherine667 · 2019-07-29T18:33:58Z

Description

Need to add .azureml folder to provide the common AzureML subscription for AzureML notebooks.

How do we replicate the bug?

Under .azureml folder, it should contain:
config.json file should look like this: "{"Id": null, "Scope": "/subscriptions/[ID]/resourceGroups/nlprg/providers/Microsoft.MachineLearningServices/workspaces/[Workspace Name]"}", which can be downloaded from workspace.

Related notebooks:
GenSen #199 and BERT #191 notebook testing

related to #143

Expected behavior (i.e. solution)

Other Comments

#262

miguelgfierro · 2019-08-02T11:56:28Z

I'll take this one as a continuation of #25

miguelgfierro · 2019-08-07T13:12:24Z

miguelgfierro · 2019-09-25T11:42:07Z

error in bidaf quickstart:

export NLP_SUBSCRIPTION_ID=XXXX
pytest tests/integration/test_notebooks_question_answering.py::test_bidaf_quickstart -q --subscription_id=$NLP_SUBSCRIPTION_ID --resource_group="nlpbp_project_resources" --workspace_name="nlpazuremltestws" --workspace_region="eastus2"

when trying to run the notebook, I get:

2019/09/25 11:29:32 Downloading source code...
2019/09/25 11:29:33 Finished downloading source code
2019/09/25 11:29:33 Using acb_vol_dfb4e0a6-16c9-4ad2-b8e6-1649950c3ae7 as the home volume
2019/09/25 11:29:33 Creating Docker network: acb_default_network, driver: 'bridge'
2019/09/25 11:29:33 Successfully set up Docker network: acb_default_network
2019/09/25 11:29:33 Setting up Docker configuration...
2019/09/25 11:29:36 Successfully set up Docker configuration
2019/09/25 11:29:36 Logging in to registry: nlpazuremltece0e443a.azurecr.io
2019/09/25 11:29:37 Successfully logged into nlpazuremltece0e443a.azurecr.io
2019/09/25 11:29:37 Executing step ID: acb_step_0. Timeout(sec): 2400, Working directory: '', Network: 'acb_default_network'
2019/09/25 11:29:37 Scanning for dependencies...
2019/09/25 11:29:38 Successfully scanned dependencies
2019/09/25 11:29:38 Launching container with name: acb_step_0
Sending build context to Docker daemon  6.144kB


Step 1/13 : FROM mcr.microsoft.com/azureml/o16n-base/python-slim:latest.eastus2
latest.eastus2: Pulling from azureml/o16n-base/python-slim
Digest: sha256:9835e3046eb51f2156c47a3da17375a7dcfee0f20e01fd50820bc80352dd8b1c
Status: Downloaded newer image for mcr.microsoft.com/azureml/o16n-base/python-slim:latest.eastus2
 ---> 7ba54ac048ac
Step 2/13 : ENV AZUREML_MODEL_DIR=azureml-models/bidaf/7
 ---> Running in 149431ddac7e
Removing intermediate container 149431ddac7e
 ---> 2f91e6d13c99
Step 3/13 : RUN mkdir -p '/var/azureml-app' && /var/azureml-util/download_asset.sh 'https://nlpazuremltest6466659734.blob.core.windows.net/azureml/LocalUpload/3988aa0b/tmpypdexfxu.py?sv=2018-11-09&sr=b&sig=8nvT28VjHCUv7ppGT4eVw105XVENUMqnieBN%2BWTlpDI%3D&st=2019-09-25T11%3A19%3A29Z&se=2019-09-25T19%3A29%3A29Z&sp=r' '/var/azureml-app/tmpypdexfxu.py'
 ---> Running in ed09ebe5215d
Downloading https://nlpazuremltest6466659734.blob.core.windows.net/azureml/LocalUpload/3988aa0b/tmpypdexfxu.py?sv=2018-11-09&sr=b&sig=8nvT28VjHCUv7ppGT4eVw105XVENUMqnieBN%2BWTlpDI%3D&st=2019-09-25T11%3A19%3A29Z&se=2019-09-25T19%3A29%3A29Z&sp=r as /var/azureml-app/tmpypdexfxu.py
�[91m--2019-09-25 11:29:41--  https://nlpazuremltest6466659734.blob.core.windows.net/azureml/LocalUpload/3988aa0b/tmpypdexfxu.py?sv=2018-11-09&sr=b&sig=8nvT28VjHCUv7ppGT4eVw105XVENUMqnieBN%2BWTlpDI%3D&st=2019-09-25T11%3A19%3A29Z&se=2019-09-25T19%3A29%3A29Z&sp=r
�[0m�[91mResolving nlpazuremltest6466659734.blob.core.windows.net (nlpazuremltest6466659734.blob.core.windows.net)... �[0m�[91m52.239.223.19
Connecting to nlpazuremltest6466659734.blob.core.windows.net (nlpazuremltest6466659734.blob.core.windows.net)|52.239.223.19|:443... �[0m�[91mconnected.
�[0m�[91mHTTP request sent, awaiting response... �[0m�[91m200 OK
Length: 3210 (3.1K) [application/octet-stream]
Saving to: ‘/var/azureml-app/tmpypdexfxu.py’
�[0m�[91m
     0K                                    100%  352M=0s

2019-09-25 11:29:41 (352 MB/s) - ‘/var/azureml-app/tmpypdexfxu.py’ saved [3210/3210]

�[0mRemoving intermediate container ed09ebe5215d
 ---> e8bbc5685c1a
Step 4/13 : RUN mkdir -p '/var/azureml-app' && /var/azureml-util/download_asset.sh 'https://nlpazuremltest6466659734.blob.core.windows.net/azureml/LocalUpload/520b1998/2f053bd7.tar.gz?sv=2018-11-09&sr=b&sig=FHNcCepjdUcRSFCDhUknsNCG1oZWJutjmIYZSPMZfAI%3D&st=2019-09-25T11%3A19%3A29Z&se=2019-09-25T19%3A29%3A29Z&sp=r' /tmp/temp.tar.gz && tar xzf /tmp/temp.tar.gz -C '/var/azureml-app' && rm -f /tmp/temp.tar.gz
 ---> Running in 4a294179b36e
Downloading https://nlpazuremltest6466659734.blob.core.windows.net/azureml/LocalUpload/520b1998/2f053bd7.tar.gz?sv=2018-11-09&sr=b&sig=FHNcCepjdUcRSFCDhUknsNCG1oZWJutjmIYZSPMZfAI%3D&st=2019-09-25T11%3A19%3A29Z&se=2019-09-25T19%3A29%3A29Z&sp=r as /tmp/temp.tar.gz
�[91m--2019-09-25 11:29:43--  https://nlpazuremltest6466659734.blob.core.windows.net/azureml/LocalUpload/520b1998/2f053bd7.tar.gz?sv=2018-11-09&sr=b&sig=FHNcCepjdUcRSFCDhUknsNCG1oZWJutjmIYZSPMZfAI%3D&st=2019-09-25T11%3A19%3A29Z&se=2019-09-25T19%3A29%3A29Z&sp=r
Resolving nlpazuremltest6466659734.blob.core.windows.net (nlpazuremltest6466659734.blob.core.windows.net)... �[0m�[91m52.239.223.19
Connecting to nlpazuremltest6466659734.blob.core.windows.net (nlpazuremltest6466659734.blob.core.windows.net)|52.239.223.19|:443... �[0m�[91mconnected.
�[0m�[91mHTTP request sent, awaiting response... �[0m�[91m200 OK
Length: 503 [application/octet-stream]
Saving to: ‘/tmp/temp.tar.gz’

     0K                                    100% 70.5M=0s

�[0m�[91m2019-09-25 11:29:43 (70.5 MB/s) - ‘/tmp/temp.tar.gz’ saved [503/503]

�[0mRemoving intermediate container 4a294179b36e
 ---> d181ce793700
Step 5/13 : RUN mkdir -p '/var/azureml-app/azureml-models/bidaf/7' && /var/azureml-util/download_asset.sh 'https://nlpazuremltest6466659734.blob.core.windows.net/azureml/LocalUpload/190925T122716-601d3f04/bidaf.tar.gz?sv=2018-11-09&sr=b&sig=iRFpgYKv32EVynKNroI%2FO4FLWFEJCo6oAjrEe3K64L8%3D&st=2019-09-25T11%3A19%3A29Z&se=2019-09-25T19%3A29%3A29Z&sp=r' /tmp/temp.tar.gz && tar xzf /tmp/temp.tar.gz -C '/var/azureml-app/azureml-models/bidaf/7' && rm -f /tmp/temp.tar.gz
 ---> Running in b14bd5925626
Downloading https://nlpazuremltest6466659734.blob.core.windows.net/azureml/LocalUpload/190925T122716-601d3f04/bidaf.tar.gz?sv=2018-11-09&sr=b&sig=iRFpgYKv32EVynKNroI%2FO4FLWFEJCo6oAjrEe3K64L8%3D&st=2019-09-25T11%3A19%3A29Z&se=2019-09-25T19%3A29%3A29Z&sp=r as /tmp/temp.tar.gz
�[91m--2019-09-25 11:29:44--  https://nlpazuremltest6466659734.blob.core.windows.net/azureml/LocalUpload/190925T122716-601d3f04/bidaf.tar.gz?sv=2018-11-09&sr=b&sig=iRFpgYKv32EVynKNroI%2FO4FLWFEJCo6oAjrEe3K64L8%3D&st=2019-09-25T11%3A19%3A29Z&se=2019-09-25T19%3A29%3A29Z&sp=r
�[0m�[91mResolving nlpazuremltest6466659734.blob.core.windows.net (nlpazuremltest6466659734.blob.core.windows.net)... �[0m�[91m52.239.223.19
Connecting to nlpazuremltest6466659734.blob.core.windows.net (nlpazuremltest6466659734.blob.core.windows.net)|52.239.223.19|:443... �[0m�[91mconnected.
�[0m�[91mHTTP request sent, awaiting response... �[0m�[91m200 OK
Length: 25218039 (24M) [application/octet-stream]
Saving to: ‘/tmp/temp.tar.gz’
�[0m�[91m
     0K .�[0m�[91m.�[0m�[91m.�[0m�[91m.�[0m�[91m.�[0m�[91m.�[0m�[91m.�[0m�[91m.�[0m�[91m .�[0m�[91m.�[0m�[91m.�[0m�[91m.�[0m�[91m.�[0m�[91m.�[0m�[91m.�[0m�[91m.�[0m�[91m .�[0m�[91m.�[0m�[91m.�[0m�[91m.�[0m�[91m.�[0m�[91m.�[0m�[91m.�[0m�[91m.�[0m�[91m         100% 60.0M=0.4s

�[0m�[91m2019-09-25 11:29:45 (60.0 MB/s) - ‘/tmp/temp.tar.gz’ saved [25218039/25218039]

�[0m�[91m
gzip: stdin: not in gzip format
�[0m�[91mtar: Child returned status 1
tar: Error is not recoverable: exiting now
�[0mThe command '/bin/sh -c mkdir -p '/var/azureml-app/azureml-models/bidaf/7' && /var/azureml-util/download_asset.sh 'https://nlpazuremltest6466659734.blob.core.windows.net/azureml/LocalUpload/190925T122716-601d3f04/bidaf.tar.gz?sv=2018-11-09&sr=b&sig=iRFpgYKv32EVynKNroI%2FO4FLWFEJCo6oAjrEe3K64L8%3D&st=2019-09-25T11%3A19%3A29Z&se=2019-09-25T19%3A29%3A29Z&sp=r' /tmp/temp.tar.gz && tar xzf /tmp/temp.tar.gz -C '/var/azureml-app/azureml-models/bidaf/7' && rm -f /tmp/temp.tar.gz' returned a non-zero code: 2
2019/09/25 11:29:45 Container failed during run: acb_step_0. No retries remaining.
failed to run step ID: acb_step_0: exit status 2

Run ID: ch8 failed after 15s. Error: failed during run, err: exit status 1

miguelgfierro · 2019-09-25T15:20:37Z

error in deepdive:

pytest tests/integration/test_notebooks_question_answering.py::test_bidaf_deep_dive -q --subscription_id=$NLP_SUBSCRIPTION_ID --resource_group="nlpbp_project_resources" --workspace_name="nlpazuremltestws" --workspace_region="eastus2"



tests/integration/test_notebooks_question_answering.py:31:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../anaconda/envs/nlp_gpu/lib/python3.6/site-packages/papermill/execute.py:104: in execute_notebook
    raise_for_execution_errors(nb, output_path)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

nb = {'cells': [{'cell_type': 'code', 'metadata': {'inputHidden': True, 'hide_input': True}, 'execution_count': None, 'sour...-BiDAF-deepdive_1569422780_7e733af1'}}}, 'version_major': 2, 'version_minor': 0}}}, 'nbformat': 4, 'nbformat_minor': 2}
output_path = 'output.ipynb'

    def raise_for_execution_errors(nb, output_path):
        """Assigned parameters into the appropriate place in the input notebook

        Parameters
        ----------
        nb : NotebookNode
           Executable notebook object
        output_path : str
           Path to write executed notebook
        """
        error = None
        for cell in nb.cells:
            if cell.get("outputs") is None:
                continue

            for output in cell.outputs:
                if output.output_type == "error":
                    error = PapermillExecutionError(
                        exec_count=cell.execution_count,
                        source=cell.source,
                        ename=output.ename,
                        evalue=output.evalue,
                        traceback=output.traceback,
                    )
                    break

        if error:
            # Write notebook back out with the Error Message at the top of the Notebook.
            error_msg = ERROR_MESSAGE_TEMPLATE % str(error.exec_count)
            error_msg_cell = nbformat.v4.new_code_cell(
                source="%%html\n" + error_msg,
                outputs=[
                    nbformat.v4.new_output(output_type="display_data", data={"text/html": error_msg})
                ],
                metadata={"inputHidden": True, "hide_input": True},
            )
            nb.cells = [error_msg_cell] + nb.cells
            write_ipynb(nb, output_path)
>           raise error
E           papermill.exceptions.PapermillExecutionError:
E           ---------------------------------------------------------------------------
E           Exception encountered at "In [21]":
E           ---------------------------------------------------------------------------
E           RuntimeError                              Traceback (most recent call last)
E           <ipython-input-21-f75fe2c7a501> in <module>
E           ----> 1 model = Predictor.from_path(LOGS_FOLDER+"/logs")
E
E           ~/anaconda/envs/nlp_gpu/lib/python3.6/site-packages/allennlp/predictors/predictor.py in from_path(cls, archive_path, predictor_name)
E               142         A Predictor instance.
E               143         """
E           --> 144         return Predictor.from_archive(load_archive(archive_path), predictor_name)
E               145
E               146     @classmethod
E
E           ~/anaconda/envs/nlp_gpu/lib/python3.6/site-packages/allennlp/models/archival.py in load_archive(archive_file, cuda_device, overrides, weights_file)
E               228                        weights_file=weights_path,
E               229                        serialization_dir=serialization_dir,
E           --> 230                        cuda_device=cuda_device)
E               231
E               232     return Archive(model=model, config=config)
E
E           ~/anaconda/envs/nlp_gpu/lib/python3.6/site-packages/allennlp/models/model.py in load(cls, config, serialization_dir, weights_file, cuda_device)
E               325         # This allows subclasses of Model to override _load.
E               326         # pylint: disable=protected-access
E           --> 327         return cls.by_name(model_type)._load(config, serialization_dir, weights_file, cuda_device)
E               328
E               329     def extend_embedder_vocab(self, embedding_sources_mapping: Dict[str, str] = None) -> None:
E
E           ~/anaconda/envs/nlp_gpu/lib/python3.6/site-packages/allennlp/models/model.py in _load(cls, config, serialization_dir, weights_file, cuda_device)
E               273         model.extend_embedder_vocab()
E               274
E           --> 275         model_state = torch.load(weights_file, map_location=util.device_mapping(cuda_device))
E               276         model.load_state_dict(model_state)
E               277
E
E           ~/anaconda/envs/nlp_gpu/lib/python3.6/site-packages/torch/serialization.py in load(f, map_location, pickle_module, **pickle_load_args)
E               384         f = f.open('rb')
E               385     try:
E           --> 386         return _load(f, map_location, pickle_module, **pickle_load_args)
E               387     finally:
E               388         if new_fd:
E
E           ~/anaconda/envs/nlp_gpu/lib/python3.6/site-packages/torch/serialization.py in _load(f, map_location, pickle_module, **pickle_load_args)
E               578     for key in deserialized_storage_keys:
E               579         assert key in deserialized_objects
E           --> 580         deserialized_objects[key]._set_from_file(f, offset, f_should_read_directly)
E               581         if offset is not None:
E               582             offset = f.tell()
E
E           RuntimeError: unexpected EOF, expected 7358545 more bytes. The file might be corrupted.

../../anaconda/envs/nlp_gpu/lib/python3.6/site-packages/papermill/execute.py:188: PapermillExecutionError

1 failed, 6 warnings in 1880.95 seconds

miguelgfierro · 2019-09-25T16:54:11Z

26/9/2019:

test passing: 1 passed, 6 warnings in 2836.78 seconds
after fixing a couple of bugs

nohup pytest tests/integration/test_notebooks_text_classification.py::test_tc_bert_azureml -q --subscription_id=$NLP_SUBSCRIPTION_ID --resource_group="nlpbp_project_resources" --workspace_name="nlpazuremltestws" --workspace_region="eastus2" &

miguelgfierro · 2019-09-26T15:45:48Z

error in:

nohup pytest tests/integration/test_notebooks_question_answering.py::test_bert_qa_runs -q --subscription_id=$NLP_SUBSCRIPTION_ID --resource_group="nlpbp_project_resources" --workspace_name="nlpazuremltestws" --workspace_region="eastus2" &

E           ---------------------------------------------------------------------------
E           Exception encountered at "In [4]":
E           ---------------------------------------------------------------------------
E           WorkspaceException                        Traceback (most recent call last)
E           <ipython-input-4-f5cb6f8cb77e> in <module>
E                 1 if os.path.exists(AZUREML_CONFIG_PATH):
E           ----> 2     ws = azureml_utils.get_or_create_workspace(config_path=AZUREML_CONFIG_PATH)
E                 3 else:
E                 4     ws = azureml_utils.get_or_create_workspace(
E                 5         subscription_id=subscription_id,
E
E           ~/repos/nlp/utils_nlp/azureml/azureml_utils.py in get_or_create_workspace(config_path, subscription_id, resource_group, workspace_name, workspace_region)
E                66                 subscription_id=subscription_id,
E                67                 resource_group=resource_group,
E           ---> 68                 auth=get_auth(),
E                69             )
E                70
E
E           ~/anaconda/envs/nlp_gpu/lib/python3.6/site-packages/azureml/core/workspace.py in get(name, auth, subscription_id, resource_group)
E               349
E               350         if not subscription_id:
E           --> 351             subscription_id = Workspace._fetch_subscription(auth)
E               352
E               353         result_dict = Workspace.list(subscription_id, auth=auth, resource_group=resource_group)
E
E           ~/anaconda/envs/nlp_gpu/lib/python3.6/site-packages/azureml/core/workspace.py in _fetch_subscription(auth)
E               743         if len(all_subscriptions) > 1:
E               744             raise WorkspaceException("You have access to more than one subscriptions. "
E           --> 745                                      "Please specify one from this list = {}".format(all_subscriptions))
E               746
E               747         if len(all_subscriptions) == 1:
E
E           WorkspaceException: WorkspaceException:
E               Message: You have access to more than one subscriptions. Please specify one from this list = ([

I tried to set the subscription with az account set -s <sub id> but got the same error. Interestingly, the first time I tried this, it worked

miguelgfierro · 2019-09-26T16:01:50Z

error in:

nohup pytest tests/integration/test_notebooks_sentence_similarity.py::test_bert_senteval -q --subscription_id=$NLP_SUBSCRIPTION_ID --resource_group="nlpbp_project_resources" --workspace_name="nlpazuremltestws" --workspace_region="eastus2" &


E           ---------------------------------------------------------------------------
E           Exception encountered at "In [19]":
E           ---------------------------------------------------------------------------
E           AttributeError                            Traceback (most recent call last)
E           ~/anaconda/envs/nlp_gpu/lib/python3.6/site-packages/numpy/core/fromnumeric.py in _wrapfunc(obj, method, *args, **kwds)
E                55     try:
E           ---> 56         return getattr(obj, method)(*args, **kwds)
E                57
E
E           AttributeError: 'list' object has no attribute 'reshape'
E
E           During handling of the above exception, another exception occurred:
E
E           ValueError                                Traceback (most recent call last)
E           <ipython-input-19-14f9ea6ddd1c> in <module>
E                 9     np.reshape(
E                10         [r["STSBenchmark"]["pearson"] for r in results],
E           ---> 11         (len(EXP_PARAMS["layer_index"]), len(EXP_PARAMS["pooling_strategy"])),
E                12     ).T,
E                13     index=[s.value for s in EXP_PARAMS["pooling_strategy"]],
E
E           ~/anaconda/envs/nlp_gpu/lib/python3.6/site-packages/numpy/core/fromnumeric.py in reshape(a, newshape, order)
E               290            [5, 6]])
E               291     """
E           --> 292     return _wrapfunc(a, 'reshape', newshape, order=order)
E               293
E               294
E
E           ~/anaconda/envs/nlp_gpu/lib/python3.6/site-packages/numpy/core/fromnumeric.py in _wrapfunc(obj, method, *args, **kwds)
E                64     # a downstream library like 'pandas'.
E                65     except (AttributeError, TypeError):
E           ---> 66         return _wrapit(obj, method, *args, **kwds)
E                67
E                68
E
E           ~/anaconda/envs/nlp_gpu/lib/python3.6/site-packages/numpy/core/fromnumeric.py in _wrapit(obj, method, *args, **kwds)
E                44     except AttributeError:
E                45         wrap = None
E           ---> 46     result = getattr(asarray(obj), method)(*args, **kwds)
E                47     if wrap:
E                48         if not isinstance(result, mu.ndarray):
E
E           ValueError: cannot reshape array of size 0 into shape (12,2)

../../anaconda/envs/nlp_gpu/lib/python3.6/site-packages/papermill/execute.py:188: PapermillExecutionError

This error comes because the size of results should be 24, but it is in fact 13.

miguelgfierro · 2019-09-26T16:51:11Z

error in

nohup pytest tests/integration/test_notebooks_entailment.py::test_entailment_xnli_bert_azureml -q --subscription_id=$NLP_SUBSCRIPTION_ID --resource_group="nlpbp_project_resources" --workspace_name="nlpazuremltestws" --workspace_region="eastus2" &


E           ---------------------------------------------------------------------------
E           Exception encountered at "In [6]":
E           ---------------------------------------------------------------------------
E           TypeError                                 Traceback (most recent call last)
E           <ipython-input-6-20e11f16a9dd> in <module>
E                 1 try:
E           ----> 2     compute_target = ComputeTarget(workspace=ws, name=cluster_name)
E                 3     print("Found compute target: {}".format(cluster_name))
E                 4 except ComputeTargetException:
E                 5     print("Creating new compute target: {}".format(cluster_name))
E
E           ~/anaconda/envs/nlp_gpu/lib/python3.6/site-packages/azureml/core/compute/compute.py in __new__(cls, workspace, name)
E                76                                              'provided workspace'.format(name))
E                77         else:
E           ---> 78             return super(ComputeTarget, cls).__new__(cls)
E                79
E                80     def __init__(self, workspace, name):
E
E           TypeError: Can't instantiate abstract class ComputeTarget with abstract methods _initialize, _validate_get_payload, delete, deserialize, detach, refresh_state, serialize

../../anaconda/envs/nlp_gpu/lib/python3.6/site-packages/papermill/execute.py:188: PapermillExecutionError

miguelgfierro · 2019-09-27T12:37:01Z

error in:

nohup pytest tests/integration/test_notebooks_sentence_similarity.py::test_automl_local_deployment_aci -q --subscription_id=$NLP_SUBSCRIPTION_ID --resource_group="nlpbp_project_resources" --workspace_name="nlpazuremltestws" --workspace_region="eastus2" &



E           ---------------------------------------------------------------------------
E           Exception encountered at "In [24]":
E           ---------------------------------------------------------------------------
E           WebserviceException                       Traceback (most recent call last)
E           <ipython-input-24-e204d6e8e957> in <module>
E                15 )
E                16
E           ---> 17 image.wait_for_creation(show_output=True)
E
E           ~/anaconda/envs/nlp_gpu/lib/python3.6/site-packages/azureml/core/image/image.py in wait_for_creation(self, show_output)
E               426                                       'current state: {}\n'
E               427                                       'Error response from server:\n'
E           --> 428                                       '{}'.format(self.creation_state, error_response), logger=module_logger)
E               429
E               430         print('Image creation operation finished for image {}, operation "{}"'.format(self.id, operation_state))
E
E           WebserviceException: WebserviceException:
E               Message: Image creation polling reached non-successful terminal state, current state: Failed
E           Error response from server:
E           StatusCode: 400
E           Message: Docker image build failed.
E               InnerException None
E               ErrorResponse
E           {
E               "error": {
E                   "message": "Image creation polling reached non-successful terminal state, current state: Failed\nError response from server:\nStatusCode: 400\nMessage: Docker image build failed."
E               }
E           }

../../anaconda/envs/nlp_gpu/lib/python3.6/site-packages/papermill/execute.py:188: PapermillExecutionError

devjav786 · 2019-12-11T16:40:34Z

@miguelgfierro how You fixed this issue?

catherine667 added the bug Something isn't working label Jul 29, 2019

catherine667 mentioned this issue Jul 29, 2019

GenSen AzureML notebook Test #199

Merged

3 tasks

miguelgfierro self-assigned this Aug 2, 2019

miguelgfierro changed the title ~~[FEATURE] Add the AzureML configuration to integration tests~~ [FEATURE] Check that all AzureML notebooks are tested Aug 9, 2019

miguelgfierro mentioned this issue Aug 9, 2019

Make sure that AzureML test can run programmatically (WORK IN PROGRESS) #262

Merged

3 tasks

sharatsc added the release-blocker label Nov 25, 2019

miguelgfierro closed this as completed Nov 25, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FEATURE] Check that all AzureML notebooks are tested #207

[FEATURE] Check that all AzureML notebooks are tested #207

catherine667 commented Jul 29, 2019 •

edited by miguelgfierro

miguelgfierro commented Aug 2, 2019

miguelgfierro commented Aug 7, 2019 •

edited

miguelgfierro commented Sep 25, 2019 •

edited

miguelgfierro commented Sep 25, 2019 •

edited

miguelgfierro commented Sep 25, 2019 •

edited

miguelgfierro commented Sep 26, 2019 •

edited

miguelgfierro commented Sep 26, 2019 •

edited

miguelgfierro commented Sep 26, 2019

miguelgfierro commented Sep 27, 2019

devjav786 commented Dec 11, 2019

[FEATURE] Check that all AzureML notebooks are tested #207

[FEATURE] Check that all AzureML notebooks are tested #207

Comments

catherine667 commented Jul 29, 2019 • edited by miguelgfierro

Description

How do we replicate the bug?

Expected behavior (i.e. solution)

Other Comments

miguelgfierro commented Aug 2, 2019

miguelgfierro commented Aug 7, 2019 • edited

Check all AzureML notebooks have a test:

Check all AzureML notebooks are passing the tests:

miguelgfierro commented Sep 25, 2019 • edited

miguelgfierro commented Sep 25, 2019 • edited

miguelgfierro commented Sep 25, 2019 • edited

miguelgfierro commented Sep 26, 2019 • edited

miguelgfierro commented Sep 26, 2019 • edited

miguelgfierro commented Sep 26, 2019

miguelgfierro commented Sep 27, 2019

devjav786 commented Dec 11, 2019

catherine667 commented Jul 29, 2019 •

edited by miguelgfierro

miguelgfierro commented Aug 7, 2019 •

edited

miguelgfierro commented Sep 25, 2019 •

edited

miguelgfierro commented Sep 25, 2019 •

edited

miguelgfierro commented Sep 25, 2019 •

edited

miguelgfierro commented Sep 26, 2019 •

edited

miguelgfierro commented Sep 26, 2019 •

edited