07_Train_With_AML_Pipeline.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Publish a Training Pipeline\n",
    "In this notebook, we will show how to automate the training/retraining of model using HyperDrive and registering best model. Once this training pipeline is published/created, it provides a REST endpoint which can be called to run this pipeline without using the Azure Machine Learning Service SDK.\n",
    "\n",
    "The steps in this notebook are\n",
    "- [import libraries](#import),\n",
    "- [read in the Azure ML workspace](#workspace),\n",
    "- [upload the data to the cloud](#upload),\n",
    "- [define a hyperparameter search configuration](#configuration),\n",
    "- [create an estimator](#estimator),\n",
    "- [Azure Machine Learning Pipelines overview](#aml_pipeline_overview)\n",
    "- [create the pipeline tuning step](#aml_pipeline_tune_step),\n",
    "- [create the pipeline best parameters step](#aml_pipeline_bh_step),\n",
    "- [create the pipeline best model step](#aml_pipeline_bm_step),\n",
    "- [create the pipeline register model step](#aml_pipeline_rm_step),\n",
    "- [create the pipeline itself](#create_aml_pipeline),\n",
    "- [publish the pipeline](#publish_aml_pipeline), and\n",
    "- [run the published pipeline using its REST endpoint](#run_publish_aml_pipeline).\n",
    "\n",
    "## Imports  <a id='import'></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import pandas as pd\n",
    "import requests\n",
    "\n",
    "from azureml.core import Workspace, Experiment\n",
    "from azureml.core.datastore import Datastore\n",
    "from azureml.core.compute import ComputeTarget, AmlCompute\n",
    "from azureml.exceptions import ComputeTargetException\n",
    "from azureml.data.data_reference import DataReference\n",
    "from azureml.pipeline.steps import HyperDriveStep, PythonScriptStep, EstimatorStep\n",
    "from azureml.pipeline.core import Pipeline, PipelineData, PipelineParameter\n",
    "from azureml.core.runconfig import RunConfiguration, CondaDependencies\n",
    "from azureml.train.estimator import Estimator\n",
    "from azureml.train.hyperdrive import (\n",
    "    RandomParameterSampling, choice, PrimaryMetricGoal,\n",
    "    HyperDriveConfig, MedianStoppingPolicy)\n",
    "\n",
    "import azureml.core\n",
    "from get_auth import get_auth\n",
    "print('azureml.core.VERSION={}'.format(azureml.core.VERSION))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Read in the Azure ML workspace  <a id='workspace'></a>\n",
    "Read in the the workspace created in a previous notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "auth = get_auth()\n",
    "ws = Workspace.from_config(auth=auth)\n",
    "ws_details = ws.get_details()\n",
    "print('Name:\\t\\t{}\\nLocation:\\t{}'\n",
    "      .format(ws_details['name'],\n",
    "              ws_details['location']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Upload the data to the cloud <a id='upload'></a>\n",
    "We put the data in a particular directory on the workspace's default data store. This will show up in the same location in the file system of every job running on the Batch AI cluster.\n",
    "\n",
    "Get a handle to the workspace's default data store."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds = ws.get_default_datastore()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Upload the data. We use `overwrite=False` to avoid taking the time to re-upload the data should files with the same names be already present. If you change the data and want to refresh what's uploaded, use `overwrite=True`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ds.upload(src_dir=os.path.join('.', 'data'), target_path='data', overwrite=False, show_progress=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define a hyperparameter search configuration <a id='configuration'></a>\n",
    "Define the hyperparameter space for a random search.  We will use a constant value for the number of estimators that is enough to let us reliably identify the best of the parameter configurations. Once we have the best combination, we will build a model using a larger number of estimators to boost the performance. The table below should give you an idea of the trade-off between the number of estimators and the modeling run time, model size, and model gain.\n",
    "\n",
    "| Estimators | Run time (s) | Size (MB) | Gain@1 | Gain@2 | Gain@3 |\n",
    "|------------|--------------|-----------|------------|------------|------------|\n",
    "|        100 |           40 |  2 | 25.02% | 38.72% | 47.83% |\n",
    "|       1000 |          177 |  4 | 46.79% | 60.80% | 69.11% |\n",
    "|       2000 |          359 |  7 | 51.38% | 65.93% | 73.09% |\n",
    "|       4000 |          628 | 12 | 53.39% | 67.40% | 74.74% |\n",
    "|       8000 |          904 | 22 | 54,62% | 67.77% | 75.35% |\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hyperparameter_sampling = RandomParameterSampling({\n",
    "    'ngrams': choice(range(1, 5)),\n",
    "    'match': choice(range(2, 41)),\n",
    "    'min_child_samples': choice(range(1, 31)),\n",
    "    'unweighted': choice('Yes', 'No')\n",
    "})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This hyperparameter space specifies a grid of 9,360 unique configuration points (4 `ngrams` X 39 `match` X 30 `min_child_samples` X 2 `unweighted`). We control the resources used by the search through specifying a maximum number of configuration points to sample as `max_total_runs`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "parameters"
    ]
   },
   "outputs": [],
   "source": [
    "max_total_runs = 96"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also possible to specify a maximum duration for the tuning experiment by setting `max_duration_minutes`. If both of these parameters are specified, any remaining runs are terminated once `max_duration_minutes` have passed.\n",
    "\n",
    "Specify the primary metric to be optimized as the gain at 3, and that it should be maximized. This metric is logged by the training script."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "primary_metric_name = \"gain@3\"\n",
    "primary_metric_goal = PrimaryMetricGoal.MAXIMIZE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The training script logs the metric throughout training, so we may specify an early termination policy. If no policy is specified, the hyperparameter tuning service will let all training runs run to completion. We use a median stopping policy that terminates runs whose best metrics on the tune dataset are worse than the median of the running averages of the metrics on all training runs, and we delay the policy's application until each run's fifth metric report."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "policy = MedianStoppingPolicy(delay_evaluation=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create an estimator <a id='estimator'></a>\n",
    "Create an estimator that specifies the location of the script, sets up its fixed parameters, including the location of the data, the compute target, and specifies the packages needed to run the script. It may take a while to prepare the run environment the first time an estimator is used, but that environment will be used until the list of packages is changed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "compute_target = 'hypetuning'\n",
    "estimator = Estimator(source_directory=os.path.join('.', 'scripts'),\n",
    "                      entry_script='TrainClassifier.py',\n",
    "                      compute_target=compute_target,\n",
    "                      conda_packages=['pandas==0.23.4',\n",
    "                                      'scikit-learn==0.21.3',\n",
    "                                      'lightgbm==2.2.1'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Put the estimator and the configuration information together into an HyperDrive run configuration object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hyperdrive_run_config = HyperDriveConfig(\n",
    "    estimator=estimator,\n",
    "    hyperparameter_sampling=hyperparameter_sampling,\n",
    "    policy=policy,\n",
    "    primary_metric_name=primary_metric_name,\n",
    "    primary_metric_goal=primary_metric_goal,\n",
    "    max_total_runs=max_total_runs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Azure Machine Learning Pipelines: Overview <a id='aml_pipeline_overview'></a>\n",
    "\n",
    "A common scenario when using machine learning components is to have a data workflow that includes the following steps:\n",
    "\n",
    "- Preparing/preprocessing a given dataset for training, followed by\n",
    "- Training a machine learning model on this data, and then\n",
    "- Deploying this trained model in a separate environment, and finally\n",
    "- Running a batch scoring task on another data set, using the trained model.\n",
    "\n",
    "Azure's Machine Learning pipelines give you a way to combine multiple steps like these into one configurable workflow, so that multiple agents/users can share and/or reuse this workflow. Machine learning pipelines thus provide a consistent, reproducible mechanism for building, evaluating, deploying, and running ML systems.\n",
    "\n",
    "To get more information about Azure machine learning pipelines, please read our [Azure Machine Learning Pipelines overview](https://aka.ms/pl-concept), or the [getting started notebook](https://github.com/Azure/MachineLearningNotebooks/blob/master/how-to-use-azureml/machine-learning-pipelines/intro-to-pipelines/aml-pipelines-getting-started.ipynb).\n",
    "\n",
    "Let's create a data reference for the raw data to be used in HyperDrive run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_folder = DataReference(datastore=ds, data_reference_name=\"data_folder\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the name of the model that will be registered."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_name = PipelineParameter(name=\"model_name\", default_value=\"FAQ_ranker\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create AML Pipeline Tuning Step <a id='aml_pipeline_tune_step'></a>\n",
    "We create a HyperDrive step in the AML pipeline to perform a search for hyperparameters. The `tune_estimators` pipeline parameter that controls the number of estimators used in tuning deliberately has a low default value for the speed of pipeline testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tune_step_name=\"tune_model\"\n",
    "tune_estimators = PipelineParameter(name=\"tune_estimators\", default_value=1)  # Set to 1000 when running the pipeline.\n",
    "tune_step = HyperDriveStep(\n",
    "    name=tune_step_name,\n",
    "    hyperdrive_config=hyperdrive_run_config,\n",
    "    estimator_entry_script_arguments=[\"--data-folder\", data_folder,\n",
    "                                      \"--estimators\", tune_estimators],\n",
    "    inputs=[data_folder],\n",
    "    allow_reuse=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create AML Pipeline Best Parameters Step <a id='aml_pipeline_bh_step'></a>\n",
    "This Python script step gets the best hyperparameters found by a HyperDrive step and writes out them to file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile scripts/Best_Hyperparameters.py\n",
    "\n",
    "from __future__ import print_function\n",
    "import os\n",
    "import json\n",
    "import argparse\n",
    "import pandas as pd\n",
    "\n",
    "from azureml.core import Run\n",
    "from azureml.pipeline.core import PipelineRun\n",
    "import azureml.core\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    \n",
    "    print(\"azureml.core.VERSION={}\".format(azureml.core.VERSION))\n",
    "\n",
    "    parser = argparse.ArgumentParser(description=\"Retrieve the hyperparameters \"\n",
    "                                     \"of the best run\")\n",
    "    parser.add_argument(\"--hd-step\", dest=\"hd_step\",\n",
    "                        help=\"the name of the HyperDrive step\")\n",
    "    parser.add_argument(\"--output-steps-data\", dest=\"output_steps_data\",\n",
    "                        help=\"to share data between different steps in a pipeline\",\n",
    "                        default=\"outputs\")\n",
    "    parser.add_argument(\"--hyperparameters\", help=\"the hyperparameters of the best run\",\n",
    "                        default=\"hyperparameters.json\")\n",
    "    parser.add_argument(\"--delete\",\n",
    "                        help=\"comma-separated list of hyperparameters to remove\",\n",
    "                        default=\"estimators\")\n",
    "    args = parser.parse_args()\n",
    "    \n",
    "    # Path to write best hyperparameters.\n",
    "    os.makedirs(args.output_steps_data, exist_ok=True)\n",
    "    hyperparameters_path = os.path.join(args.output_steps_data, args.hyperparameters)\n",
    "    \n",
    "    # Get the HyperDrive run.\n",
    "    run = Run.get_context()\n",
    "    print(run)\n",
    "    pipeline_run = PipelineRun(run.experiment, run.parent.id)\n",
    "    print(pipeline_run)\n",
    "    hd_step_run = pipeline_run.find_step_run(args.hd_step)[0]\n",
    "    print(hd_step_run)\n",
    "    hd_run = list(hd_step_run.get_children())[0]\n",
    "    print(hd_run)\n",
    "    \n",
    "    # Get the best run.\n",
    "    hd_run.wait_for_completion(show_output=True, wait_post_processing=True)\n",
    "    best_run = hd_run.get_best_run_by_primary_metric()\n",
    "    if best_run is None:\n",
    "        raise Exception(\"No best run was found\")\n",
    "    print(best_run)\n",
    "    \n",
    "    # Get its hyperparameters as a dict.\n",
    "    parameter_values = best_run.get_details()[\"runDefinition\"][\"arguments\"]\n",
    "    best_parameters = dict(zip(map(str.strip, parameter_values[::2], \n",
    "                                   len(parameter_values[::2]) * [\"-\"]),\n",
    "                               parameter_values[1::2]))\n",
    "    \n",
    "    # Remove these hyperparameters.\n",
    "    for key in args.delete.split(\",\"):\n",
    "        if key in best_parameters:\n",
    "            del best_parameters[key]\n",
    "        \n",
    "    # Print out the hyperparameters.\n",
    "    print(\"Best run hyperparameters:\")\n",
    "    print(pd.Series(best_parameters, name=\"Value\").to_frame())\n",
    "    \n",
    "    # Write them out to file.\n",
    "    print(\"Writing best run hyperparameters to {}\".format(hyperparameters_path))\n",
    "    with open(hyperparameters_path, \"w\") as fp:\n",
    "        json.dump(best_parameters, fp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Creating PythonScript Step for AML pipeline to get the best run's hyperparameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bh_step_name = \"best_parameters\"\n",
    "bh_steps_data = PipelineData(\"bh_steps_data\", datastore=ds)\n",
    "bh_hyperparameters_file = \"hyperparameters.json\"\n",
    "bh_run_config = RunConfiguration(conda_dependencies=CondaDependencies.create(\n",
    "    conda_packages=[\"pandas\"],\n",
    "    pip_packages=[\"azure-cli\", \"azureml-sdk\", \"azureml-pipeline\"]))\n",
    "bh_run_config.environment.docker.enabled = True\n",
    "bh_step = PythonScriptStep(\n",
    "    name=bh_step_name,\n",
    "    script_name=\"Best_Hyperparameters.py\",\n",
    "    compute_target=compute_target,\n",
    "    source_directory=os.path.join(\".\", \"scripts\"),\n",
    "    arguments=[\"--hd-step\", tune_step_name,\n",
    "               \"--output-steps-data\", bh_steps_data,\n",
    "               \"--hyperparameters\", bh_hyperparameters_file],\n",
    "    outputs=[bh_steps_data],\n",
    "    runconfig=bh_run_config,\n",
    "    allow_reuse=False)\n",
    "bh_step.run_after(tune_step)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create AML Pipeline Best Model Step <a id='aml_pipeline_estimator_step'></a>\n",
    "This step passes the hyperparameters file from the previous step to the training script to create the best model. The `best_estimators` pipeline parameter that controls the number of estimators used in getting the best model deliberately has a low default value for the speed of pipeline testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bm_step_name=\"best_model\"\n",
    "bm_estimators = PipelineParameter(name=\"best_estimators\", default_value=1)  # Set to 8000 when running the pipeline\n",
    "bm_estimator = Estimator(source_directory=os.path.join('.', 'scripts'),  # Use a new Estimator as a bug workaround\n",
    "                         entry_script='TrainClassifier.py',\n",
    "                         compute_target=compute_target,\n",
    "                         conda_packages=['pandas==0.23.4',\n",
    "                                         'scikit-learn==0.21.3',\n",
    "                                         'lightgbm==2.2.1'])\n",
    "bm_step = EstimatorStep(\n",
    "    name=bm_step_name,\n",
    "    estimator=bm_estimator,\n",
    "    estimator_entry_script_arguments=[\"--data-folder\", data_folder,\n",
    "                                      \"--estimators\", bm_estimators,\n",
    "                                      \"--input-steps-data\", bh_steps_data,\n",
    "                                      \"--hyperparameters\", bh_hyperparameters_file,\n",
    "                                      \"--save\", model_name],\n",
    "    compute_target=compute_target,\n",
    "    inputs=[data_folder, bh_steps_data],\n",
    "    allow_reuse=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create AML Pipeline Register Model Step <a id='aml_pipeline_rm_step'></a>\n",
    "This Python script step registers the best model found created by an estimator step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile scripts/Register_Model.py\n",
    "\n",
    "from __future__ import print_function\n",
    "import os\n",
    "import json\n",
    "import argparse\n",
    "import pandas as pd\n",
    "\n",
    "from azureml.core import Run\n",
    "from azureml.pipeline.core import PipelineRun\n",
    "import azureml.core\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    \n",
    "    print(\"azureml.core.VERSION={}\".format(azureml.core.VERSION))\n",
    "\n",
    "    parser = argparse.ArgumentParser(description=\"Register the model created by\"\n",
    "                                     \" an estimator step\")\n",
    "    parser.add_argument(\"--es-step\", dest=\"es_step\",\n",
    "                        help=\"the name of the estimator step\")\n",
    "    parser.add_argument(\"--outputs\", help=\"the model file outputs directory\")\n",
    "    parser.add_argument(\"--model-name\", dest=\"model_name\",\n",
    "                        help=\"the model file base name\")\n",
    "    args = parser.parse_args()\n",
    "    \n",
    "    model_name = args.model_name\n",
    "    model_file = \"{}.pkl\".format(model_name)\n",
    "    model_path = os.path.join(args.outputs, model_file)\n",
    "    \n",
    "    # Get the Estimator run.\n",
    "    run = Run.get_context()\n",
    "    print(run)\n",
    "    pipeline_run = PipelineRun(run.experiment, run.parent.id)\n",
    "    print(pipeline_run)\n",
    "    es_run = pipeline_run.find_step_run(args.es_step)[0]\n",
    "    print(es_run)\n",
    "    \n",
    "    # Register the Estimator step\"s model\n",
    "    es_run.wait_for_completion(show_output=True)\n",
    "    model = es_run.register_model(model_name=model_name, model_path=model_path)\n",
    "    print(\"Estimator model registered\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Creating PythonScript Step for AML pipeline to register the best model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "rm_step_name = \"register_model\"\n",
    "rm_run_config = RunConfiguration(conda_dependencies=CondaDependencies.create(\n",
    "    conda_packages=[\"pandas\"],\n",
    "    pip_packages=[\"azure-cli\", \"azureml-sdk\", \"azureml-pipeline\"]))\n",
    "rm_run_config.environment.docker.enabled = True\n",
    "rm_step = PythonScriptStep(\n",
    "    name=rm_step_name,\n",
    "    script_name=\"Register_Model.py\",\n",
    "    compute_target=compute_target,\n",
    "    source_directory=os.path.join(\".\", \"scripts\"),\n",
    "    arguments=[\"--es-step\", bm_step_name,\n",
    "               \"--outputs\", \"outputs\",\n",
    "               \"--model-name\", model_name],\n",
    "    runconfig=rm_run_config,\n",
    "    allow_reuse=False)\n",
    "rm_step.run_after(bm_step)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create & Run a Pipeline <a id='create_aml_pipeline'></a>\n",
    "When we specify the bm_step, Pipeline walks the dependency graph to include the other steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "experiment_name = \"hypetuning\"\n",
    "exp = Experiment(workspace=ws, name=experiment_name)\n",
    "pipeline = Pipeline(workspace=ws, steps=[rm_step])\n",
    "pipeline.validate()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the pipeline before publishing it. Wait for the run to complete."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "pipeline_run = exp.submit(pipeline, continue_on_step_failure=True)\n",
    "pipeline_run.wait_for_completion(show_output=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Publish The Pipeline  <a id='publish_aml_pipeline'></a>\n",
    "You may read more about why to publish a pipeline and how the published pipeline can be triggered [here](https://docs.microsoft.com/en-us/azure/machine-learning/service/concept-ml-pipelines)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "published_pipeline = pipeline.publish(name=\"HyperDrive Pipeline\",\n",
    "                                      description=\"HyperDrive Pipeline\",\n",
    "                                      continue_on_step_failure=True)\n",
    "published_pipeline.endpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run published pipeline using its REST endpoint <a id='run_publish_aml_pipeline'></a>\n",
    "This step shows how to call the REST endpoint of a published pipeline to trigger the pipeline run. You can use this method in programs that do not have the Azure Machine Learning SDK installed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "aad_token = auth.get_authentication_header()\n",
    "rest_endpoint = published_pipeline.endpoint\n",
    "print(\"You can perform HTTP POST on URL {} to trigger this pipeline\".format(rest_endpoint))\n",
    "response = requests.post(rest_endpoint, \n",
    "                         headers=aad_token, \n",
    "                         json={\"ExperimentName\": experiment_name,\n",
    "                               \"RunSource\": \"SDK\"})\n",
    "run_id = response.json()[\"Id\"]\n",
    "print(run_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [next notebook](08_Tear_Down.ipynb) shows how to delete the components created by this tutorial."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}