Use metrics package in the notebook. #2130

* updated the notebook * updated the e2e sample notebook * removed test value * linting * removed unwanted cell Co-authored-by: Ali M <amah@microsoft.com>

* Adding test for AML in a day * Update tutorials-azureml-in-a-day-azureml-in-a-day.yml * Update tutorials-azureml-in-a-day-azureml-in-a-day.yml * Update tutorials-azureml-in-a-day-azureml-in-a-day.yml * Update azureml-in-a-day.ipynb * Adding tests for more tutorials * Update azureml-in-a-day.ipynb * Adding README files * Update readme.py * Update azureml-in-a-day.ipynb * Added Description to all files * Update README.md * Update readme.py

…#1901) * Updated automl regression notebook to work with the latest matplotlib * Updated generate_workflows.py based on move to v1 folder * Updated automl_env files for SDK 1.47.0

* Revert "Revert "Update v1 Many Models and HTS Notebook" (#1763)" This reverts commit 13f44d0. * fix PR comments * added new line at end * fix black issue Co-authored-by: Rahul Kumar <rahulkuma@microsoft.com>

…on (#1904) * renaming multi-label image classification git workflow; correcting yaml name * Updates in github workflows from PR (#1879) after running cli/readme.py * Revert "Updates in github workflows from PR (#1879) after running cli/readme.py" This reverts commit 3bdee93. * Updates in github workflows after running cli/readme.py * Updates in github workflows for model/job-model-as-output and model/job-model-as-input * Removing workflows related to model-job-model-as-input/output

* Security scanner for Azure ML Compute Instance detecting malware and vulnerabilities * Apply black formatting * Apply doc template * Address PR feedback * Update doc * Update README.md * Address PR feedback

…age (#1895) * Adding checks for k8 extension and check for attaching the compute to workspace * Adding a note on the top of all the workflows about regenerating using python script * Update header in the workflows * Adding the template for account and container * Update the resource names for November to recreate resources * Adding some basic samples for autogenerated SDKv2 for TypeSript language

* Adding checks for k8 extension and check for attaching the compute to workspace * Adding a note on the top of all the workflows about regenerating using python script * Update header in the workflows * Adding the template for account and container * Update the resource names for November to recreate resources * Adding some basic samples for autogenerated SDKv2 for TypeSript language * Reformat the files to address the linting issue

Add metadata to cells for docs

* Rename automl-nlp-text-classification-multiclass-task-sentiment-mlflow.ipynb to automl-nlp-classification-multiclass-sentiment-mlflow.ipynb remove text and task from the name * Rename automl-nlp-text-classification-multiclass-task-sentiment.ipynb to automl-nlp-classification-multiclass-sentiment.ipynb * Rename automl-nlp-text-classification-multilabel-task-paper-cat.ipynb to automl-nlp-classification-multilabel-paper-cat.ipynb * remove classification in the names * workflow file changes and README.md * remove original workflow files with long name

* Update data.ipynb * Update data.ipynb * Update data.ipynb * Update data.ipynb

Co-authored-by: Nikolay Rovinskiy <nirovins@microsoft.com>

* Update workspace sample notebook * Fix errors in the sample notebook * Update workspace.ipynb * Update workspace.ipynb fix code formatting * update formatting> * Update the sdk_helper to clear the hacks * revert sdk_helper hacks * Update workspace.ipynb update ws name in the last cell for cleanup * Update black formatting :*( * Update kernelspec * change to python310-sdkv2 Co-authored-by: Yisheng Yao <yishyao@microsoft.com>

* add metadata to cells * delete comma

* Update cli-responsibleaidashboard-housing-classification.yml * Update cli-responsibleaidashboard-programmer-regression.yml

…epo (#1941)

* add wait() to delete deployment * rename the endpoint name * add back time string to endpoint * reformat notebook code * data.ipynb is not related to this PR, but build failed due to the reformat requirement * endpoint name number of char 3 to 32 * Update data.ipynb * revert change to data.ipynb * revert change to data.ipynb

Adjust syntax

*added preprocessing component/node in automl-pipelines example for sdk and cli

* Fix mlflow-for-batch-tabular notebook * Wait for job to finish to download * Update notebook * Update path * Fix format Co-authored-by: nancy-mejia <namejia@microsoft.com>

* Fix custom output batch notebook * Update format Co-authored-by: nancy-mejia <namejia@microsoft.com>

* add metadata to another cell * fix metadata * fix syntax

… a job) (#1952) Co-authored-by: Pritam Kumar Das <pritamd@microsoft.com>

* update component env style * rename and update env yaml

…pecified folder (#1911) * AutoML Images: Generating MLTable on the fly; downloading and prepping the data in user specified folder * AutoML Images: Clearing output of notebooks * AutoML Images: Cleaning up * AutoML Images: black and black-nb formatting * AutoML Images: refactoring * AutoML Images: black formatting * AutoML Images: black formatting * AutoML Images: Fixing typo; refactoring * Reverting metadata updates

…l-as-output (#1949)

* updated model example structure * remove on needed file reference * updated path * added updated workflows * Update README.md

* test fixes

* Remove IcMs from readme.py * Update workflows

Co-authored-by: nancy-mejia <namejia@microsoft.com>

* move mlflow examples into python sdk v2 folder * checking black and readme.py * readme Co-authored-by: Facundo Santiago <fasantia@microsoft.com>

* Update responsibleaidashboard-diabetes-decision-making.ipynb * Update responsibleaidashboard-diabetes-regression-model-debugging.ipynb * Update responsibleaidashboard-housing-classification-model-debugging.ipynb * Update responsibleaidashboard-housing-decision-making.ipynb * Update responsibleaidashboard-programmer-regression-model-debugging.ipynb

* Update cli-responsibleaidashboard-housing-classification.yml * Update cli-responsibleaidashboard-programmer-regression.yml

* updated mm related notebooks * fix format for black * fix PR comments Co-authored-by: Rahul Kumar <rahulkuma@microsoft.com>

* v1 multi-class batch scoring notebook * v2 vision notebooks * minor updates * minor version updates * column name update * links updated * removed multi-class batch scoring notebook * link updates * format changes * format changes 1 * format changes * format changes * minor update * minor updates * format changes * format changes * minor updates * link update * minor update * minor update * minor update * format changes

* Multiclass mlflow image classification notebook with AutoMode. * AutoMode for the rest of the notebooks. * AutoMode via CLI. * Reformat with black. * Wait after AutoMode runs. * Patch compute cluster in configuration files. * Add GitHub actions for AutoMode CLI configuration files. * Clarify description of AutoMode. * Patch merge bugs. * Lead with paragraph about AutoMode. * Patch multilabel notebook. * Lead with Automode in CLI examples. * Move some configuration functions in AutoMode section. * v1 multi-class batch scoring notebook * v2 vision notebooks * minor updates * minor version updates * column name update * Attempt to fix AutoMode CLI example for multiclass classification. * Reformat three main image notebooks. * Fix setup for AutoMode YAML job definitions. * Unbreak scoring code and data (fix bad merge). * Newline character at end of multiclass-batch-scoring file. * More fixing of bad merge. * Newline character at end of multiclass batch scoring notebook. * Use the term AutoMode side-by-side with automatic sweeps in notebooks. * Use the term AutoMode side-by-side with automatic sweeps in CLI instructions. * Make YAML files for AutoMode look more similar to regular sweeps. Co-authored-by: Vadthyavath Ram <7171558+vadthyavath@users.noreply.github.com>

* avoid warning * update reason for dropping column * update data_preprocessing_tabular script Co-authored-by: Rahul Kumar <rahulkuma@microsoft.com>

* distributed training of yolov5 files * Add files via upload editing connecting to workspace cells. * Add files via upload ran black on the notebook for formatting * Add files via upload edited the handle to workspace (ml_client) * Add files via upload renamed compute_name to "gpu-cluster" * Delete sdk/python/jobs/single-step/pytorch/distributed-training-yolov5/yolov5/datasets directory deleting dataset directory * Delete sdk/python/jobs/single-step/pytorch/distributed-training-yolov5/yolov5/data directory deleted data folder and is uploaded in https://azuremlexamples.blob.core.windows.net/datasets/yolov5/data/ * Add files via upload Changed the input data path to azure blob storage * Delete .pre-commit-config.yaml

* Remove timeout for AutoMode examples. * Reformat. * Update sdk/python/jobs/automl-standalone-jobs/automl-image-classification-multiclass-task-fridge-items/automl-image-classification-multiclass-task-fridge-items.ipynb Co-authored-by: sharma-riti <52715641+sharma-riti@users.noreply.github.com> * Update sdk/python/jobs/automl-standalone-jobs/automl-image-classification-multilabel-task-fridge-items/automl-image-classification-multilabel-task-fridge-items.ipynb Co-authored-by: sharma-riti <52715641+sharma-riti@users.noreply.github.com> * Update sdk/python/jobs/automl-standalone-jobs/automl-image-object-detection-task-fridge-items/automl-image-object-detection-task-fridge-items.ipynb Co-authored-by: sharma-riti <52715641+sharma-riti@users.noreply.github.com> * Update sdk/python/jobs/automl-standalone-jobs/automl-image-instance-segmentation-task-fridge-items/automl-image-instance-segmentation-task-fridge-items.ipynb Co-authored-by: sharma-riti <52715641+sharma-riti@users.noreply.github.com> * Fix the suggestions. Co-authored-by: sharma-riti <52715641+sharma-riti@users.noreply.github.com>

* install docker dependency * install docker package before local deployment * set single step spark directly * update filter * fix wrong directory * remove 4b_datastore_datapath_uri

* Directly take the csv file from the batch endpoint * Change file extensions and comments * Fix * Add separator notation. * Add option to use the comma instead of tab separated files. Co-authored-by: Nikolay Rovinskiy <nirovins@microsoft.com>

* fix: AZ_BATCH_JOB_ID => AZUREML_RUN_ID * fix: black

Co-authored-by: Nikolay Rovinskiy <nirovins@microsoft.com>

changed from `ml_client.begin_create_or_update` to `ml_client.online_endpoints.begin_create_or_update`

* AutoML Images: Add mltable asset links; update docs * renaming classes back to labels * Reverting metadata update * Fixing typo * Fixing typo * Refactoring mltable to MLTable

update max_trials for cli as well

Co-authored-by: Nick Mecklenburg <nmecklenburg@microsoft.com>

* add codegen example v2 * add explanation * add mlflow import * add mlflow in yaml * add tracking uri

* remove RAI examples under Pipelines * remove related GitHub Actions

* move mlflow examples into python sdk v2 folder * checking black and readme.py * readme * feat: new example for MLflow custom deployment * build * fix: PR comments Co-authored-by: msakande <17515964+msakande@users.noreply.github.com>

* try new distribution config * update CLI example config

#1990) * [bhavanatumma/mlflow-model-deploy-v3] - conflict resolution with new and workflow fix * [bhavanatumma/mlflow-model-deploy-v3] - papermill fix for the flow * [bhavanatumma/mlflow-model-deploy-v3] - papermill fix for the flow * [bhavanatumma/mlflow-model-deploy-v3] - papermill fix for the flow Co-authored-by: bhavanatumma <bhavanatumma@microsoft.com>

* fiw wrong filter * add more time for automl-dynamic job * remove duplicate step

…ted for the first time (#1995) * Address LROPoller error message that appears when cpu-cluster is created for the first time * Removing local deployments

* feat: changes * comments * formatting * fix * fix: adding predict with deployment client

* Avoid use common path as output path in azureml-examples * update * update * update * update readme Co-authored-by: Ying Chen <2601502859@qq.com>

Co-authored-by: Miseon Park <mipark@microsoft.com>

* Refactor the script for adding identity with compute

Change begin delete statement to address intermittent failures in: - Safe Rollout - SAI/UAI

* Remove archiving model step No need to archive an anonymous model registered by the system. Eventually, this action will trigger an error given it is a stage transition not allowed. * Removing archive model step No need to archive an anonymous model registered by the system. Eventually, this action will trigger an error given it is a stage transition not allowed. * Update online-endpoints-triton-cc.ipynb * Update online-endpoints-triton.ipynb

Co-authored-by: Nikolay Rovinskiy <nirovins@microsoft.com>

Co-authored-by: Ubuntu <fvm@azureml.hdto2glhnacenmzoxwwt5zmpza.phxx.internal.cloudapp.net>

data format change

* Update auto-ml-forecasting-bike-share.ipynb * Update automl-forecasting-orange-juice-sales-mlflow.ipynb * Update automl-forecasting-task-energy-demand-advanced.ipynb * Update automl-forecasting-in-pipeline.ipynb

* Decode the image read to utf-8 encoding * Added escape char before double quotes in raw .ipynb file * Formated notebooks as per black-nb

* delay registry component workflows by 59 min * fix cleanup.sh Co-authored-by: Ubuntu <fvm@azureml.hdto2glhnacenmzoxwwt5zmpza.phxx.internal.cloudapp.net> Co-authored-by: Jun Qi <junqi@microsoft.com>

* Add SDK release candidate * Add CLI release candidate * Update sdk release candidate * Add Default Values for Environment Variables in Workspace Connections Notebook (#2010) * Add Default Value for ACR Username/Password * add defaults for git_pat and python_feed_sas * Update connections.ipynb * Update connections.ipynb * update workflow file * add step in readme to update workflow with env vars * revert deletion * revert changes to notebook * Update connections.ipynb * Update sdk release candidate * Update SDK release candidate * Use released versions of azure-ai-ml (version 1.3.0) and ml extension (version 2.13.0) Co-authored-by: Neehar Duvvuri <40341266+needuv@users.noreply.github.com>

…as Linux (#2035) * Updated the cli readme.py to support Windows

…cs (#2036)

This fixed the kernel for some notebooks and cleaned some whitespace in workflow files.

* add deepspeed example * fix mlflow logging in train.py * move examples to cli folder * move to jobs, add job.yaml files * generate workflow file * add comment in train.py * add latest tag to env * try different way of starting run * set up job as pipeline * move to pipelines folder * add temp yaml file to pass validation * modify readme.py to support deepspeed * move generate-yml script * recreate workflow files. * move generated key location * change environment * change env for both examples * fix env * try v100 computes * create nd40 cluster * add compute create line * change max number of nodes * change data directory * change command to block * move data files into src * remove unused data thing * change data file type * fix mlflow get run fail * change command style in job * change location of job * change generate key path * change code: path * rename results folder * change back key path * change max number of nd40 nodes * try moving autotuning folder into src * change output file name * try overwriting output * add dockerfile for custom env * change env in yaml * update README and move autotune example * move dockerfile to parent dir

* resolve log_param conflict issue * reslove black format --------- Co-authored-by: Clement Wang <clwan@microsoft.com>

* Changed schedule to reduce quota usage * Changed to schedule every 12 hours

#2054)

* rectified resource sections under new rules * remove redundant changes * remove redundant changes --------- Co-authored-by: Mingda Jia <mingj@microsoft.com>

* fix code cell If creating the compute cluster, the print didn't work. This change fixes that. (Default code won't hit this, because the compute cluster already exists. But only occurs when that default cluster is not present.) * Update azureml-in-a-day.ipynb * Update azureml-in-a-day.ipynb * move print and revert var change

* add timeout for deepspeed jobs * re format readme with black * change timeout length * change dockerfile to use acpt image * add training custom env * fix hostfile bug * fix bash generation * address comments * increase number of gpus being used * make sure deepspeed is upgraded to latest version * write to hostfile in single process

* Update PRS examples * Update PRS examples * Update prs examples * Update PRS examples * Update format * Update by comments * fix sdk default datastore issue. * Update wording

* remove DLLogger dependency * update * fix black --------- Co-authored-by: Clement Wang <clwan@microsoft.com>

* Update cli-responsibleaidashboard-housing-classification.yml * Update cli-responsibleaidashboard-programmer-regression.yml

* Init * Readme

* [add-codegen-block] - re-added codegen part * [bhavanatumma/add-codegen-block] - review comments --------- Co-authored-by: bhavanatumma <bhavanatumma@microsoft.com>

…trics (#2076) So far, the scorecard metrics were classification metrics leading to a validation error during execution. Changing to regression metrics fixes the problem.

* add hostfile check with only one node * change logic * add changes to training example

* update data store path * output file * sleep * using folder

* Update tensorflow version to 2.1.4 * Revert change for failing cli examples * Upgrade tensorflow in cli jobs * Revert upgrade tensorflow in failing jobs

* delete this .sh to unblock the PR check. The creation test is somewhere else. * deleted related workflow

* install dependency * unpin version * refine * refine * refine * pin 1.2.0 * refine * tf1 --------- Co-authored-by: Xiaole Wen <xiwe@microsoft.com>

…r. (#2049) * Add role assignment cleanup * Add verbose logging for troubleshooting the exit code 100 issue * bypass the error for grub-efi-amd64-signed * Adding the redirections back

* minor updates added to AzureML in day file * add text to modify notebook * fix meta data * fix meta data --------- Co-authored-by: Kenwiggan <kwiggan@microsoft.com> Co-authored-by: Sheri Gilley <sgilley@microsoft.com>

…ow folder rename (#2091)

) * update * update * [PipelineJob]: Add Typescript/JavaScript samples for SDKv2 (#2087) * add sample * update * update * fix indent * Update sdk/typescript/src/resources/jobs/pipelines/pipelineJobCreateOrUpdateSample.ts Co-authored-by: guanghechen <42513619+guanghechen@users.noreply.github.com> * update gitignore * update folder structure * remove not supported delete sample --------- Co-authored-by: guanghechen <42513619+guanghechen@users.noreply.github.com> Co-authored-by: Clement Wang <clwan@microsoft.com> --------- Co-authored-by: Ying Chen <2601502859@qq.com> Co-authored-by: Han Wang <phoenix.seek@gmail.com> Co-authored-by: guanghechen <42513619+guanghechen@users.noreply.github.com> Co-authored-by: Clement Wang <clwan@microsoft.com>

…mple (#2089) * Update responsibleaidashboard-diabetes-decision-making.ipynb * Update responsibleaidashboard-diabetes-regression-model-debugging.ipynb * Update responsibleaidashboard-housing-classification-model-debugging.ipynb * Update responsibleaidashboard-housing-decision-making.ipynb * Update responsibleaidashboard-programmer-regression-model-debugging.ipynb

* changed link in enegy demand * updated URL in remaning notebooks

* Updated automl_env.yml to SDK 1.49.0 * Simplified automl_env_linux.yml * Changed validation to use ubuntu 20.04 * Updated papermill to avoid conflict with packaging package

* init using-mltable * print data * deleted created artifact * following contributing guidelines * remove from_config for ws details * formatted train.py * removing files not in this PR * added local-to-cloud example * local-to-cloud example added * removed nuasance files

* change training example * fix workflow * fix syntax * update readmes * edit readme * remove env variables

* initial commit after deleting previous branch due to rebase issues * data prep notebook * generated yml files and modified README.md * changed the name of dataset name for data from public blob. This is necessary because the data-prep notebook uploads the data with the same name and causes version mismatch. Hence, the test and train data have different grains * added create new version option to dataset registration * addressed all PR comments * removed variable that was referenced before declaration * changed compute target creation cell from markdown to code

* new schema changes * black format

* First V2 notebook validation * Updated MLFlow download_artifact to remove warning

* Moved V1 validation definitions to under .github for consistency with V2 validation definitions

* Added validation for more V2 notebooks * Fixed deprecation warnings * Add mlflow for validation

* Changed training V1 notebooks to ubuntu 20.04 * Updated requirements.txt to speed up install * Updated creds, workspace and resource group * Added pin for mlflow-skinny

* delete registry after creation * always run deletion * init env * set working dir * add sdk_helper * add sub_id * edit for debugging * try debugging * try deletion inside the notebook * delete in the same step * print registry name for debugging * remove deletion for testing. * change everything back for final pr * modify format * make deletion inside the ipnb * try sdk deletion * try sdk * use verified api. * change format * Update registry-create.ipynb

* Update share-models-components-environments.ipynb * Update online-endpoints-keyvault.ipynb * Update share-models-components-environments.ipynb

Adding best-practices for large-scale deep learning workloads.

New tutorial series

Co-authored-by: nancy-mejia <namejia@microsoft.com>

Fixing broken links under BestPractices folder, used relative paths.

* Added basic validation for automl nlp notebooks * Added exception for Readonly warning * Updated download_artifacts * Fixed check endpoint exception

* more MLTable example notebooks * included requirements file install * updated requirements file * pinned package versions * fixed typo * fixed the file path * upversioned azureml-dataprep * in cmd job ensure latest asset version is pulled * updated versioning strategy * removed delta example * remove delta gh workflow as example removed

Co-authored-by: nancy-mejia <namejia@microsoft.com>

* Table of content fix and added smoke yaml * added sample page_type * changed description * updated description

* Update README.md * resolved comments * removed space * Changed Monitoring and optimization to Bold as well.

fix typo

* Reindexing the quantiles dataframe * Updated fix for reindex issue

…2073) * [bhavanatumma/forecasting-revert] - revert batch deployment changes * [bhavanatumma/forecasting-revert] - revert batch deployment changes * [bhavanatumma/forecasting-revert] - edited readme * [bhavanatumma/forecasting-revert] - adding workflow * [bhavanatumma/forecasting-revert] - readme modification * update image for inference --------- Co-authored-by: bhavanatumma <bhavanatumma@microsoft.com> Co-authored-by: Rahul Kumar <rahulkuma@microsoft.com>

* Added validation for forecast automl notebooks * Ignore downloading artifact message

* remove experiment_name * remove bullet

* batch endpoint CLI folders --------- Co-authored-by: santiagxf <fasantia@microsfot.com>

* update * update

* batch endpoint CLI folders * fixes * docs sync * black --------- Co-authored-by: santiagxf <fasantia@microsfot.com>

* fixing languages in the sample metadata * fixing title in the regression automl CLI pipeline sample

…is-datastore-file (#2179)

#2178) * giving unique titles to each of the pipelines samples * improved description on TF sample

* Clean old resource group as well * Added escape in regular expression for "AzureML Metrics Writer (preview)". So that more Role Assignments are cleaned

Co-authored-by: nancy-mejia <namejia@microsoft.com>

* creating first level markdown title.

* init * add identity * init * re-structure * update * update * update readme * update * bring back spark component example * fix typos * fix typo * add inputs * add tag for doc * add CI * remove pipeline component * remove is_deterministic * reformat --------- Co-authored-by: Clement Wang <clwan@microsoft.com>

* For data-import feature public preview * Fixed broken smoke test test case * Fixing failed smoketests * smoke test fixes * format fixes * Fix smoketest error * updated package to fix smoketest * smoketest fixes * smoke test format fixes * format corrected * package name corrected * package name correction

* example added to use registred component to create pipeline * reformatted notebook with black * reformatted notebook with black * reformatted notebook with black * reformatted notebook with black * changed model example from ai4bharat to hugging face * changed model example from ai4bharat to hugging face * Update automl-classification-bankmarketing-in-pipeline.ipynb * Update automl-classification-bankmarketing-in-pipeline.ipynb * updated notebook * Update pipeline_with_registered_components.ipynb * Update pipeline_with_registered_components.ipynb * Update pipeline_with_registered_components.ipynb * Update pipeline_with_registered_component_in_registry.ipynb * Update bert-base-uncased-test.json * notebook added for pipeline component * notebook added using pipeline component and individual components * notebook added using pipeline component and individual components * notebook update with bug bash review * Update Model Import Pipeline.ipynb * Update pipeline_with_registered_components.ipynb * Update pipeline_with_registered_component_in_registry.ipynb * Update pipeline_with_registered_components.ipynb * Update Model Import Pipeline.ipynb * Update pipeline_with_registered_component_in_registry.ipynb * notebook update * notebook updated with review changes * notebook updated with review changes * review comments implemented * updated the folder structure * Update import_model_into_registry.ipynb * notebook updated * code updated * workflow added * workflow added * registyr updated * notebook updsted for preview-test1 * review comments addressed * review comments addressed * error for false task value updated in description * cell added to show the registered model * header fonts changed * nit formatted * Update online-endpoints-simple-deployment.ipynb * Update online-endpoints-simple-deployment.ipynb * Update online-endpoints-simple-deployment.ipynb * Update online-endpoints-simple-deployment.ipynb * headers changed * registry name changed * string to int for version logic * nit formatted * nit formatted * Removed a modified file from pull request * Removed a modified file from pull request * Update import_model_into_registry.ipynb * Update import_model_into_registry.ipynb * Update import_model_into_registry.ipynb * Update import_model_into_registry.ipynb * Update import_model_into_registry.ipynb * image updated in notebook * Update import_model_into_registry.ipynb * Update import_model_into_registry.ipynb * Update import_model_into_registry.ipynb * Update import_model_into_registry.ipynb * Update import_model_into_registry.ipynb * Update import_model_into_registry.ipynb

* Add workflows for all endpoints * Fixed online vs batch endpoint workflow command bug * Testing workflows catting endpoint .yml files * Removed all newly generated non-endpoint related workflows that were under endpoint folder * Testing workflows deleting endpoints before creation if exists * Add continue on failure for delete endpoint * Testing replacing endpoint names during action and deleting * Testing extra delete section * Testing -n over -f * Testing new endpoint names * Reduced endpoint names to 32 chars max * Replaced lowercase endpoint_name with uppercase, added quotes * Removed uai create endpoint workflow * Added create deployment sections to workflows if exist * Standardized all usages of conda*.yml to use .yaml ending in cli/endpoints * Standardized all usages of conda*.yml to use .yaml ending in cli/endpoints * Used same modified endpoint name for deployments in workflow * Temporarily removed deployments and endpoints which were failing due to malformed .yml and other transient reasons * Removed extra print statement and comment * Added missing name for deployment * Added 1-sai-create-endpoint to ignore list

* updating python version (#2196) Co-authored-by: Raghu Ramaswamy <raghura@microsoft.com> * Added endpoint paths to trigger create deployment workflows to test Python ver change * Removed unrelated workflows that are failing * Removed unrelated workflows * Ignore kubernetes green deployment * Fixed typo in pip and joblib versions --------- Co-authored-by: Raghu Ramaswamy <13340619+raghutillu@users.noreply.github.com> Co-authored-by: Raghu Ramaswamy <raghura@microsoft.com>

* Rename conda.yml to conda.yaml * Re-ran sdk/python/readme.py to update workflows

* Shorten validation configuration file paths

* use nc6s_v3 gpu for OD tasks * update gpu name

* add nebulaml examples readme * add nebulaml examples readme * remove unused readme * add some fix * run python readme.py * add cifar10 case * Add README file and other examples of Nebula * reformat files * reformat files in the nebulaml examples * reformat files in the nebulaml examples --------- Co-authored-by: luwei <luwei@microsoft.com> Co-authored-by: Ziqi Wang <zikeiwong@outlook.com>

* Adding OBO sample * Formatted file * Review comments to add readme * Adding readme for obo sample * Updating langauge for sample * Update cli/jobs/single-step/on-behalf-of/README.md Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com> * Update cli/jobs/single-step/on-behalf-of/README.md Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com> * Update job.yaml --------- Co-authored-by: Bala P V <33712765+balapv@users.noreply.github.com>

… nirovins/add_metrics_package

* fix od batch scoring notebook * format file * update broken path

* Update RAI component version to 0.7.0 * Update cli-responsibleaidashboard-programmer-regression.yml

* Update deploy-and-run.sh * Update deploy-and-run.sh * Update deploy-and-run.sh * Update deploy-and-run.sh * Update deploy-and-run.sh

* Upgrade the mode training environment for RAI CLI examples * Update train_programmers.yml

* Change torch_nebula to nebulaml * Renaming the doc as README file * Rename the package name from torch_nebula to nebulaml --------- Co-authored-by: xiaoranli <xiaoranli@microsoft.com> Co-authored-by: Ziqi Wang <zikeiwong@outlook.com>

* foundation model samples - text classification - emotion detection `date` * foundation model samples - text classification - emotion detection Wed Mar 15 23:03:38 PDT 2023 * foundation model samples - text classification - emotion detection Wed Mar 15 23:13:42 PDT 2023 * foundation model samples - text classification - emotion detection Wed Mar 22 11:15:27 PDT 2023 * foundation model samples - text classification - emotion detection Wed Mar 22 17:03:46 PDT 2023 * foundation model samples - text classification - emotion detection Wed Mar 22 18:43:05 PDT 2023 * foundation model samples - text classification - emotion detection Wed Mar 22 21:56:19 PDT 2023 * foundation model samples - text classification - emotion detection Wed Mar 22 22:02:51 PDT 2023 * foundation model samples - text classification - emotion detection Wed Mar 22 23:02:34 PDT 2023 * foundation model samples - text classification - emotion detection Wed Mar 22 23:10:16 PDT 2023 * foundation model samples - text classification - emotion detection Thu Mar 23 11:09:07 PDT 2023 * summarization sample * ft notebooks - fixes for eval and latest label * inference sample (#2159) * emotion detectio cli sample * cli sample emotion detection * cli sample emotion detection * cli sample emotion detection * cli sample emotion detection * fill mask inference cli sample * fill mask inference cli sample * Updating FT notebooks and CLI examples (#2160) * notebook samples added for translation and token classification * Adding cli examples --------- Co-authored-by: HrishikeshGeedMS <hgeed@microsoft.com> * mlflow for metrics * adding eval components * FT notebook updates - added dependencies installation, command to find number of GPU's in SKU and Metrics (#2162) * Adding requirements install, cmd to find gpu number and metrics to notebooks * Adding deployment supported SKU list to notebooks * Removing version dependency for azure-ai-ml * Updating gpus query to python code * renaming compute_model_selector -> compute_model_import * evaluation text classification sample * renaming compute_model_selector -> compute_model_import * evaluation text classification sample * evaluation text classification sample * evaluation text classification sample * correcting the evaluation config extension from jsonl -> json * updating the logic to calculate the gpu count * evaluation text classification sample * inference text classification sample * Adding github workflows for FT notebooks * model eval dashboard screenshot * Inference samples for all tasks (#2175) * Sample scripts for fill-mask and translation * Add generic inference scripts * remove task-specific folders * remove whisper files * Update sample inputs * use task specific scripts * Sample scripts for fill-mask and translation * Add generic inference scripts * remove task-specific folders * remove whisper files * Update sample inputs * use task specific scripts * Sample scripts for fill-mask and translation * Add generic inference scripts * remove task-specific folders * remove whisper files * Update sample inputs * use task specific scripts * Add generic inference scripts * remove task-specific folders * remove whisper files * Update sample inputs * use task specific scripts * inference samples for all tasks * Add sample script for text-classification * Remove generic samples * remove token-classification samples * delete extra dataset * Add token-classification-samples (#2176) * Model Evaluation sample notebooks * Model Evaluation sample notebooks - Removing cell outputs * Model Evaluation sample notebooks - Cleaning up notebooks * Model Evaluation sample notebooks - Adding Git actions * Model Evaluation sample notebooks - Renaming cluster name * Model Evaluation sample notebooks - Adding model evaluation dashboard screenshots * Model Evaluation sample notebooks - Fixing black workflow runs * add whisper files and update scripts (#2183) * Model Evaluation sample notebooks - Adding configs for text-gen and fill-mask tasks * Model Evaluation sample notebooks - Removing multilabel Github workflow * Model Evaluation sample notebooks - Modifying eval-configs for fill-mask and text-gen * Model Evaluation sample notebooks - Fixing missing info for fill-mask and text-gen notebooks * Add ground truth comparison, update sample scores (#2186) * Fixing github runners (#2188) * Try to read workspace details from config * Reformatted versions with black * Fixing text classification notebook issue * Adding mlflow installation * Update QnA metric * Handeling runs with None * Model Evaluation sample notebooks - Adding documentation for evaluation configs * Model Evaluation sample notebooks - Fixing black check run * added ymls for model-evaluation-subgraph cli (#2189) * added ymls for model-evaluation-subgraph cli * changed type to pipeline in cli files * updated model id * Removing input_column_names from FT cli * batch sample for text classification (#2199) * batch sample * batch sample * batch samples * batch samples for text-classification * batch samples for text-classification * Updating FT notebooks to use latest NCD input format * updating config for text and token classification. (#2204) Co-authored-by: Chandra Sekhar Gupta Aravpalli <caravapalli@microsoft.com> * Model Evaluation sample notebooks - Modifying to Azure production registry * Model Evaluation sample notebooks - Fixing black runs * Updating registry_name to azureml-preview-test1 temporarily * Model Evaluation sample notebooks - Fixing text-classification notebook * Model Evaluation sample notebooks - Fixing text-classification notebook * Model Evaluation sample notebooks - Changing registry for model fetching * Model Evaluation sample notebooks - Changing registry for model fetching * Model Evaluation sample notebooks - Fixing fill mask masks * Model Evaluation sample notebooks - Finalising notebooks * Model Evaluation sample notebooks - Fixing black runs * task name typo (#2207) * Setting up FT notebooks cron to run at midnight daily * Changing NCD compute * Changing registry to azureml * Aditisingh/update names (#2212) * renamed directories * renamed cli directories * updated data paths * Updating FT notebooks inference compute * updating notebooks for evaluation of base models (#2216) * updating evaluation text classification notebooks. * addding notebook for evaluation of sentiment analysis models. --------- Co-authored-by: Chandra Sekhar Gupta Aravpalli <caravapalli@microsoft.com> * Add batch fill mask example. (#2208) Add fill mask batch endpoint example notebook. * Add remaining batch examples for HF foundation models (#2217) adding batch endpoint example notebooks for all except summarization and asr, which did not pass. * Pmanoj/foundational models cli issues (#2223) * fixing cli script bugs * Changing inference sku --------- Co-authored-by: Pavan Manoj Jonnalagadda <pavanmanojj@microsoft.com> * Reformatting inference files * Model Evaluation sample notebooks - Fixing Github workflows --------- Co-authored-by: Manoj Bableshwar <mabables@microsoft.com> Co-authored-by: skanakamedal <116672436+skanakamedal@users.noreply.github.com> Co-authored-by: HrishikeshGeedMS <hgeed@microsoft.com> Co-authored-by: Narayanan Madhu <naraymad@microsoft.com> Co-authored-by: SitaRam Chaitanya Kanakamedala <skanakamedal@microsoft.com> Co-authored-by: Pavan Manoj Jonnalagadda <pavanmanojj@microsoft.com> Co-authored-by: Sumadhva Sridhar <109793745+susridhar@users.noreply.github.com> Co-authored-by: Sarthak Singhal <sarsinghal@microsoft.com> Co-authored-by: Aditi Singh <114134940+s-aditi@users.noreply.github.com> Co-authored-by: Chandra Sekhar Gupta <38103118+guptha23@users.noreply.github.com> Co-authored-by: Chandra Sekhar Gupta Aravpalli <caravapalli@microsoft.com> Co-authored-by: amymsft <94562419+amymsft@users.noreply.github.com> Co-authored-by: Haritha Pallavi Bendapudi <hbendapudi@microsoft.com>

* prototype notebookes for jsonl conversion: multiclass, multilabel, object detection coco, object detection voc, instance segmentation coco, instance segmentation voc * implement jsonl conversion for multiclass and multilabel classification, demonstrate in jsonl-conversion/notebooks and modify automl-image-classification-multiclass-task-fridge-items notebook to use the new implementation * change multilabel notebook to use new jsonl conversion * implement coco jsonl converter for object detection and change notebook to use new implementation, verified that new implementation produces the same jsonl file as the old implementation * implement voc jsonl converter for object detection and demonstrate in object detection notebook, verified that new implementation produces the same train and validation json files as the original * clear outputs in changed notebooks * add instance segmentation to voc jsonl converter, verify that it generates the same train and val annotation files as original, verify that it generates the same annotation file as the coco jsonl converter for object detection * implement coco to jsonl conversion for instance segmentation for iscrowd==0, tested with example notebook from Azure/medical-imaging which ignores crowd annotations, also verified that this does not break the coco to jsonl code for object detection * implement coco to jsonl for instance segmentation for iscrowd==1, generate coco data for instance segmentation notebook, add coco usage to instance segmentation notebook, verified that the jsonl files generated for both voc to jsonl and coco to jsonl (using the newly generated data) are equivalent * refactor mask to polygon using automl.dnn.vision helpers * handling for compressed and uncompressed rle in coco 2 jsonl converter * generate odFridgeObjects data in coco format using rle instead of polygons * demonstrate coco to jsonl for rle data * add docstrings to jsonl conversion code, test with notebooks again, clean up extraneous files * remove extraneous imports * add azureml-automl-dnn-vision pip install for voc to jsonl conversion * reformat with black * add od batch scoring notebook * respond to pr comments: remove unnecessary pip installs, revert notebook metadata, revert modified experiment names, remove az login calls * restore pip install for azureml-automl-dnn-vision, needed to pass gate * revert notebook metadata * copy masktools helpers from azureml-automl-dnn-vision directly into source code * remove unnecessary pip installs, reformat with black, restore metadata * include imports for pycocotools and simplification, necessary for jsonl converison * add skimage pip install * fix skimage -> scikit-image pip install * clarify markdown for pip install prompts Co-authored-by: sharma-riti <52715641+sharma-riti@users.noreply.github.com> --------- Co-authored-by: Rehaan Bhimani <rbhimani@microsoft.com> Co-authored-by: sharma-riti <52715641+sharma-riti@users.noreply.github.com>

Co-authored-by: Aishani Bhalla <aibhalla@microsoft.com>

* notebook review changes done * Update import_model_into_registry.ipynb * notebook review changes done * notebook review changes done * Update import_model_into_registry.ipynb * review changes * Update import_model_into_registry.ipynb changed kernel * review changes * review changes * review changes * review changes

* rename models for ncd scenario * fix lightgbm model name --------- Co-authored-by: Aishani Bhalla <aibhalla@microsoft.com>

* notebook updated for cela * notebook updated for cela * notebook updated for cela * notebook updated for cela tnc * notebook updated for cela * notebook updated for cela * notebook updated for cela * notebook updated for cela * notebook updated for cela * notebook updated for cela * notebook updated for cela * notebook updated for cela

Add speech recognition example using batch endpoints.

Add summarization example using batch endpoints. Includes a cropping step on the data, which will need to be removed when the model no longer needs this to be done by the user and crops inputs during preprocessing.

used local scoped variable within the cluster creation function.

Co-authored-by: Miseon Park <mipark@microsoft.com>

* Updating model versions * Rename ft pipeline names in notebooks * updating the model version and the registry * changing the job name --------- Co-authored-by: Pavan Manoj Jonnalagadda <pavanmanojj@microsoft.com>

Co-authored-by: jgussman <jgussman@iu.edu>

… nirovins/add_metrics_package

…dency (#2249) * update * update --------- Co-authored-by: Ying Chen <2601502859@qq.com>

…r facing sample (#2252) * Remove kubernetes-compute* workflows that don't appear to have a customer facing example * Remove .github/kubernetes-compute

… nirovins/add_metrics_package

* many models demand forecasting * updated README.md * updated azure workflows * changed compute to dedicated, removed references to TCN * resolved most of Nikolay's comments * changed VM type * fixed formatting * changed to a smaller compute * Changed credentials call in yml * Revert "Changed credentials call in yml" This reverts commit 0de650c. * Changed credentials call in yml

Co-authored-by: xiaoranli <xiaoranli@microsoft.com>

#2250) * remove news summary dataset from repo, download using script in sample * fix to cli sample to remove dataset from repo * typo fix * formatting

…um or snake_case (#2266) Co-authored-by: Rehaan Bhimani <rbhimani@microsoft.com>

… nirovins/add_metrics_package

Commits on Mar 24, 2023

Update README.md (#2147 )

fix typo

ccozianu committed Mar 24, 2023

Configuration menu

View commit details

Copy full SHA for 38a77d6

Browse repository at this point

Copy the full SHA

38a77d6 View commit details

Browse the repository at this point in the history

Use metrics package in the notebook. #2130

Use metrics package in the notebook. #2130

Commits on Nov 21, 2022

Commits on Nov 22, 2022

Commits on Nov 23, 2022

Commits on Nov 24, 2022

Commits on Nov 28, 2022

Commits on Nov 29, 2022

Commits on Nov 30, 2022

Commits on Dec 1, 2022

Commits on Dec 2, 2022

Commits on Dec 5, 2022

Commits on Dec 6, 2022

Commits on Dec 7, 2022

Commits on Dec 8, 2022

Commits on Dec 10, 2022

Commits on Dec 12, 2022

Commits on Dec 13, 2022

Commits on Dec 14, 2022

Commits on Dec 15, 2022

Commits on Dec 16, 2022

Commits on Dec 19, 2022

Commits on Dec 20, 2022

Commits on Dec 22, 2022

Commits on Dec 26, 2022

Commits on Dec 28, 2022

Commits on Dec 30, 2022

Commits on Dec 31, 2022

Commits on Jan 3, 2023

Commits on Jan 6, 2023

Commits on Jan 9, 2023

Commits on Jan 10, 2023

Commits on Jan 11, 2023

Commits on Jan 19, 2023

Commits on Jan 20, 2023

Commits on Jan 23, 2023

Commits on Jan 24, 2023

Commits on Jan 25, 2023

Commits on Jan 26, 2023

Commits on Jan 30, 2023

Commits on Jan 31, 2023

Commits on Feb 1, 2023

Commits on Feb 2, 2023

Commits on Feb 6, 2023

Commits on Feb 8, 2023

Commits on Feb 9, 2023

Commits on Feb 10, 2023

Commits on Feb 14, 2023

Commits on Feb 16, 2023

Commits on Feb 17, 2023

Commits on Feb 21, 2023

Commits on Feb 23, 2023

Commits on Feb 24, 2023

Commits on Feb 27, 2023

Commits on Mar 1, 2023

Commits on Mar 2, 2023

Commits on Mar 8, 2023

Commits on Mar 9, 2023

Commits on Mar 10, 2023

Commits on Mar 13, 2023

Commits on Mar 14, 2023

Commits on Mar 15, 2023

Commits on Mar 16, 2023

Commits on Mar 17, 2023

Commits on Mar 20, 2023

Commits on Mar 21, 2023

Commits on Mar 22, 2023

Commits on Mar 23, 2023

Commits on Mar 24, 2023

Commits on Mar 27, 2023

Commits on Mar 28, 2023

Commits on Mar 29, 2023

Commits on Mar 30, 2023

Commits on Mar 31, 2023

Commits on Apr 3, 2023

Commits on Apr 4, 2023

Commits on Apr 5, 2023

Commits on Apr 6, 2023

Commits on Apr 7, 2023

Commits on Apr 12, 2023