Permalink
Switch branches/tags
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
61 lines (42 sloc) 2.8 KB

Training AI models in PAI

PAI(Platform for AI) is a cluster management tool and resource scheduling platform, jointly designed and developed by Microsoft Research (MSR) and Microsoft Search Technology Center (STC). The platform incorporates some mature design that has a proven track record in large scale Microsoft production environment, and is tailored primarily for academic and research purpose.

Add a PAI cluster

To add a PAI cluster, right click PAI - Clusters node and select Add Configuration.

Add a PAI cluster

Command AI: Add Platform Configuration can also be used. Execute this command in Command Palette (Ctrl+Shift+P) and then select a config provider:

Add a PAI cluster

A configuration file is created and opened in editor window for review.

{
    "username": "username",
    "password": "password",
    "rest_server_uri": "10.0.3.9:9186",
    "webhdfs_uri": "10.0.3.9:50070",
    "grafana_uri": "10.0.3.9:3000",
    "k8s_dashboard_uri": "10.0.3.9:9090"
}

Click Finish to add a configuration.

Add a PAI cluster

Submit training jobs to a PAI cluster

For detailed guide about general submitting steps, please refer to Submitting Jobs

Right click the PAI cluster node in AI EXPLORER and select AI: Submit Job

Job submission to a PAI cluster

A QuickPick window pops up. Please choose whether to upload script automatically or not:

Job submission to a PAI cluster

[!NOTE] For PAI clusters, users can store their scripts and data in cloud storage and specify the path in job properties. In this case, please select No.

A new ai-job-properties.json file is then created and opened in editor window for review.

Job submission to a PAI cluster

In ai-job-properties.json, PAI-specific job properties are under platform.PAI.jobConfig:

    "platform": {
        "PAI": {
            "jobConfig": {
                "jobName": "iris_sklearn_43f7f1a7",
                "image": "aiplatform/pai.build.base",

For detailed information about PAI job properties, please refer to the PAI Job Tutorial

Click Finish button to submit the job.

Check the job status and download its assets

Now the job has been submitted to target PAI cluster. You can check the job status and download its assets via Job View. Meanwhile Storage Explorer is also a good place to access the job assets.