We need way more infos on MLTable

[danielsc]

Our documentation is lacking deeper information on the MLTable format.
I cannot seem to find any reference docs, nor a JSON schema, nor any samples that go beyond referencing a single csv or jsonl file.

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: cf5a19a0-c37e-4567-0c30-7efe1c433dfb
• Version Independent ID: 8c4e79d0-0f84-3a28-bfb4-9f972e02a37f
• Content: az ml data
• Content Source: latest/docs-ref-autogen/ml/data.yml
• GitHub Login: @rloutlaw
• Microsoft Alias: routlaw

[yonzhan]

route to CXP team

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @azureml-github.

Issue Details

---

Our documentation is lacking deeper information on the MLTable format.
I cannot seem to find any reference docs, nor a JSON schema, nor any samples that go beyond referencing a single csv or jsonl file.

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: cf5a19a0-c37e-4567-0c30-7efe1c433dfb
• Version Independent ID: 8c4e79d0-0f84-3a28-bfb4-9f972e02a37f
• Content: az ml data
• Content Source: latest/docs-ref-autogen/ml/data.yml
• GitHub Login: @rloutlaw
• Microsoft Alias: routlaw

Author:	danielsc
Assignees:	-
Labels:	Service Attention, Machine Learning, Auto-Assign
Milestone:	-

[Man-MSFT]

Hey @danielsc , we have documentation for mltable here: https://docs.microsoft.com/en-us/azure/machine-learning/how-to-create-register-data-assets?tabs=CLI#create-a-mltable-data-asset, or were you looking for something on GitHub explicitly?

I think we can add a link to the JSON schema. I also could not find it in docs, for reference, the latest MLTable JSON schema would be here: https://azuremlschemasprod.azureedge.net/latest/MLTable.schema.json

[danielsc]

Yes, I believe we need way more than that, for instance:

• make sure the schema is available and used in all samples
• provide a reference the YAML so folks can read up on each one of them
• show where to find and how to use the mltable data plane sdk. Where does that library live? Who owns it? Where is it documented?

[Man-MSFT]

Those are all good points and I agree. For points 1+2, we will take an action item to update current docs to add the $schema line for examples to include the schema and point to schema very obviously.

For the third point, the library and pip link is referenced in the how to create doc, but the pip page is lacking. @QianqianNie can you take ownership on this? https://pypi.org/project/mltable
Few things I'm seeing that can be improved: Description, sample usage, project link back to the msdoc.

[QianqianNie]

@SturgeonMi is working on adding sample notebook

[Man-MSFT]

@QianqianNie , @SturgeonMi , gentle reminder on the pip package information too! Thanks!

[maoding]

More information would be a great deal. I created a dataset in Azure ML via the GUI but now I struggle to do so using CLI az ml data create since I do not have enough information how to write the MLTable file residing next to the parquet files.
From the serveral steps in the GUI I can more or less assume what is needed but I do not find any reference how to write the MLTable file properly.

E.g. tabular data from blobstorage, include columns, change type and specify date format.

[eavanvalkenburg]

Documentation is indeed lacking a lot! I have been able to create a mltable deltalake set, but reading that into a dataframe based on the data asset is causing all sorts of issues with the error messages that do not help and the docs lacking examples!