Update docs to show parquet files need to be serialised with version 1.0 to be visible in the Azure ML studio

[adriantorrie]

[Enter feedback here]
When saving parquet files as mltables, with one of the following ways, the file data is not viewable

# Assume we are going to save a pandas dataframe once split into train/test sets
 
# 1 doesn't work
df.to_parquet(Path(output_dir, "data.parquet"))

# nor 2
df.to_parquet(Path(output_dir, "data.parquet"), engine='pyarrow')

# nor 3
pyarrow_table = pa.Table.from_pandas(df)
pq.write_table(pyarrow_table, Path(output_dir, "data.parquet"))

# and finally save the MLTable file
table = mltable.from_parquet_files(paths=[{"pattern": "./*parquet"}])
table.save(Path(output_dir), overwrite=True)

Error:

However, if I explicitly use the Parquet version:

# Note the `version` flag within the `pq.write_table` function
pyarrow_table = pa.Table.from_pandas(df)
pq.write_table(pyarrow_table, Path(output_dir, "data.parquet"), version="1.0")

# and finally save the MLTable file
table = mltable.from_parquet_files(paths=[{"pattern": "./*parquet"}])
table.save(Path(output_dir), overwrite=True)

We can view the data:

---

Document Details

⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

• ID: 916be6d5-bb68-5606-d124-66d56878c9c0
• Version Independent ID: c03b4af4-1a7e-00bd-46c3-658c86b8d95a
• Content: Working with tables in Azure Machine Learning - Azure Machine Learning
• Content Source: articles/machine-learning/how-to-mltable.md
• Service: machine-learning
• Sub-service: mldata
• GitHub Login: @samuel100
• Microsoft Alias: samkemp

[ManoharLakkoju-MSFT]

@adriantorrie
Thanks for your feedback! We will investigate and update as appropriate.

[RamanathanChinnappan-MSFT]

@adriantorrie

I've delegated this to @samuel100, a content author, to review and share their valuable insights.

[adriantorrie]

Any updates?