Technique example: data loader, Python to parquet #1422
Conversation
|
can we link to https://arrow.apache.org/docs/python/generated/pyarrow.parquet.write_table.html explicitly mentioning that there are many options (and recommend compression); and maybe show compression in action? |
Co-authored-by: Philippe Rivière <fil@rezo.net>
|
@Fil I added the compression codec explicitly in the loader (compression="snappy"), and include a sentence pointing to the write_table docs and different compression algorithms. Look okay?
|
There was a problem hiding this comment.
Mind copying the new virtual environment pattern from #1468?
<div class="note">
To run this data loader, you’ll need python3 and the geopandas, matplotlib, io, and sys modules installed and available on your `$PATH`.
</div>
<div class="tip">
We recommend using a [Python virtual environment](https://observablehq.com/framework/loaders#venv), such as with venv or uv, and managing required packages via `requirements.txt` rather than installing them globally.
</div>
|
Quick question - would dbt work here? |
| Plot.barX(dams, | ||
| Plot.groupY({x: "count"}, {y: "Primary Purpose", fill: "Hazard Potential Classification", sort: {y: "x", reverse: true} | ||
| }) | ||
| ) |
There was a problem hiding this comment.
Please prettier this. 🙏
Also, you can use sort: {y: "-x"} to shorten.
Deployed: https://observablehq.observablehq.cloud/framework-example-loader-python-to-parquet/