Publish PUDL Intake Catalog #1179
Labels
cloud
Stuff that has to do with adapting PUDL to work in cloud computing context.
epic
Any issue whose primary purpose is to organize other issues into a group.
intake
Issues related to intake data catalogs.
metadata
Anything having to do with the content, formatting, or storage of metadata. Mostly datapackages.
output
Exporting data from PUDL into other platforms or interchange formats.
packaging
Software packaging and distribution of PUDL via pypi, etc.
Description
Once we have automated our data builds (#1177) and are able to output and test new data products with minimal human intervention, we want to enable the automatic distribution of those data products to users via Intake catalogs.
Motivation
Benefits of Using Intake:
We could just upload our SQLite DBs and Parquet files to some cloud buckets, and point people at those resources. Using Intake potentially provides some additional benefits:
Tasks / Issues
v0.2.0
release includingepacems
,ferc1
,pudl
andcensusdp1tract
datasets.epacems
catalog metadataferc1
catalog metadatapudl
(SQLite) catalog metadatacensusdp1tract
catalog metadatav0.3.0
release with the new, complete metadata.pudl-catalog
branches and tags synchronized with the nightly build outputs from the mainpudl
repository.pudl-catalog
package intstallable from PyPI andconda-forge
pudl-catalog
(andpudl
?) to using calendar versioning when nightly builds + intake catalogs are working. E.g. a new Bastille Day release might be taggedv2022.07.14
Out of Scope
Billing
Hours should be billed to Data Distribution under the Sloan grant.
The text was updated successfully, but these errors were encountered: