New sgr cloud commands, fleshing out the splitgraph.yml (ex-repositories.yml), codegen for Splitgraph Cloud projects.

[mildbyte]

New sgr cloud commands

These let users manipulate Splitgraph Cloud and ingestion jobs from the CLI:

• sgr cloud status: view the status of ingestion jobs in the current project
• sgr cloud logs: view job logs
• sgr cloud csv: upload a CSV file to Splitgraph Cloud (without using the engine)
• sgr cloud sync: trigger a one-off load of a dataset
• sgr cloud stub: generate a splitgraph.yml file
• sgr cloud seed: generate a Splitgraph Cloud project with a splitgraph.yml, GitHub Actions, dbt etc
• sgr cloud validate: merge multiple project files and output the result (like docker-compose config)

splitgraph.yml

Default various commands that use repositories.yml to splitgraph.yml instead. Allow "mixing in" multiple .yml files Docker Compose-style (mostly useful for keeping credentials separate from

Wrote some documentation on the new format, GitHub Actions workflow reference-style (a header for every field with its full path in the YAML). It temporarily lives here while we can't easily deploy the docs site: https://github.com/splitgraph/splitgraph.com/blob/f7ac524cb5023091832e8bf51b277991c435f241/content/docs/0900_splitgraph-cloud/0500_splitgraph-yml.mdx

Sample project generation

sgr cloud seed generates a sample Splitgraph Cloud project from a base64-encoded "seed" (e.g. eyJuYW1lc3BhY2UiOiJtaWxkYnl0ZSIsInBsdWdpbnMiOlsicG9zdGdyZXNfZmR3Iiwic25vd2ZsYWtlIl0sImluY2x1ZGVfZGJ0Ijp0cnVlfQo=).

This is mostly for our marketing website which will let people "check out" with a Splitgraph Cloud project that contains their chosen data sources + a dbt transformation. Interested CLI users can still use it by encoding a JSON as base64:

{"namespace":"mildbyte","plugins":["postgres_fdw","snowflake"],"include_dbt":true}

and passing it to sgr cloud seed.

The intended usage is:

• https://github.com/splitgraph/splitgraph-cloud-template is a template that a user can clone
• Run the "seed" Action in their cloned repo to set up the initial project
• Next steps: set up credentials, dbt models etc.
• Example output https://github.com/mildbyte/template-test (see the README there for more information)

Miscellaneous

• Add lightweight SVG icons to builtin plugins (not used in the CLI but used in Splitgraph Cloud).
• Allow setting initial repo visibility in sgr cloud sync/sgr cloud load (pass --initial-private to create the repo as private if it doesn't yet exist)
• Use Unified GQL API instead of separate GQL endpoints
• Start using pytest-snapshot for tests that involve asserting long CLI/file outputs