v0.3.1

Fix sgr cloud sync invocation issue (#589)

v0.3.0

Fleshing out the splitgraph.yml (aka repositories.yml) format that defines a Splitgraph Cloud "project" (datasets, their sources and metadata).

Existing users of repositories.yml don't need to change anything, though note that sgr cloud commands using the YAML format will now default to splitgraph.yml unless explicitly set to repositories.yml.

New sgr cloud commands

See #582 and #587

These let users manipulate Splitgraph Cloud and ingestion jobs from the CLI:

• sgr cloud status: view the status of ingestion jobs in the current project
• sgr cloud logs: view job logs
• sgr cloud upload: upload a CSV file to Splitgraph Cloud (without using the engine)
• sgr cloud sync: trigger a one-off load of a dataset
• sgr cloud stub: generate a splitgraph.yml file
• sgr cloud seed: generate a Splitgraph Cloud project with a splitgraph.yml, GitHub Actions, dbt etc
• sgr cloud validate: merge multiple project files and output the result (like docker-compose config)
• sgr cloud download: download a query result from Splitgraph Cloud as a CSV file, bypassing time/query size limits.

splitgraph.yml

Change various commands that use repositories.yml to default to splitgraph.yml instead. Allow "mixing in" multiple .yml files Docker Compose-style, useful for splitting credentials (and not checking them in) and data settings.

Temporary location for the new full documentation on splitgraph.yml: https://github.com/splitgraph/splitgraph.com/blob/f7ac524cb5023091832e8bf51b277991c435f241/content/docs/0900_splitgraph-cloud/0500_splitgraph-yml.mdx

Miscellaneous

• Initial backend support for "transforming" Splitgraph plugins, including dbt (#574)
• Dump scheduled ingestion/transformation jobs with sgr cloud dump (#577)

Full set of changes: v0.2.18...v0.3.0

v0.2.18

• Splitfile speedups (#567)
• Various query speedups (#563, #561)
• More robust CSV querying (#562)

Full set of changes: v0.2.17...v0.2.18

v0.2.17

• Code refactor / optimizations (#531)
• Support for pluggable authorization logic (#542, #549)
• FDW JSONSchema fixes (#545)
• Upgrade pglast to 3.4 to fix issues with Splitfile validation (#534)
• Speed up Splitfile builds (#550)

Full set of changes: v0.2.16...v0.2.17

v0.2.16

v0.2.16 (2021-08-18)

• Various Airbyte ingestion improvements and support for different normalization modes, including a custom dbt model (#510, #513, #514)
• Fix mount for data source with empty credentials schema (#515)
• Fix sgr cloud load/dump (#520)

Full set of changes: v0.2.15...v0.2.16

v0.2.15

• API functionality to get the raw URL for a data source (#457)
• LQ scan / filtering simplification to speed up writes / Singer loads (#464, #489)
• API functionality for Airbyte support (AirbyteDataSource class, #493)
• Speed up sgr cloud load by bulk API calls (#500)

Full set of changes: v0.2.14...v0.2.15

v0.2.14

  * Functionality to dump and load a Splitgraph catalog to/from a spe…

…cial `repositories.yml` format (https://github.com/splitgraph/splitgraph/pull/445)

v0.2.13

  * Various fixes to CSV inference and querying (https://github.com/s…

…plitgraph/splitgraph/pull/433)

  * Add customizable fetch size to the Snowflake data source (https://github.com/splitgraph/splitgraph/pull/434)
  * Fix issue with changing the engine password (https://github.com/splitgraph/splitgraph/pull/437)
  * Data source refactor (https://github.com/splitgraph/splitgraph/pull/438):
    * MySQL: parameter `remote_schema` has been renamed to `dbname`
    * Mongo: parameter `coll` has been renamed to `collection`; `db` to `database`
    * Table options are now a separate parameter that is passed to the
    * Introspection now returns a dictionary of tables and proposed table options OR error classes for tables that we weren't able to introspect (allowing for partial failures)
    * Mounting can now return a list of mount errors (caller can choose to ignore).
    * CSV data source: allow passing a partially initialized list of table options without a schema, making it introspect just those S3 keys and fill out the missing table options.
  * Postgres-level notices are now available in the `PsycopgEngine.notices` list after a `run_sql` invocation.
  * Multicorn: fix bug where server-level FDW options would override table-level FDW options.

Full set of changes: [`v0.2.12...v0.2.13`](https://github.com/splitgraph/splitgraph/compare/v0.2.12...v0.2.13)

v0.2.12

Bump version: 0.2.11 → 0.2.12

  * Fixes to the Snowflake data source (https://github.com/splitgraph/splitgraph/pull/421)
  * Add automatic encoding, newline and dialect inference to the CSV data source (https://github.com/splitgraph/splitgraph/pull/432)

v0.2.11

• Snowflake data source improvements:
  • Allow passing envvars to set HTTP proxy parameters, fix incorrect query string generation when passing a warehouse (#414, #413)
  • Support for authentication using a private key (#418)
• Splitfiles: relax AST restrictions to support all SELECT/INSERT/UPDATE/DELETE statements (#411)
• Change the default installation port to 6432 and handle port conflicts during install (#375)
• Add retry logic to fix registry closing the SSL connection after 30 seconds, close remote connections in some places (#417)