Skip to content

Releases: splitgraph/sgr

v0.2.10

17 Mar 18:18
Compare
Choose a tag to compare
  • Fix CSV schema inference not supporting BIGINT data types (#407)
  • Fix Splitfiles only expecting tags to contain alphanumeric characters (#407)
  • Speedups for the Snowflake / SQLAlchemy data soure (#405)

v0.2.9

12 Mar 18:17
Compare
Choose a tag to compare

Add a Snowflake data source, backed by a SQLAlchemy connector (#404)

v0.2.8

09 Mar 20:52
Compare
Choose a tag to compare

Allow deleting tags on remote registries (#403)

v0.2.7

09 Mar 14:26
Compare
Choose a tag to compare

Fix MySQL plugin crashes on binary data types.

v0.2.6

04 Mar 16:52
Compare
Choose a tag to compare
  • Fix querying when there are NULLs in primary keys (#373)
  • Data source and foreign data wrapper for querying CSV files in S3 buckets and HTTP (#397)
  • Ctrl+C can now interrupt long-running PostgreSQL queries and stop sgr (#398)
  • Support for updating miscellaneous repository metadata from the sgr cloud metadata CLI (#399)

v0.2.5

25 Jan 11:57
Compare
Choose a tag to compare
  • Fix piping CSV files from stdin (#350)
  • Truncate commit comments if they're above the max field size (currently 4096) (#353)
  • Add support for updating repository topics from the CLI (sgr cloud metadata) (#371)

v0.2.4

08 Dec 10:49
Compare
Choose a tag to compare
  • Mount handlers are now called "data sources", a generalization that will make them more pluggable
    and support sources beyond FDWs. See #324 for more documentation and necessary steps to migrate.
  • Added sgr singer target, a Singer-compatible target that can read Singer tap output from stdin and build Splitgraph images. It's based on a fork of https://github.com/transferwise/pipelinewise-singer-python with additions that let us produce deltas and ingest them directly as Splitgraph objects.
  • Support for dynamically loading plugins without specifying them in .sgconfig, by looking up plugins in a certain directory (see #329)

v0.2.3

08 Dec 10:48
Compare
Choose a tag to compare
Bump version: 0.2.2 → 0.2.3

  * Socrata now correctly emits `IS NULL / IS NOT NULL`, same with ES (using ES query syntax).
  * Fix array handling (`a IN (1,2,3)` queries get rewritten and pushed down correctly).
  * Output more query information in `EXPLAIN` for Socrata/LQ.

v0.2.2

16 Sep 09:01
Compare
Choose a tag to compare

Add ability to pass extra server args to postgres_fdw:

sgr mount postgres_fdw schema -c [connstr] -o@- <<EOF
{
  "extra_server_args": {
    "use_remote_estimate": "true",
	"extensions": "postgis",
	"fetch_size": "10000"
  }
}
EOF

Add ability to rename object files in-engine (utility function for some ingestion).

Allow disabling IMPORT FOREIGN SCHEMA and passing a table schema in Postgres/MySQL FDWs. For example:

sgr mount mysql_fdw schema -c [connstr] -o@- <<EOF
{
  "remote_schema": "mysql_schema",
  "tables": {
    "table_1": {
      "t1_col_1": "integer",
      "t1_col_2": "text"
    },
    "table_2": {
      "t2_col_1": "bigint",
      "t2_col_2": "integer"
    }
  }
}
EOF

Add a fork (https://github.com/splitgraph/postgres-elasticsearch-fdw) of https://github.com/matthewfranglen/postgres-elasticsearch-fdw to sgr mount, letting others mount ES indexes:

sgr mount elasticsearch -c elasticsearch:9200 -o@- <<EOF
{
  "table_spec": {
    "table_1": {
      "schema": {
        "id": "text",
        "@timestamp": "timestamp",
        "query": "text",
        "col_1": "text",
        "col_2": "boolean",
      }
      "index": "index-pattern*",
      "rowid_column": "id",
      "query_column": "query",
    }
  }
}
EOF

Differences:

  • Pass qualifiers as ElasticSearch queries using the query DSL (was using the query=... qual as a Lucene query string, which is useless in JOINs. Now we combine both the query implied from the quals and the Lucene query string, if passed)
  • Close the search context on end_scan (otherwise many ES queries to the FDW in a 10 minute span would cause it to error with a "too many scroll contexts" exception)
  • Add EXPLAIN support (outputs the used ES query)

v0.2.1

02 Sep 09:00
Compare
Choose a tag to compare
  • Add ability to skip config injection at the end of config-manipulating functions (pass -s) and don't fail if the Docker socket isn't reachable