Skip to content

Releases: sqlc-dev/sqlc

v1.26.0

28 Mar 20:48
25b942f
Compare
Choose a tag to compare

Release notes

This release is mainly a bug fix release. It also includes an important security fix for users using output plugins.

Changes

Bug Fixes

  • (docker) Use distroless base image instead of scratch (#3111)
  • (generate) Ensure files are created inside output directory (#3195)
  • (mysql) BREAKING: Use int16 for MySQL SMALLINT and YEAR (#3106)
  • (mysql) BREAKING: Use int8 for MySQL TINYINT (#3298)
  • (mysql) Variables not resolving in ORDER BY statements (#3115)
  • (opts) Validate SQL package and driver options (#3241)
  • (postgres/batch) Ignore query_parameter_limit for batches
  • (scripts) Remove deprecated test output regeneration script (#3105)
  • (sqlite) Correctly skip unknown statements (#3239)

Documentation

  • (postgres) Add instructions for PostGIS/GEOS (#3182)
  • Improve details on TEXT (#3247)

Features

  • (generate) Avoid generating empty Go imports (#3135)
  • (mysql) Add NEXTVAL() to the MySQL catalog (#3147)
  • (mysql) Support json.RawMessage for LOAD DATA INFILE (#3099)

Build

  • (deps) Bump github.com/jackc/pgx/v5 to 5.5.5 (#3259)
  • (deps) Bump modernc.org/sqlite to 1.29.5 (#3200)
  • (deps) Bump github.com/go-sql-driver/mysql to 1.8.0 (#3257)
  • (deps) Bump github.com/tetratelabs/wazero to 1.7.0 (#3096)
  • (deps) Bump github.com/pganalyze/pg_query_go to v5 (#3096)

v1.25.0

04 Jan 00:26
2b73120
Compare
Choose a tag to compare

What's new

Add tags to push and verify

You can add tags when pushing schema and queries to sqlc Cloud. Tags operate like git tags, meaning you can overwrite previously-pushed tag values. We suggest tagging pushes to associate them with something relevant from your environment, e.g. a git tag or branch name.

$ sqlc push --tag v1.0.0

Once you've created a tag, you can refer to it when verifying changes, allowing you
to compare the existing schema against a known set of previous queries.

$ sqlc verify --against v1.0.0

C-ya, cgo

Over the last month, we've switched out a few different modules to remove our reliance on cgo. Previously, we needed cgo for three separate functions:

With the help of the community, we found cgo-free alternatives for each module:

For the first time, Windows users can enjoy full PostgreSQL support without using WSL. It's a Christmas miracle!

If you run into any issues with the updated dependencies, please open an issue.

Bug Fixes

  • (codegen) Wrong yaml annotation in go codegen options for output_querier_file_name (#3006)
  • (codegen) Use derived ArrayDims instead of deprecated attndims (#3032)
  • (codegen) Take the maximum array dimensions (#3034)
  • (compiler) Skip analysis of queries without a name annotation (#3072)
  • (codegen/golang) Don't import "strings" for sqlc.slice() with pgx (#3073)

Documentation

  • Add name to query set configuration (#3011)
  • Add a sidebar link for push, add Go plugin link (#3023)
  • Update banner for sqlc-gen-typescript (#3036)
  • Add strict_order_by in doc (#3044)
  • Re-order the migration tools list (#3064)

Features

  • (analyzer) Return zero values when encountering unexpected ast nodes (#3069)
  • (codegen/go) add omit_sqlc_version to Go code generation (#3019)
  • (codgen/go) Add emit_sql_as_comment option to Go code plugin (#2735)
  • (plugins) Use wazero instead of wasmtime (#3042)
  • (push) Add tag support (#3074)
  • (sqlite) Support emit_pointers_for_null_types (#3026)

Testing

  • (endtoend) Enable for more build targets (#3041)
  • (endtoend) Run MySQL and PostgreSQL locally on the runner (#3095)
  • (typescript) Test against sqlc-gen-typescript (#3046)
  • Add tests for omit_sqlc_version (#3020)
  • Split schema and query for test (#3094)

Build

  • (deps) Bump idna from 3.4 to 3.6 in /docs (#3010)
  • (deps) Bump sphinx-rtd-theme from 1.3.0 to 2.0.0 in /docs (#3016)
  • (deps) Bump golang from 1.21.4 to 1.21.5 (#3043)
  • (deps) Bump actions/setup-go from 4 to 5 (#3047)
  • (deps) Bump github.com/jackc/pgx/v5 from 5.5.0 to 5.5.1 (#3050)
  • (deps) Upgrade to latest version of github.com/wasilibs/go-pgquery (#3052)
  • (deps) Bump google.golang.org/grpc from 1.59.0 to 1.60.0 (#3053)
  • (deps) Bump babel from 2.13.1 to 2.14.0 in /docs (#3055)
  • (deps) Bump actions/upload-artifact from 3 to 4 (#3061)
  • (deps) Bump modernc.org/sqlite from 1.27.0 to 1.28.0 (#3062)
  • (deps) Bump golang.org/x/crypto from 0.14.0 to 0.17.0 (#3068)
  • (deps) Bump google.golang.org/grpc from 1.60.0 to 1.60.1 (#3070)
  • (deps) Bump google.golang.org/protobuf from 1.31.0 to 1.32.0 (#3079)
  • (deps) Bump github.com/tetratelabs/wazero from 1.5.0 to 1.6.0 (#3096)
  • (sqlite) Update to antlr 4.13.1 (#3086)
  • (sqlite) Disable modernc for WASM (#3048)
  • (sqlite) Switch from mattn/go-sqlite3 to modernc.org/sqlite (#3040)

New Contributors

Full Changelog: v1.24.0...v1.25.0

v1.24.0

22 Nov 18:11
Compare
Choose a tag to compare

What's new

Verifying database schema changes

Schema updates and poorly-written queries often bring down production databases. That’s bad.

Out of the box, sqlc generate catches some of these issues. Running sqlc vet with the sqlc/db-prepare rule catches more subtle problems. But there is a large class of issues that sqlc can’t prevent by looking at current schema and queries alone.

For instance, when a schema change is proposed, existing queries and code running in production might fail when the schema change is applied. Enter sqlc verify, which analyzes existing queries against new schema changes and errors if there are any issues.

Let's look at an example. Assume you have these two tables in production.

CREATE TABLE users (
  id UUID PRIMARY KEY
);

CREATE TABLE user_actions (
  id UUID PRIMARY KEY,
  user_id UUID NOT NULL,
  action TEXT,
  created_at TIMESTAMP
);

Your application contains the following query to join user actions against the users table.

-- name: GetUserActions :many
SELECT * FROM users u
JOIN user_actions ua ON u.id = ua.user_id
ORDER BY created_at;

So far, so good. Then assume you propose this schema change:

ALTER TABLE users ADD COLUMN created_at TIMESTAMP;

Running sqlc generate fails with this change, returning a column reference "created_at" is ambiguous error. You update your query to fix the issue.

-- name: GetUserActions :many
SELECT * FROM users u
JOIN user_actions ua ON u.id = ua.user_id
ORDER BY u.created_at;

While that change fixes the issue, there's a production outage waiting to happen. When the schema change is applied, the existing GetUserActions query will begin to fail. The correct way to fix this is to deploy the updated query before applying the schema migration.

It ensures migrations are safe to deploy by sending your current schema and queries to sqlc cloud. There, we run the queries for your latest push against your new schema changes. This check catches backwards incompatible schema changes for existing queries.

Here sqlc verify alerts you to the fact that ORDER BY "created_at" is ambiguous.

$ sqlc verify
FAIL: app query.sql

=== Failed
=== FAIL: app query.sql GetUserActions
    ERROR: column reference "created_at" is ambiguous (SQLSTATE 42702)

By the way, this scenario isn't made up! It happened to us a few weeks ago. We've been happily testing early versions of verify for the last two weeks and haven't had any issues since.

This type of verification is only the start. If your application is deployed on-prem by your customers, verify could tell you if it's safe for your customers to rollback to an older version of your app, even after schema migrations have been run.

Rename upload command to push

We've renamed the upload sub-command to push. We changed the data sent along in a push request. Upload used to include the configuration file, migrations, queries, and all generated code. Push drops the generated code in favor of including the plugin.GenerateRequest, which is the protocol buffer message we pass to codegen plugins.

We also add annotations to each push. By default, we include these environment variables if they are present:

GITHUB_REPOSITORY
GITHUB_REF
GITHUB_REF_NAME
GITHUB_REF_TYPE
GITHUB_SHA

Like upload, push should be run when you tag a release of your application. We run it on every push to main, as we continuously deploy those commits.

MySQL support in createdb

The createdb command, added in the last release, now supports MySQL. If you have a cloud project configured, you can use sqlc createdb to spin up a new ephemeral database with your schema and print its connection string to standard output. This is useful for integrating with other tools. Read more in the managed databases documentation.

Plugin interface refactor

This release includes a refactored plugin interface to better support future functionality. Plugins now support different methods via a gRPC service interface, allowing plugins to support different functionality in a backwards-compatible way.

By using gRPC interfaces, we can even (theoretically) support remote plugins, but that's something for another day.

New Contributors

Full Changelog: v1.23.0...v1.24.0

v1.23.0

24 Oct 19:48
Compare
Choose a tag to compare

What's new

Database-backed query analysis

With a database connection configured, sqlc generate will gather metadata from that database to support its query analysis. Turning this on resolves a large number of issues in the backlog related to type inference and more complex queries. The easiest way to try it out is with managed databases.

The database-backed analyzer currently supports PostgreSQL, with MySQL and SQLite support planned in the future.

New createdb command

When you have a cloud project configured, you can use the new sqlc createdb command to spin up a new ephemeral database with your schema and print its connection string to standard output. This is useful for integrating with other tools. Read more in the managed databases documentation.

Support for pgvector

If you're using pgvector, say goodbye to custom overrides! sqlc now generates code using pgvector-go as long as you're using pgx. The pgvector extension is also available in managed databases.

Go build tags

With the new emit_build_tags configuration parameter you can set build tags for sqlc to add at the top of generated source files.

What's Changed

New Contributors

Read more

v1.22.0

26 Sep 17:58
0de01ab
Compare
Choose a tag to compare

What's new

Managed databases for sqlc vet

If you're using sqlc vet to write rules that require access to a running database, sqlc can now start and manage that database for you. PostgreSQL support is available today, with MySQL on the way.

When you turn on managed databases, sqlc will use your schema to create a template database that it can copy to make future runs of sqlc vet very performant.

This feature relies on configuration obtained via sqlc Cloud. If you're interested in early access you can sign up here or send us an email at hello@sqlc.dev.

Read more in the managed databases documentation.

What's changed

New Contributors

  • @XanderAtGoingDutch made their first contribution in #2727

Full Changelog: v1.21.0...v1.22.0

v1.21.0

06 Sep 18:51
63b5f90
Compare
Choose a tag to compare

What's new

MySQL engine improvements

sqlc previously didn't know how to parse a CALL statement when using the MySQL engine,
which meant it was impossible to use sqlc with stored procedures in MySQL databases.

Additionally, sqlc now supports IS [NOT] NULL in queries. And LIMIT and OFFSET clauses
now work with UNION.

SQLite engine improvements

GitHub user @orisano continues to bring bugfixes and
improvements to sqlc's SQLite engine. See the "Changes" section below for the
full list.

Plugin access to environment variables

If you're authoring a sqlc plugin, you can now configure
sqlc to pass your plugin the values of specific environment variables.

For example, if your plugin
needs the PATH environment variable, add PATH to the env list in the
plugins collection.

version: '2'
sql:
- schema: schema.sql
  queries: query.sql
  engine: postgresql
  codegen:
  - out: gen
    plugin: test
plugins:
- name: test
  env:
  - PATH
  wasm:
    url: https://github.com/sqlc-dev/sqlc-gen-test/releases/download/v0.1.0/sqlc-gen-test.wasm
    sha256: 138220eae508d4b65a5a8cea555edd155eb2290daf576b7a8b96949acfeb3790

A variable named SQLC_VERSION is always included in the plugin's
environment, set to the version of the sqlc executable invoking it.

What's Changed

  • feat(endtoend/testdata): added two sqlite CAST tests and rearranged postgres tests for same by @andrewmbenton in #2551
  • build(deps): Bump wasmtime from v8.0.0 to v11.0.0 by @kyleconroy in #2553
  • feat(docs): add a reference to type overriding in datatypes.md by @andrewmbenton in #2557
  • docs: Update Docker installation instructions by @kyleconroy in #2552
  • docs: Modify config file version of mysql tutorial by @ken2403 in #2555
  • myriad formatting changes by @verygoodsoftwarenotvirus in #2558
  • build(deps): bump golang from 1.20.6 to 1.20.7 by @dependabot in #2563
  • build(deps): bump chardet from 5.1.0 to 5.2.0 in /docs by @dependabot in #2562
  • fix(engine/sqlite): support quoted identifier by @orisano in #2556
  • feat(engine/sqlite): support COLLATE for sqlite WHERE clause by @orisano in #2554
  • fix(engine/sqlite): fix compile error by @orisano in #2564
  • fix(engine/sqlite): fixed detection of column alias without AS by @orisano in #2560
  • fix(ci): bump go version for govulncheck in ci.yml by @andrewmbenton in #2568
  • fix: remove references to deprecated --experimental flag by @andrewmbenton in #2567
  • fix(postgres): Fixed a problem with array dimensions disappearing when using "ALTER TABLE ADD COLUMN" by @orisano in #2572
  • fix: remove GitHub sponsor integration by @andrewmbenton in #2574
  • fix(docs): improve discussion of prepared statements support by @andrewmbenton in #2604
  • build(deps): bump github.com/pganalyze/pg_query_go/v4 from 4.2.1 to 4.2.3 by @dependabot in #2583
  • docs: Reword phrase about pgx and multidimensional arrays by @rollulus in #2619
  • Gitignore Vim swap files by @Jille in #2616
  • build(deps): bump golang from 1.20.7 to 1.21.0 by @dependabot in #2596
  • build(deps): bump github.com/jackc/pgx/v5 from 5.4.2 to 5.4.3 by @dependabot in #2582
  • build(deps): bump pygments from 2.15.1 to 2.16.1 in /docs by @dependabot in #2584
  • build(deps): bump sphinxcontrib-applehelp from 1.0.4 to 1.0.7 in /docs by @dependabot in #2620
  • build(deps): bump sphinxcontrib-qthelp from 1.0.3 to 1.0.6 in /docs by @dependabot in #2622
  • build(deps): bump github.com/google/cel-go from 0.17.1 to 0.17.6 by @dependabot in #2650
  • fix Go struct tag parsing by @josharian in #2606
  • build(deps): bump sphinxcontrib-serializinghtml from 1.1.5 to 1.1.9 in /docs by @dependabot in #2641
  • build: Upgrade from Go 1.20 to Go 1.21 by @kyleconroy in #2665
  • refactor(astutils): remove redundant nil check in Walk by @Juneezee in #2660
  • build(deps): bump sphinxcontrib-devhelp from 1.0.2 to 1.0.5 in /docs by @dependabot in #2621
  • build(deps): Bump github.com/bytecodealliance/wasmtime-go from v11.0.0 to v12.0.0 by @kyleconroy in #2666
  • fix(compiler): Fix to not scan children under ast.RangeSubselect when retrieving table listing by @orisano in #2573
  • feat(mysql): Add parser support for IS [NOT] NULL by @Jille in #2651
  • fix(engine/sqlite): support NOT IN by @orisano in #2587
  • fix(codegen/golang): Fixed detection of the used package by @orisano in #2597
  • fix(engine/dolphin): Fixed problem that LIMIT OFFSET cannot be used with UNION ALL by @orisano in #2613
  • fix(compiler): support identifiers with schema by @orisano in #2579
  • fix(compiler): Fix column expansion to work with quoted non-keyword identifiers by @orisano in #2576
  • feat(engine/dolphin): support CALL statement by @orisano in #2614
  • fix(codegen): Compare define type in codegen by @tychy in #2578
  • build(deps): bump sphinx-rtd-theme from 1.2.2 to 1.3.0 in /docs by @dependabot in #2670
  • build(deps): bump sphinxcontrib-htmlhelp from 2.0.1 to 2.0.4 in /docs by @dependabot in #2671
  • feat(codegen): Allow plugins to access environment variables by @kyleconroy in #2669
  • fix(engine/sqlite): Fix ast when using compound operator by @orisano in #2673
  • fix(engine/sqlite): Fix to handle join clauses correctly by @orisano in #2674
  • fix(codegen): use correct Go types for bit strings and cid/oid/tid/xid with pgx/v4 by @andrewmbenton in #2668
  • build(deps): bump github.com/google/cel-go from 0.17.6 to 0.18.0 by @dependabot in #2691
  • docs: Missing emit_pointers_for_null_types configuration option in version 2 (#2682) by @topazur in #2683
  • fix(endtoend): Ensure all SQL works against PostgreSQL by @kyleconroy in #2684
  • build(deps): bump actions/checkout from 3 to 4 by @dependabot in #2694
  • build(deps): bump pytz from 2023.3 to 2023.3.post1 in /docs by @dependabot in #2695
  • chore: fix typo by @remyleone in #2696
  • chore: fix typo by @remyleone in #2697
  • docs: Document sqlc.* macros by @kyleconroy in #2698
  • docs(mysql): Document parseTime=true requirement by @kyleconroy in #2699
  • docs: add atlas to the list of supported migration frameworks by @andrewmbenton in #2700
  • docs: minor updates to insert howto by @andrewmbenton in #2701
  • build(devenv): bump go from 1.20.7 to 1.21.0 by @kyleconroy in #2702
  • feat(config): Add JSON schema files for configs by @kyleconroy in #2703
  • feat(cmd/sqlc): Bump version to 1.21.0 by @andrewmbenton in #2704
  • cmd/sqlc: Bump version to v1.21.0 by @kyleconroy in #2705

New Contributors

Full Changelog: v1.20.0...v1.21.0

v1.20.0

31 Jul 19:52
98ef71d
Compare
Choose a tag to compare

What's Changed

New Contributors

Full Changelog: v1.19.1...v1.20.0

v1.19.1

13 Jul 18:45
d9422dc
Compare
Choose a tag to compare

What's Changed

New Contributors

Full Changelog: v1.19.0...v1.19.1

v1.19.0

06 Jul 18:12
b643642
Compare
Choose a tag to compare

What's new

sqlc vet

sqlc vet runs queries through a set of lint rules.

Rules are defined in the sqlc configuration file. They consist of a name, message, and a Common Expression Language (CEL) expression. Expressions are evaluated using cel-go. If an expression evaluates to true, an error is reported using the given message.

While these examples are simplistic, they give you a flavor of the types of rules you can write.

version: 2
sql:
  - schema: "query.sql"
    queries: "query.sql"
    engine: "postgresql"
    gen:
      go:
        package: "authors"
        out: "db"
    rules:
      - no-pg
      - no-delete
      - only-one-param
      - no-exec
rules:
  - name: no-pg
    message: "invalid engine: postgresql"
    rule: |
      config.engine == "postgresql"
  - name: no-delete
    message: "don't use delete statements"
    rule: |
      query.sql.contains("DELETE")
  - name: only-one-param
    message: "too many parameters"
    rule: |
      query.params.size() > 1
  - name: no-exec
    message: "don't use exec"
    rule: |
      query.cmd == "exec"

Database connectivity

vet also marks the first time that sqlc can connect to a live, running database server. We'll expand this functionality over time, but for now it powers the sqlc/db-prepare built-in rule.

When a database is configured, the sqlc/db-prepare rule will attempt to prepare each of your queries against the connected database and report any failures.

version: 2
sql:
  - schema: "query.sql"
    queries: "query.sql"
    engine: "postgresql"
    gen:
      go:
        package: "authors"
        out: "db"
    database:
      uri: "postgresql://postgres:password@localhost:5432/postgres"
    rules:
      - sqlc/db-prepare

To see this in action, check out the authors example.

Please note that sqlc does not manage or migrate your database. Use your migration tool of choice to create the necessary database tables and objects before running sqlc vet.

Omit unused structs

Added a new configuration parameter omit_unused_structs which, when set to true, filters out table and enum structs that aren't used in queries for a given package.

Suggested CI/CD setup

With the addition of sqlc diff and sqlc vet, we encourage users to run sqlc in your CI/CD pipelines. See our suggested CI/CD setup for more information.

Simplified plugin development

The sqlc-gen-kotlin and sqlc-gen-python plugins have been updated use the upcoming WASI support in Go 1.21. Building these plugins no longer requires TinyGo.

What's Changed

Read more

v1.18.0

27 Apr 17:51
e4b1c18
Compare
Choose a tag to compare

What's New

Remote code generation

Developed by @andrewmbenton

At its core, sqlc is powered by SQL engines, which include parsers, formatters,
analyzers and more. While our goal is to support each engine on each operating
system, it's not always possible. For example, the PostgreSQL engine does not
work on Windows.

To bridge that gap, we're announcing remote code generation, currently in
private alpha. To join the private alpha, sign up for the waitlist.

To configure remote generation, configure a cloud block in sqlc.json.

{
  "version": "2",
  "cloud": {
    "organization": "<org-id>",
    "project": "<project-id>",
  },
  ...
}

You'll also need to the SQLC_AUTH_TOKEN environment variable.

export SQLC_AUTH_TOKEN=<token>

When the cloud configuration exists, sqlc generate will default to remote
generation. If you'd like to generate code locally, pass the --no-remote
option.

sqlc generate --no-remote

Remote generation is off by default and requires an opt-in to use.

sqlc.embed

Developed by @nickjackson

Embedding allows you to reuse existing model structs in more queries, resulting
in less manual serilization work. First, imagine we have the following schema
with students and test scores.

CREATE TABLE students (
  id   bigserial PRIMARY KEY,
  name text,
  age  integer
)

CREATE TABLE test_scores (
  student_id bigint,
  score integer,
  grade text
)

We want to select the student record and the highest score they got on a test.
Here's how we'd usually do that:

-- name: HighScore :many
WITH high_scores AS (
  SELECT student_id, max(score) as high_score
  FROM test_scores
  GROUP BY 1
)
SELECT students.*, high_score::integer
FROM students
JOIN high_scores ON high_scores.student_id = students.id;

When using Go, sqlc will produce a struct like this:

type HighScoreRow struct {
	ID        int64
	Name      sql.NullString
	Age       sql.NullInt32
	HighScore int32
}

With embedding, the struct will contain a model for the table instead of a
flattened list of columns.

-- name: HighScoreEmbed :many
WITH high_scores AS (
  SELECT student_id, max(score) as high_score
  FROM test_scores
  GROUP BY 1
)
SELECT sqlc.embed(students), high_score::integer
FROM students
JOIN high_scores ON high_scores.student_id = students.id;
type HighScoreRow struct {
	Student   Student
	HighScore int32
}

sqlc.slice

Developed by Paul Cameron and Jille Timmermans

The MySQL Go driver does not support passing slices to the IN operator. The
sqlc.slice function generates a dynamic query at runtime with the correct
number of parameters.

/* name: SelectStudents :many */
SELECT * FROM students 
WHERE age IN (sqlc.slice("ages"))
func (q *Queries) SelectStudents(ctx context.Context, arges []int32) ([]Student, error) {

This feature is only supported in MySQL and cannot be used with prepared
queries.

Batch operation improvements

When using batches with pgx, the error returned when a batch is closed is
exported by the generated package. This change allows for cleaner error
handling using errors.Is.

errors.Is(err, generated_package.ErrBatchAlreadyClosed)

Previously, you would have had to check match on the error message itself.

err.Error() == "batch already closed"

The generated code for batch operations always lived in batch.go. This file
name can now be configured via the output_batch_file_name configuration
option.

Configurable query parameter limits for Go

By default, sqlc will limit Go functions to a single parameter. If a query
includes more than one parameter, the generated method will use an argument
struct instead of positional arguments. This behavior can now be changed via
the query_parameter_limit configuration option. If set to 0, every
generated method will use a argument struct.

What's Changed

Read more