Skip to content

v0.9.0

Choose a tag to compare

@coenbakker coenbakker released this 07 Aug 17:54
· 35 commits to main since this release

Added

  • Blink.RowError, raised when a row's keys differ from the first row's. The message names the table, the offending row's index, and the missing and extra keys.
  • Telemetry spans for the whole pipeline: [:blink, :build, :start/:stop/:exception] around each table or context builder (builders run at declaration time, so their cost — usually the bulk of a seed's wall time — was invisible to the copy events), [:blink, :run, :start/:stop/:exception] around Blink.Seeder.run/3, and a [:blink, :copy, :stop] event with a :row_count measurement completing the existing :start. See Blink.Telemetry for the full event reference.
  • Blink.Telemetry.attach_default_logger/1 (and detach_default_logger/0), which logs run start, stop, and failure at a configurable level and per-declaration build and per-table copy timings at :debug — replacing the hand-rolled :timer.tc + Logger.info wrappers around seed scripts.
  • A :reset_sequences option (default: false) on Blink.Seeder.run/3, Blink.copy_to_table/4, and Blink.Adapter.Postgres, also settable per-table. After a table's copy it advances the sequence behind each serial or identity primary key column past the highest copied value, so the application's next ordinary insert no longer collides with a seeded row. Primary keys without a sequence (uuid, self-managed integers) are skipped. This replaces the setval query the Getting Started guide told you to run by hand. Intended for seed-time use: on a table receiving concurrent inserts, the reset can move the sequence backwards past values already handed out to in-flight transactions.
  • with_table/2,3 (defined by use Blink) accepts a list of table names, declaring each in order as if by one call per name, with the options applying to every table in the list: new() |> with_table(["users", "posts", "comments"]).
  • Blink.fetch_row!/3 (imported by use Blink), which fetches the first row of a declared table matching the given field values and raises a descriptive ArgumentError on a miss — replacing hand-rolled Enum.find/2 lookups whose nil result surfaces later as a crash far from the cause. Atom and string table names are interchangeable, matching the rest of the seeder API. Use it with list-backed tables only; the lookup enumerates the table, so it would consume a single-use stream.

Changed

  • Added an explicit telemetry ~> 1.1 requirement. Blink has emitted telemetry events for several releases but depended on telemetry only transitively through Ecto, leaving the real requirement unstated; the new spans rely on :telemetry.span/3 and the test suite on :telemetry_test (telemetry 1.1+), so it is now declared.
  • Breaking: Every row must now have the same keys as the first row, or the copy raises Blink.RowError. The column list is read from the keys of the first row, so previously a key missing from a later row was silently inserted as NULL (or failed the COPY on a NOT NULL column without naming the culprit) and an extra key was silently dropped — the documented "all maps must have the same keys" contract was never enforced. Validation runs as rows are consumed: with atomic: true (the default) a failed seed leaves nothing behind; with atomic: false a mismatch surfaces like any other mid-copy failure, with earlier batches possibly committed. Rows from sparse sources (optional JSON fields and the like) that relied on missing keys becoming NULL must now be normalized first, e.g. Map.merge(defaults, row) — which also frees the result from depending on which row happened to come first.

Documentation

  • Added the Building Rows guide: build plain maps, not schema structs. It explains the three Repo.insert habits that do not carry over to COPY (column selection, autogeneration, nil handling), shows a canonical seeder shape (fixed rows in module attributes, a call/0 entrypoint, string table names, one timestamp for the whole seed), and covers passing calendar structs directly, leaving columns to their database defaults, entrypoint naming (run/2,3 on your module overrides Blink's — the documented mechanism, whether you meant it or not), scoping the dependency per mix env, CI timeouts, testing seeders under Ecto.Adapters.SQL.Sandbox (atomic seeds enroll in the sandbox, so async: true works), and trigger behavior during COPY.
  • Rewrote the ExMachina guide around the two real pairings: map factories dedicated to seeding (the simple path — the maps are the rows), and struct factories shared with the test suite, which should stay structs and be converted at the seeder boundary. The guide now gives the conversion recipe — to_row/2 (Map.from_struct/1 + Map.take(__schema__(:fields)) with an explicit-vs-database id policy) and drop_all_nil_columns/1 for schemas that lean on database defaults — instead of pretending the struct case does not exist.
  • Added the Bulk Imports Outside Seeding guide. Blink's copy path is a general bulk-insert primitive, but every existing doc assumed an idle, disposable database; this guide covers what changes on a live one — the atomicity trade-off reread for production tables (a fully atomic import is one long transaction; atomic: false batch commits need a re-runnable design), the staging-table pattern for upserts (COPY has no ON CONFLICT; use a real or UNLOGGED staging table, since a TEMPORARY one is invisible to the parallel copy connections), normalizing rows from sparse external sources, why reset_sequences must not run against concurrent inserts, pool sizing so an import cannot starve the application, and observing imports through the copy telemetry events.