Skip to content

Laravel Batch v3.3.0

Latest

Choose a tag to compare

@mavinoo mavinoo released this 05 Oct 08:42
· 2 commits to master since this release
384bcb9

Counter and conditional upserts, rows back from upserts, a report of skipped duplicates, fast
imports, row validation and exports. Everything is additive, so upgrading from 3.2 needs no code
changes:

composer update mavinoo/laravel-batch

Added

Counter upserts

Batch::upsert(new Stock, $rows, ['sku'], ['qty' => ['+']]);   // qty = stored qty + new qty
  • ['-'], ['max'] and ['min'] work the same way, on every database.

Conditional upserts

Batch::upsert(new Product, $rows, ['sku'], null, ['onlyIf' => ['updated_at' => '>']]);
  • Updates an existing row only when the new value wins. New rows are always inserted.

Rows back, and the duplicates that were skipped

$users  = Batch::upsertReturning(new User, $rows, ['email'], ['name'], ['id', 'email']);
$result = Batch::insertOrIgnoreRows(new User, $rows, ['email']);
// ['inserted' => 98, 'skipped' => [['email' => 'dup@example.com', ...], ...]]
  • insertOrIgnoreRows() was checked with 8 concurrent writers inserting overlapping keys on
    MySQL 8.4, MariaDB and PostgreSQL. Every row was reported by the connection that inserted it.

Fast imports

Batch::import(new Order, storage_path('orders.csv'), ['fast' => true]);
  • Uses COPY on PostgreSQL and LOAD DATA LOCAL INFILE on MySQL and MariaDB.
  • Bad values that MySQL would silently store with a warning throw instead, and nothing is written.
1,000,000-row CSV import fast
PostgreSQL 18.7 s 6.6 s
MySQL 8.4 18.7 s 8.4 s
MariaDB 12.6 s 6.4 s

Row validation in imports

Batch::import(new User, 'users.csv', ['rules' => ['email' => 'required|email']]);
// ['totalRows' => 9988, 'skipped' => 12, 'errors' => [15 => ['email' => [...]]]]

Export

User::where('active', true)->exportTo(storage_path('users.csv'), ['maxRows' => 100000]);
  • Writes CSV, TSV or JSON Lines files (also gzipped), a chunk at a time with constant memory.
  • maxRows splits the output into numbered parts.

Turn timestamps off

User::withoutTimestamps(fn () => Batch::update(new User, $rows)) now has documentation and tests
for every method (#97).

Changed

  • upsert() takes an optional fifth $options argument.
  • import() also returns an errors key.

Full changelog: https://github.com/mavinoo/laravelBatch/blob/master/CHANGELOG.md
Pull request: #128