Repository navigation
First release.
Features
BatchFlow\Csv::scanandCsv::scanStream: lazy, immutableQueryobjects over a CSV file or PHP stream, with a
typed schema (int64,float64,bool,string), RFC 4180 dialect rules, an optionaldelimiter(,,;, tab
or|) and a leading UTF-8 BOM.- Filters
whereEq,whereNe,whereGt,whereGte,whereLt,whereLteandwhereIn; terminalscount,sum,
min,max,avg,aggregate,summary,groupCount,groupSum,groupAgg(one to four keys, up to 16
aggregates),toArrayandcursor()(Cursor::fetch,Cursor::close). BatchFlow\Table: a preloaded typed column store with optional dictionary columns, the same query methods as a
streamingQuery, byte-identical results, androwCount,columnNames,schemaandmemoryBytes.Runtime::info()andphp --ri batchflowreport the version and the compiled and selected SIMD function.- Exceptions with exact messages that carry row, line, offset and column but never record bytes:
SchemaException,
CsvException,LimitException,ResultTooLargeException,SumOverflowException,StreamException, all
extendingBatchFlowException. - Bounded memory: every growth is preflighted against
memory_limit, so a limit is aLimitException, not a PHP
fatal (best-effort on a fragmented heap, seedocs/THREAT_MODEL.md). LimitsmaxRowBytes,maxFieldBytes,
maxInputRows,maxGroups,maxGroupBytes,maxTableBytesandmaxDictionarySize. - Group keys are hashed with SipHash-1-3 under a per-run random key, so hostile keys cannot aim collisions.
- Structural scan with AVX2 (selected at run time on x86-64), NEON (arm64) and a scalar fallback, all
byte-identical. - Platforms: PHP 8.3, 8.4 and 8.5 NTS on Linux x86-64 and arm64; PHP 8.6 (pre-release) NTS, non-blocking
until PHP 8.6.0 is released. macOS arm64 builds and runs but has no CI lane. ZTS is not supported. - PIE metadata in
composer.json(php-ext, NTS only, Linux and macOS). The source package
(git archive) leaves outtests/,fuzz/,benchmarks/and.github/.
Performance
Measured on one x86-64 machine with AVX2 (see benchmarks/RESULTS.md): group-by 3.7x to 13.6x the fastest JIT'd PHP loop and 2.6x to
5.5x single-threaded DuckDB; streaming count and sum 5.2x to 15.4x the PHP loop at 0.84 times its peak memory; toArray 2.8x to
7.5x the fastest PHP loader. Rows with many unreferenced trailing columns are framed by popcount, and group-by on tables of
16,384 or more slots probes in batches of up to 16 rows with the slots prefetched. No arm64 speed claim.
Hardening
- CI (
.github/workflows/ci.yml, actions pinned by commit SHA): PHPT on PHP 8.3, 8.4 and 8.5 on x86-64 and arm64;
PHP 8.6 from source, non-blocking; ASan, UBSan and Zend-allocator PHPT runs with fuzz corpus replay on both
architectures; a provisional ZTS lane that does not fail the workflow. - A libFuzzer harness over the native core (all compiled mask functions, the reference parser, chunked reads,
group table, column store), with a committed seed and regression corpus. docs/THREAT_MODEL.md,docs/DESIGN.md.
Known limitations
See docs/THREAT_MODEL.md. In short: the memory preflight is best-effort for small blocks on a fragmented heap,
ZTS is not supported, and a stream's position after an exception is unspecified.