Skip to content

Commit 607b868

Browse files
committed
faster 10 %
1 parent a2c6969 commit 607b868

File tree

8 files changed

+1036
-35
lines changed

8 files changed

+1036
-35
lines changed

Makefile

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,12 @@ test:
99
lint:
1010
golangci-lint run
1111

12-
# Run benchmarks
12+
# Run quick benchmarks (small and medium only)
1313
bench:
14+
go test -bench='BenchmarkProcessCSV_(Small|Medium)' -benchmem -run=^$$ ./...
15+
16+
# Run all benchmarks including large datasets
17+
bench-all:
1418
go test -bench=. -benchmem -run=^$$ ./...
1519

1620
# Run benchmarks with memory profiling

README.md

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -393,7 +393,30 @@ fileprep loads the **entire file into memory** for processing. This enables rand
393393

394394
For compressed inputs (gzip, bzip2, xz, zstd), memory usage is based on **decompressed** size.
395395

396-
**Benchmark target available**: Run `make bench` to measure processing performance on your system.
396+
## Performance
397+
398+
Benchmark results processing CSV files with a complex struct containing 21 columns. Each field uses multiple preprocessing and validation tags:
399+
400+
**Preprocessing tags used:** trim, lowercase, uppercase, keep_digits, pad_left, strip_html, strip_newline, collapse_space, truncate, fix_scheme, default
401+
402+
**Validation tags used:** required, alpha, numeric, email, uuid, ip_addr, url, oneof, min, max, len, printascii, ascii, eqfield
403+
404+
| Records | Time | Memory | Allocs/op |
405+
|--------:|-----:|-------:|----------:|
406+
| 100 | 0.6 ms | 0.9 MB | 7,654 |
407+
| 1,000 | 6.1 ms | 9.6 MB | 74,829 |
408+
| 10,000 | 69 ms | 101 MB | 746,266 |
409+
| 50,000 | 344 ms | 498 MB | 3,690,281 |
410+
411+
```bash
412+
# Quick benchmark (100 and 1,000 records)
413+
make bench
414+
415+
# Full benchmark (all sizes including 50,000 records)
416+
make bench-all
417+
```
418+
419+
*Tested on AMD Ryzen AI MAX+ 395, Go 1.24, Linux. Results vary by hardware.*
397420

398421
## Related or inspired Projects
399422

0 commit comments

Comments
 (0)