feat(pyamber): report batch element failures

[carloea2]

What changes were proposed in this PR?

Route per-element batch failures through the normal exception reporter, then check the recorded exception after the batch finishes iterating. This preserves the existing attempt-all-rows behavior while preventing a workflow from silently succeeding with missing rows.

Any related issues, documentation, discussions?

Closes #8233

How was this PR tested?

python -c "import sys,pytest; sys.path[:0]=[worktree amber source, main amber source]; raise SystemExit(pytest.main(['amber/src/test/python/core/runnables/test_main_loop.py::TestMainLoop::test_a_failing_element_does_not_abandon_the_rest_of_the_batch','amber/src/test/python/core/runnables/test_data_processor.py','amber/src/test/python/core/architecture/managers/test_exception_manager.py','-q','-p','no:cacheprovider']))"
12 passed

ruff check amber/src/main/python amber/src/test/python
All checks passed

ruff format --check amber/src/main/python amber/src/test/python
213 files already formatted

The regression processes two failing rows, confirms both were attempted, and confirms both failures were retained in exception history before the worker enters exception handling.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Codex

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @aglinxinyuan
  You can notify them by mentioning @aglinxinyuan in a comment.

[github-actions]

Backport auto-label report

This fix: PR was checked against each actively-supported release branch. release/* labels drive the post-merge backport, so add or remove one to change where this fix lands.

Release branch	Analysis
⚠️ release/v1.3	Not labeled automatically — none of the files this PR modifies exist on this branch (amber/src/main/python/core/runnables/main_loop.py, amber/src/test/python/core/runnables/test_main_loop.py). The fix may target code that isn't on this release, or the files were moved/renamed after the branch was cut. Please check and add release/v1.3 by hand if this fix should be backported here.
✅ release/v1.2	Already labeled — this fix is queued to backport here.

Auto-label run.

[github-actions]

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 10 worse · ⚪ 5 noise (<±5%) · 0 without baseline

Compared against main 70c2114 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	643	0.392	14,940/21,041/21,041 us	🔴 -16.6% / 🔴 +35.1%
🔴	bs=100 sw=10 sl=64	1,644	1.004	58,785/99,099/99,099 us	🔴 +6.3% / 🟢 +63.2%
🔴	bs=1000 sw=10 sl=64	2,053	1.253	481,276/553,920/553,920 us	🔴 +6.0% / 🟢 +97.0%

Baseline details

Latest main 70c2114 from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	643 tuples/sec	771 tuples/sec	787.98 tuples/sec	-16.6%	-18.4%
bs=10 sw=10 sl=64	MB/s	0.392 MB/s	0.47 MB/s	0.481 MB/s	-16.6%	-18.5%
bs=10 sw=10 sl=64	p50	14,940 us	13,812 us	12,593 us	+8.2%	+18.6%
bs=10 sw=10 sl=64	p95	21,041 us	19,351 us	15,579 us	+8.7%	+35.1%
bs=10 sw=10 sl=64	p99	21,041 us	19,351 us	18,786 us	+8.7%	+12.0%
bs=100 sw=10 sl=64	throughput	1,644 tuples/sec	1,727 tuples/sec	1,008 tuples/sec	-4.8%	+63.1%
bs=100 sw=10 sl=64	MB/s	1.004 MB/s	1.054 MB/s	0.615 MB/s	-4.7%	+63.2%
bs=100 sw=10 sl=64	p50	58,785 us	55,916 us	100,701 us	+5.1%	-41.6%
bs=100 sw=10 sl=64	p95	99,099 us	93,267 us	107,244 us	+6.3%	-7.6%
bs=100 sw=10 sl=64	p99	99,099 us	93,267 us	116,122 us	+6.3%	-14.7%
bs=1000 sw=10 sl=64	throughput	2,053 tuples/sec	2,080 tuples/sec	1,042 tuples/sec	-1.3%	+97.0%
bs=1000 sw=10 sl=64	MB/s	1.253 MB/s	1.27 MB/s	0.636 MB/s	-1.3%	+97.0%
bs=1000 sw=10 sl=64	p50	481,276 us	477,022 us	981,959 us	+0.9%	-51.0%
bs=1000 sw=10 sl=64	p95	553,920 us	522,602 us	1,023,080 us	+6.0%	-45.9%
bs=1000 sw=10 sl=64	p99	553,920 us	522,602 us	1,051,697 us	+6.0%	-47.3%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,311.05,200,128000,643,0.392,14939.56,21040.91,21040.91
1,100,10,64,20,1216.42,2000,1280000,1644,1.004,58785.06,99098.55,99098.55
2,1000,10,64,20,9740.65,20000,12800000,2053,1.253,481276.43,553920.29,553920.29

[Yicong-Huang]

I am converting this PR to a draft: please follow our PR template and mark it ready for review.

[carloea2]

The description now follows the current template, and this is ready for review.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.94%. Comparing base (70c2114) to head (6bdf4a7).
⚠️ Report is 12 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #8234   +/-   ##
=========================================
  Coverage     93.94%   93.94%           
  Complexity     4759     4759           
=========================================
  Files          1185     1185           
  Lines         48069    48070    +1     
  Branches       5359     5359           
=========================================
+ Hits          45159    45160    +1     
  Misses         1480     1480           
  Partials       1430     1430           

Flag	Coverage Δ		*Carryforward flag
access-control-service	81.00% <ø> (ø)		Carriedforward from 70c2114
agent-service	99.32% <ø> (ø)		Carriedforward from 70c2114
amber	89.99% <ø> (ø)		Carriedforward from 70c2114
computing-unit-managing-service	73.67% <ø> (ø)		Carriedforward from 70c2114
config-service	86.86% <ø> (ø)		Carriedforward from 70c2114
file-service	87.91% <ø> (ø)		Carriedforward from 70c2114
frontend	96.49% <ø> (ø)		Carriedforward from 70c2114
notebook-migration-service	79.31% <ø> (ø)		Carriedforward from 70c2114
pyamber	98.87% <100.00%> (+<0.01%)	⬆️
workflow-compiling-service	77.19% <ø> (ø)		Carriedforward from 70c2114

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.