Skip to content

[WIP] Add build-flow benchmark tests for FINN example models - #1650

Draft
auphelia wants to merge 34 commits into
devfrom
feature/add_finnexamples
Draft

[WIP] Add build-flow benchmark tests for FINN example models#1650
auphelia wants to merge 34 commits into
devfrom
feature/add_finnexamples

Conversation

@auphelia

@auphelia auphelia commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Adds a tests/benchmark/ suite that runs the end-to-end FINN build flow for the public FINN example models across their target boards, checking that all expected build products and per-step verification outputs are produced. This gives us regression coverage for the example designs that today only live in finn-examples.

Also includes one small correctness fix in StreamingDataflowPartition execution so the partition output is cast to the ONNX-declared output datatype during SDP execution.

What's included

  • New benchmark tests under tests/benchmark// for:
    • bnn-pynq (CNV/TFC, w1a1/w1a2/w2a2)
    • cybersecurity-mlp
    • gtsrb
    • kws
    • mobilenet_v1
    • resnet50
    • vgg10-radioml
  • Each model ships per-board folding_config/ and specialize_layers_config/ JSONs. Boards covered: Pynq-Z1, AUP-ZU3_8GB, ZCU102, ZCU104, U250.
  • Each test runs build_dataflow_cfg (generating estimate reports, stitched IP, PYNQ driver, bitfile, deployment package, rtlsim performance) and asserts the expected outputs plus the verify_*_SUCCESS.npy verification artifacts
    (initial/streamlined python, folded HLS cppsim, node-by-node rtlsim, stitched-IP rtlsim).
  • tests/benchmark/models/download_models.sh pulls the ONNX models from the finn-examples GitHub releases.
  • New finn_examples pytest marker (setup.cfg); .gitignore entries for downloaded models and verification .npy files.

Bug fix

  • streamingdataflowpartition.py: propagate the correct output datatype during SDP execution by casting ret[model_oname] to the graph output's declared dtype (previously the raw array dtype leaked through).

Test plan

  • pytest -m finn_examples with models downloaded via download_models.sh
  • Note Vivado version constraints baked into tests (2022.2 default; 2024.1 required for AUP-ZU3_8GB)

auphelia and others added 30 commits October 31, 2025 17:12
Add finn examples to finn for Pynq-Z1 and AUP-ZU3
@auphelia

Copy link
Copy Markdown
Collaborator Author

Currently quicktest is failing, this will pass when #1637 is merged into this branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants