KRIA KV260 FPGA Deployment: Flip-Flop Usage Exceeds 100% with hls4ml and Brevitas

[ggdeniz]

Hello everyone,

I am new to hls4ml and am attempting to deploy a PyTorch-trained model onto a KRIA KV260 FPGA using Vivado 2024. However, I am facing challenges with the FPGA's resource constraints, particularly with flip-flop usage exceeding 100% (Vivado reports ~107%).

Context:

• Target Hardware: KRIA KV260 (AMD Xilinx FPGA).
• Framework: PyTorch for model training.
• Tools: Vivado 2024 for implementation, hls4ml for HLS conversion.
• Issue: The model exceeds the FPGA's flip-flop capacity, even after reducing its size. Further reduction may risks significant performance degradation.
• Model Type: CNN with 3 convolutional layers and 2 dense layers (~500K parameters).

Steps Taken So Far (in no particular order):

• Reduced the model size to fit within the FPGA's constraints.
• Attempted FIFO Depth Optimization to address the flip-flop overflow issue.
• Used Brevitas for quantization to optimize resource usage.

Questions:

1. Has anyone encountered similar resource constraints (especially flip-flop usage) when targeting the KRIA KV260?
2. Are there specific optimizations or configurations in hls4ml (e.g., layer partitioning, bit-width reduction, or resource-sharing techniques) that could help mitigate this issue?

Additional Notes:

I am open to alternative approaches, such as:

• Model pruning or quantization adjustments.
• Architectural modifications (e.g., layer fusion, depth-wise separable convolutions).
• Other workflow optimizations I might be missing.

Attached Scripts:

• hls.py: Conversion script.
• EUROSAT.py: Training script.

Thank you in advance for your guidance and insights!