Restore default hw_decoder_load to 0.65 in imgcodec decoder#6366
Conversation
The nvImageCodec-based image decoder (dali/operators/imgcodec) sets the default `hw_decoder_load` to 0.90, whereas the legacy decoder (dali/operators/decoder) used 0.65. Profiling on EfficientNet+AutoAugment ImageNet training showed that the higher default increases p99 decoder latency (24.5 ms -> 34.3 ms) and the worst-case batch decode (30.8 ms -> 72.7 ms), driven by synchronous `cuMemFree_v2` calls inside nvjpeg that scale with HW-engine pressure. Aggregate throughput is unchanged on that workload. Revert the default to 0.65 to match the legacy behavior and restore decoder determinism. Users who want the prior 0.90 behavior can opt in explicitly via the operator argument. Signed-off-by: Joaquin Anton Guirao <janton@nvidia.com>
There was a problem hiding this comment.
Pull request overview
Restores the nvImageCodec-based image decoder’s default hardware JPEG routing (hw_decoder_load) to match the historical/legacy behavior, reducing unintended behavior changes for users relying on defaults.
Changes:
- Change default
hw_decoder_loadfrom0.90fto0.65fin the imgcodec decoder schema.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| the DALI pipeline and should be found empirically. More details can be found at | ||
| https://developer.nvidia.com/blog/loading-data-fast-with-dali-and-new-jpeg-decoder-in-a100)code", | ||
| 0.90f) | ||
| 0.65f) |
|
| Filename | Overview |
|---|---|
| dali/operators/imgcodec/decoder_schema.cc | Single-line change restoring hw_decoder_load default from 0.90f to 0.65f, matching the legacy decoder; copyright year already includes 2026. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Image Batch] --> B{hw_decoder_load routing}
B -->|"≤ 0.65 fraction (default)"| C[HW JPEG Decoder\nNVIDIA A100 engine]
B -->|"> 0.65 fraction"| D[Hybrid CPU+GPU Decoder\nnvJPEG software path]
C --> E[Decoded Output]
D --> E
style B fill:#f0f4ff,stroke:#3366cc
style C fill:#d4edda,stroke:#28a745
style D fill:#fff3cd,stroke:#ffc107
Reviews (1): Last reviewed commit: "Restore default hw_decoder_load to 0.65 ..." | Re-trigger Greptile
|
CI MESSAGE: [52351570]: BUILD STARTED |
|
CI MESSAGE: [52351570]: BUILD PASSED |
Summary
hw_decoder_loadargument in the nvImageCodec-based image decoder (dali/operators/imgcodec/decoder_schema.cc) from0.90fback to0.65f, matching the legacy decoder's default.Motivation
The legacy decoder (
dali/operators/decoder/image_decoder.cc) defaultshw_decoder_loadto 0.65. The new nvImageCodec-based decoder ships with 0.90, which causes a meaningful behavior change for users who relied on the default value.Nsys profiling of EfficientNet + AutoAugment ImageNet training (single-GPU, batch 256) on the two defaults shows:
Users who want the higher HW-routing fraction can still opt in explicitly via the operator argument.
Test plan
internal_tools/hw_decoder_bench.pyshows no regression at default settings