2.3.6

2.3.6

Important

Starting with onnx2tf v2.4.0, tf_converter will be deprecated and the default backend will be switched to flatbuffer_direct. With the v2.3.3 update, all backward compatible conversion options have been migrated to flatbuffer_direct, so I will only be doing minor bug fixes until April. If you provide us with ONNX sample models, I will consider incorporating them into flatbuffer_direct. I'll incorporate ai-edge-quantizer when I feel like it, but that will probably be about 10 years from now.

Summary

This PR advances the feat-torch2 line by making flatbuffer_direct-generated native PyTorch packages much more useful in Torch 2 workflows and by tightening native export/runtime behavior on models that previously fell back, misreported shapes, or failed secondary artifact generation.

From a functional perspective, the branch now goes beyond generating a PyTorch package and TorchScript artifact. It can also emit Torch 2-oriented artifacts directly from the generated native package, while keeping the flatbuffer_direct path aligned with real-model parity and validation needs.

Functional improvements

• Adds --flatbuffer_direct_output_dynamo_onnx (-fdodo) so a generated native PyTorch package can emit <model>_dynamo.onnx via torch.onnx.export(..., dynamo=True).
• Adds --flatbuffer_direct_output_exported_program (-fdoep) so the same package can emit <model>_ep.pt2 via torch.export.save.
• Exposes the same functionality in the Python API through flatbuffer_direct_output_dynamo_onnx and flatbuffer_direct_output_exported_program.
• Automatically enables flatbuffer_direct_output_pytorch when any secondary PyTorch artifact is requested, keeping the UX consistent with existing TorchScript behavior.
• Extends split export flows so split packages can also produce Dynamo ONNX and ExportedProgram artifacts, with counts and output paths surfaced in the returned metadata.

Native package and export robustness

• Adds dedicated generated-package export helpers for TorchScript, Dynamo ONNX, and ExportedProgram generation with subprocess isolation and structured metadata updates.
• Records success and failure details back into each generated package metadata.json, including example input shapes, dynamic-input state, and export errors.
• Reuses the existing example-input pipeline for Torch 2 exports, including support for shape_hints, test_data_nhwc_path, and per-input custom example data.
• Fixes a real blocker for Torch 2 export on native packages by materializing constant reduction axes directly in generated code instead of forcing tensor-to-Python integer extraction during export.
• Improves native codegen/runtime handling around layout-sensitive operations so generated packages stay exportable on more realistic models instead of only trivial cases.

flatbuffer_direct coverage and parity improvements

• Improves native/export handling for models using layout-sensitive softmax patterns, concat alignment edge cases, and additional channel-first/channel-last conversions.
• Refines 3D convolution and transposed-3D-convolution weight rewriting for native PyTorch codegen.
• Fixes TopK output shape/signature handling in the flatbuffer_direct lowering path, including cases where k is constant, stale, or runtime-derived.
• Expands runtime and evaluation support used by the generated packages so ONNX/TFLite/PyTorch comparisons remain viable on a broader set of outputs.
• Keeps user-facing reporting aligned by surfacing success/failure messages for direct and split Dynamo ONNX / ExportedProgram outputs from the CLI conversion flow.

Documentation and packaging

• Documents the new CLI and Python API options in README.md, including dynamic-input guidance and how shape_hints, -cind, and --test_data_nhwc_path apply to the new export modes.
• Bumps the package version to 2.3.6.
• Adds onnxscript==0.6.2, which is required by the Dynamo ONNX export path.

Test coverage

• Expands exporter tests for:
  • generated-package Dynamo ONNX output
  • generated-package ExportedProgram output
  • split PyTorch auxiliary artifacts
  • constant-axis reduction exportability
  • native package parity on more realistic model shapes and operators
• Expands direct-builder coverage around TopK, parity validation, and related flatbuffer_direct behavior.

Validation

• Ran pytest -q tests -k flatbuffer_direct
  • Result: 651 passed, 317 deselected, 3 warnings
• Verified the previously failing real-model workflow for pidnet_S_cityscapes_192x320.onnx now emits both:
  • pidnet_S_cityscapes_192x320_dynamo.onnx
  • pidnet_S_cityscapes_192x320_ep.pt2
    instead of only writing warning entries to the generated package metadata.

Why this matters

Before this branch, the flatbuffer_direct Torch path was useful primarily for package generation and, in some cases, TorchScript export. After this change, the generated native package becomes a practical bridge into broader Torch 2 ecosystems: users can generate Dynamo ONNX and ExportedProgram artifacts directly from the same conversion output, while the lowering/runtime fixes reduce the gap between synthetic smoke tests and real deployment models.

What's Changed

• feat(flatbuffer_direct): expand Torch 2 export artifacts and native coverage by @PINTO0309 in #906

Full Changelog: 2.3.5...2.3.6