Skip to content

NXP backend: added support for aten.conv_transpose1 and refactored convolution_converter#19004

Merged
MartinPavella merged 1 commit into
pytorch:mainfrom
nxp-upstream:feature/EIEX-681-add-transposed-conv-1d-support
May 11, 2026
Merged

NXP backend: added support for aten.conv_transpose1 and refactored convolution_converter#19004
MartinPavella merged 1 commit into
pytorch:mainfrom
nxp-upstream:feature/EIEX-681-add-transposed-conv-1d-support

Conversation

@novak-vaclav
Copy link
Copy Markdown
Contributor

@novak-vaclav novak-vaclav commented Apr 20, 2026

Summary

Added support for aten.conv_transpose1d by moving functionality from convolution_converter to brand new convert_1d_conv_to2d aten pass, and extending it.

Test plan

tests can be manually run using pytest -c /dev/null backends/nxp/tests/

cc @robert-kalmar @JakeStevens @digantdesai @MartinPavella

Copilot AI review requested due to automatic review settings April 20, 2026 16:34
@pytorch-bot
Copy link
Copy Markdown

pytorch-bot Bot commented Apr 20, 2026

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19004

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit 2b79e1a with merge base a49171d (image):

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Apr 20, 2026
@novak-vaclav
Copy link
Copy Markdown
Contributor Author

@pytorchbot label "release notes: nxp"

@pytorch-bot pytorch-bot Bot added the release notes: nxp Changes to the NXP Neutron backend delegate label Apr 20, 2026
@novak-vaclav
Copy link
Copy Markdown
Contributor Author

@pytorchbot label "module: nxp"

@pytorch-bot pytorch-bot Bot added the module: nxp Issues related to NXP Neutron NPU delegation and code under backends/nxp/ label Apr 20, 2026
@novak-vaclav novak-vaclav marked this pull request as draft April 20, 2026 16:37
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds NXP backend support for aten.conv_transpose1d by moving 1D-to-2D convolution lowering out of the IR converter and into a dedicated ATen graph rewrite pass, plus adjusts quantization handling for grouped transposed convolutions.

Changes:

  • Introduce ConvertConv1dToConv2dPass to rewrite aten.conv1d / aten.conv_transpose1d into 2D equivalents via unsqueeze/conv2d(or conv_transpose2d)/squeeze.
  • Remove 1D-convolution handling from the TFLite convolution_converter and enable the new pass in the default Neutron ATen pass pipeline.
  • Update quantizer patterns/utilities to correctly derive bias qparams for grouped conv_transpose2d and fix per-channel axis handling; add comprehensive tests for the new pass.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
backends/nxp/aten_passes/convert_1d_conv_to_2d.py New ATen pass converting 1D conv/transposed conv to 2D form with shape/meta propagation.
backends/nxp/aten_passes/neutron_aten_pass_manager.py Registers the new pass in the default Neutron ATen pass sequence.
backends/nxp/backend/ir/converter/node_converters/ops_converters/convolution_converter.py Removes 1D convolution conversion logic; now expects only 2D weights rank.
backends/nxp/quantizer/utils.py Adds helper to “pad”/repeat weight scales when deriving bias qparams for grouped transposed conv.
backends/nxp/quantizer/patterns.py Drops 1D conv patterns; updates ConvTranspose2d quantization (bias qparams + correct per-channel axis).
backends/nxp/quantizer/neutron_quantizer.py Removes the Conv1dPattern registration (since 1D conv is rewritten earlier).
backends/nxp/tests/test_convert_1d_conv_to_2d.py New test suite covering conv1d + conv_transpose1d rewrite and full pipeline delegation.
backends/nxp/tests/models.py Updates Conv1d test module API and adds ConvTranspose1d + runtime-weight conv1d models for testing.
backends/nxp/tests/ir/converter/node_converter/test_conv_converter.py Removes prior conv1d conversion tests (superseded by new pass tests).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread backends/nxp/tests/models.py
Comment thread backends/nxp/tests/models.py Outdated
Comment thread backends/nxp/aten_passes/convert_1d_conv_to_2d.py
Comment thread backends/nxp/aten_passes/convert_1d_conv_to_2d.py Outdated
@MartinPavella MartinPavella self-requested a review April 21, 2026 06:50
@novak-vaclav novak-vaclav force-pushed the feature/EIEX-681-add-transposed-conv-1d-support branch from 8568135 to 73e97a6 Compare April 29, 2026 07:58
@novak-vaclav novak-vaclav marked this pull request as ready for review April 29, 2026 07:59
Copilot AI review requested due to automatic review settings April 29, 2026 07:59
@novak-vaclav
Copy link
Copy Markdown
Contributor Author

Refactored the whole solution. Solved cases where there was batch_norm after conv1d, since insertion of squeeze prevented batch_norm fusion with the conv.
I also thought that converting conv1d to conv2d with bmm behind it would break the bmm, but it should not happen.

@novak-vaclav
Copy link
Copy Markdown
Contributor Author

Please review @MartinPavella and @StrycekSimon. Thank you 😊

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread backends/nxp/tests/models.py Outdated
Comment thread backends/nxp/tests/models.py Outdated
Comment thread backends/nxp/tests/generic_tests/test_quantizer.py
Comment thread backends/nxp/aten_passes/convert_1d_conv_to_2d.py Outdated
Comment thread backends/nxp/aten_passes/convert_1d_conv_to_2d.py Outdated
Copy link
Copy Markdown
Collaborator

@MartinPavella MartinPavella left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work!
I only have cosmetic suggestions and questions. Perhaps for future PRs you could split the implementation into multiple commits. Ultimately it would still get squashed to 1, but it would make the review easier as the commit messages would explain the motivation for some changes. Here I'm really not sure why some changes were made.

Also Copilot had some good comments, so please address them too if you see fit.

Comment thread backends/nxp/aten_passes/convert_1d_conv_to_2d.py Outdated
Comment thread backends/nxp/aten_passes/convert_1d_conv_to_2d.py Outdated
Comment thread backends/nxp/aten_passes/convert_1d_conv_to_2d.py
Comment thread backends/nxp/aten_passes/convert_1d_conv_to_2d.py
Comment thread backends/nxp/aten_passes/convert_1d_conv_to_2d.py Outdated
Comment thread backends/nxp/quantizer/utils.py Outdated
Comment thread backends/nxp/tests/generic_tests/test_batch_norm_fusion.py
Comment thread backends/nxp/tests/test_convert_1d_conv_to_2d.py
Copy link
Copy Markdown
Collaborator

@StrycekSimon StrycekSimon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, handle the 2D inputs or at least mark them as unsupported and create an issue.

Comment thread backends/nxp/aten_passes/convert_1d_conv_to_2d.py Outdated
Comment thread backends/nxp/tests/test_convert_1d_conv_to_2d.py
Comment thread backends/nxp/aten_passes/convert_1d_conv_to_2d.py
Comment thread backends/nxp/tests/test_convert_1d_conv_to_2d.py
@novak-vaclav novak-vaclav force-pushed the feature/EIEX-681-add-transposed-conv-1d-support branch from 73e97a6 to 1578982 Compare May 5, 2026 00:10
@novak-vaclav
Copy link
Copy Markdown
Contributor Author

Enabled conv_transpose1d test case in test_quantizer

Fixed some of the issues and provided comments on some. Please review it @StrycekSimon @MartinPavella. Thank you! 😊

@MartinPavella
Copy link
Copy Markdown
Collaborator

MartinPavella commented May 5, 2026

LGTM. Please resolve the conflicts so the tests can run.

Copilot AI review requested due to automatic review settings May 5, 2026 15:30
@novak-vaclav novak-vaclav force-pushed the feature/EIEX-681-add-transposed-conv-1d-support branch from 1578982 to de6a90a Compare May 5, 2026 15:30
@novak-vaclav
Copy link
Copy Markdown
Contributor Author

Rebased and resolved all comments. Please run the tests @MartinPavella
Thank you for reviews

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

backends/nxp/backend/ir/converter/node_converters/ops_converters/convolution_converter.py:437

  • After removing in-converter 1D convolution lowering, the NotImplementedError message will be hit if a model reaches IR conversion with rank-3 weights (i.e., the ConvertConv1dToConv2dPass wasn’t run). Consider updating the error message to explicitly point users to NeutronAtenPassManager / ConvertConv1dToConv2dPass so the failure is actionable.
        rank = t_op.tmp_inputs[1].shape.len()
        if rank == 4:  # Conv2D
            ops_to_add = self._convert_2d_conv(t_op, conv_params)
        else:
            raise NotImplementedError(
                f"{rank - 2}D convolution is not supported."
            )  # Should never get here.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread backends/nxp/aten_passes/convert_1d_conv_to_2d.py Outdated
Comment thread backends/nxp/aten_passes/convert_1d_conv_to_2d.py Outdated
@novak-vaclav novak-vaclav force-pushed the feature/EIEX-681-add-transposed-conv-1d-support branch from de6a90a to 4a73b63 Compare May 5, 2026 17:03
@novak-vaclav
Copy link
Copy Markdown
Contributor Author

Fixed some issues mentioned by Copilot.

Copilot AI review requested due to automatic review settings May 10, 2026 20:43
@novak-vaclav novak-vaclav force-pushed the feature/EIEX-681-add-transposed-conv-1d-support branch from 4a73b63 to 6e9b978 Compare May 10, 2026 20:43
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 7 comments.

Comment thread backends/nxp/tests/generic_tests/test_split_group_convolution.py Outdated
Comment thread backends/nxp/quantizer/patterns.py
Comment thread backends/nxp/aten_passes/convert_1d_conv_to_2d.py
Comment thread backends/nxp/aten_passes/convert_1d_conv_to_2d.py
Comment thread backends/nxp/aten_passes/convert_1d_conv_to_2d.py
@novak-vaclav novak-vaclav force-pushed the feature/EIEX-681-add-transposed-conv-1d-support branch from 6e9b978 to 2b79e1a Compare May 10, 2026 21:25
@novak-vaclav
Copy link
Copy Markdown
Contributor Author

novak-vaclav commented May 10, 2026

@MartinPavella
Had to fix one failing test suite - split_group_convolution as it assumed direct support of conv1d without its conversion to 2d.
Sorry for pushing the first commit together with the rebase. To ease the pain in the many changes, I found out that ctrl+f with string "nxp" can be used to search the relevant changes, as I did not find an option to collapse all the differences :(

Hopefully everything will pass now, please run the tests. Thank you 😊

Copy link
Copy Markdown
Collaborator

@MartinPavella MartinPavella left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Once tests pass, we can merge.

@novak-vaclav
Copy link
Copy Markdown
Contributor Author

@MartinPavella

Internal tests are passing.

The Github failing tests don't seem to be related to NXP code.

@MartinPavella
Copy link
Copy Markdown
Collaborator

The failing checks seem unrelated and the internal CI is passing. Merging.

@MartinPavella MartinPavella merged commit a018853 into pytorch:main May 11, 2026
173 of 174 checks passed
@MartinPavella MartinPavella deleted the feature/EIEX-681-add-transposed-conv-1d-support branch May 11, 2026 12:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: nxp Issues related to NXP Neutron NPU delegation and code under backends/nxp/ release notes: nxp Changes to the NXP Neutron backend delegate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants