Arm backend: Add initial module tests for Stable Diffusion 3.5 Medium #12242
Conversation
YufengShi-dudu
added
partner: arm
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/12242
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (2 Unrelated Failures)As of commit f52a160 with merge base b0dda93 ( FLAKY - The following job failed but was likely due to flakiness present on trunk:
BROKEN TRUNK - The following job failed but was present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
|
Hi @digantdesai, We added some test configs to stable_diffusion_module_test_configs.py. They're mostly copy-pasted from the diffusers library to initialize different modules for testing. We've included license notice and copyright attribution, and also links to the original source files for reference, Please let us know if you have any concerns. |
Thx for highlighting this @YufengShi-dudu. With the proper attributions in place and a clear separation (separate file), it should be fine. But pls @digantdesai or @mergennachin - pls confirm. |
- Add module tests for two text encoders: CLIP and T5 - Add module tests for VAE autoencoder - Add module tests for SD3Transformer2DModel - Add flag to exmaples/arm/setup.sh for installing Stable Diffusion dependencies - Handle int64 inputs to aten.slice_copy.Tensor using pass InsertCastForOpsWithInt64InputPass Change-Id: I4389e87749cfb4e40f837cc6bfa8a59a73fb3a3a Signed-off-by: Yufeng Shi <yufeng.shi@arm.com>
YufengShi-dudu
force-pushed
the
stable-diffusion-module-tests
branch
from
53447d4 to
42a4656
Compare
|
Hi @digantdesai, |
| In order to lower this to TOSA and int64->int32 cast need to be injected. This pass need to run very early in the lowering process and can be passed in to the to_edge_transform_and_lower() function call as an optional parameter. See example in: backends/arm/test/models/test_llama.py. | ||
| By doing this aten.embedding will be decomposed into to aten.index_select which can handle int32 indices. | ||
| Note that this additional step is only needed for pure float models. With quantization this is automatically handled during annotation before the export stage. | ||
| The current TOSA version does not support int64. However, int64 is commonly used in many models. In order to lower the operators with int64 inputs and/or outputs to TOSA, a few passes have been developed to handle the int64-related issues. The main idea behind these passes is to replace the uses of int64 with int32 where feasible. |
There was a problem hiding this comment.
this is also the case with xnnpack and other backends, perhaps we should share this pass.
There was a problem hiding this comment.
This pass currently has some tosa-specific code/log info. Could we refactor it to exir in a follow up PR if needed by other backends?
There was a problem hiding this comment.
Issue created: #12957
- Extract the logic of inserting int32 cast node into a helper function in pass InsertCastForOpsWithInt64InputPass Change-Id: I748d53921981d43c3a913c4e7a15f3daf33c836e Signed-off-by: Yufeng Shi <yufeng.shi@arm.com>
5 participants
Change-Id: I4389e87749cfb4e40f837cc6bfa8a59a73fb3a3a
cc @digantdesai @freddan80 @per @zingo @oscarandersson8218