Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove PromoteTensorLoads pass, convert ExtractOp in TensorToFlow. #6852

Merged
merged 2 commits into from
Aug 25, 2021

Conversation

ScottTodd
Copy link
Member

@ScottTodd ScottTodd commented Aug 25, 2021

Fixes #6756 (the tosa if.mlir file compiles successfully using -iree-flow-enable-linalg-detensorize with this change)

The PromoteTensorLoads pass was converting i1 loads to i8 loads using ZeroExtendIOp and TruncateIOp. That was producing weird cycles during compilation when detensoring was applied, and flow ops should be fine with i1 types. We still need to handle i1 types when going to the HAL (since storage is incompatible) on the outside (external interface) and inside (codegen).

@ScottTodd ScottTodd added the compiler/dialects Relating to the IREE compiler dialects (flow, hal, vm) label Aug 25, 2021
@google-cla google-cla bot added the cla: yes label Aug 25, 2021
@iree-github-actions-bot
Copy link
Contributor

Abbreviated Benchmark Summary

@ commit 434c94aa23bc5215ca73902a9cc3b948cba8fdde (vs. base ce7b475992b38588fe73d8a16b4d2ab64284f8ea)

Regressed Benchmarks 🚩

Benchmark Name Average Latency (ms) Median Latency (ms) Latency Standard Deviation (ms)
MobileNetV2 [fp32,imagenet] (TensorFlow) full-inference with IREE-Vulkan @ SM-G980F (GPU-Mali-G77) 84 (vs. 62, 35.48%↑) 86 5
MobileNetV3Small [fp32,imagenet] (TensorFlow) 3-thread,big-core,full-inference with IREE-Dylib @ Pixel-4 (CPU-ARMv8.2-A) 17 (vs. 15, 13.33%↑) 17 0

Improved Benchmarks 🎉

Benchmark Name Average Latency (ms) Median Latency (ms) Latency Standard Deviation (ms)
MobileBertSquad [fp32] (TensorFlow) big-core,full-inference with IREE-Dylib-Sync @ Pixel-4 (CPU-ARMv8.2-A) 738 (vs. 910, 18.90%↓) 738 1

For more information:

@ScottTodd
Copy link
Member Author

Benchmarks regressed a bit. May need to diff the IR for MobileNetV2 through the Vulkan target to see what happened...

@ScottTodd
Copy link
Member Author

Benchmarks regressed a bit. May need to diff the IR for MobileNetV2 through the Vulkan target to see what happened...

IR diff shows no change. Benchmark results appear to be in the noise on closer inspection.

@ScottTodd ScottTodd merged commit d90f0fc into iree-org:main Aug 25, 2021
@ScottTodd ScottTodd deleted the promote-tensor-loads branch August 25, 2021 18:16
@hanhanW hanhanW mentioned this pull request Aug 25, 2021
copybara-service bot pushed a commit that referenced this pull request Aug 25, 2021
* 30463f1 Adding #util.composite attribute. (#6854)
* 7a1c579 Implement function.py return type coercion (#6832)
* d90f0fc Remove PromoteTensorLoads pass, convert ExtractOp in TensorToFlow. (#6852)
* 7fa8c20 Update SwiftShader to 2021-08-25 (#6859)
* a328761 Rename Bazel repo iree_vulkan_headers to vulkan_headers (#6862)
* 7660e49 Merge pull request #6856 from google/benvanik-shared-target-backend
* ebc5eb5 Removing the pipeline caching during executable translation. (#6855)
* 2fb7a8d Reapply "Update TFLite concrete function conversion codes" (#6800)
* ce7b475 NFC: Merge ConvertToFlow passes into dedicated before/after passes. (#6850)
* 9fce2d6 Support f32 in the VM by default in the compiler. (#6744)
* b6baea9 Support global ref ops and fix passing of refs on function boundaries in the C..
* 4b5d2ed Bump flatcc version (#6853)
* c8a8f5d Add e2e tests for mhlo.bitcast_convert (#6846)

COPYBARA_INTEGRATE_REVIEW=#6865 from hanhanW:main-to-google 30463f1
PiperOrigin-RevId: 392952973
iree-github-actions-bot pushed a commit that referenced this pull request Aug 25, 2021
* 30463f1 Adding #util.composite attribute. (#6854)
* 7a1c579 Implement function.py return type coercion (#6832)
* d90f0fc Remove PromoteTensorLoads pass, convert ExtractOp in TensorToFlow. (#6852)
* 7fa8c20 Update SwiftShader to 2021-08-25 (#6859)
* a328761 Rename Bazel repo iree_vulkan_headers to vulkan_headers (#6862)
* 7660e49 Merge pull request #6856 from google/benvanik-shared-target-backend
* ebc5eb5 Removing the pipeline caching during executable translation. (#6855)
* 2fb7a8d Reapply "Update TFLite concrete function conversion codes" (#6800)
* ce7b475 NFC: Merge ConvertToFlow passes into dedicated before/after passes. (#6850)
* 9fce2d6 Support f32 in the VM by default in the compiler. (#6744)
* b6baea9 Support global ref ops and fix passing of refs on function boundaries in the C..
* 4b5d2ed Bump flatcc version (#6853)
* c8a8f5d Add e2e tests for mhlo.bitcast_convert (#6846)

PiperOrigin-RevId: 392952973
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/dialects Relating to the IREE compiler dialects (flow, hal, vm) (deprecated) buildkite:benchmark-android Deprecated. Please use benchmarks:android-*
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Lowering of i1 types conflicts with detensoring
3 participants