Vulkan: serialize non-finite floats using flatc's spelling - #22307
Open
msluszniak wants to merge 1 commit into
Open
Vulkan: serialize non-finite floats using flatc's spelling#22307msluszniak wants to merge 1 commit into
msluszniak wants to merge 1 commit into
Conversation
The Vulkan graph is serialized by dumping it to JSON with Python's json module and handing that to flatc. The two disagree on how to spell the non-finite floats: Python emits Infinity / -Infinity / NaN, and flatc accepts none of them. Any model whose graph carries a non-finite scalar therefore fails to lower, with the -inf fill value of a transformer attention mask being the common source. all-MiniLM-L6-v2 and CLIP ViT-B/32 both partition cleanly and then die here: schema.json:1: 6012: error: cannot parse value starting with: - The error points at a byte offset inside a temporary file, so nothing connects it back to the model. flatc does accept "inf" and "-inf", so emit those. The decompile direction needs the inverse rewrite, since flatc writes bare inf tokens that json.load will not parse; string literals are stepped over so a shader or key name containing "inf" is left alone. FlatBuffers JSON has no spelling for NaN at all, so raise there rather than emitting JSON that flatc cannot read. Adds round-trip coverage for the infinities, the NaN error, and the string-literal case.
pytorch-bot
Bot
added
the
module: vulkan
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/22307
Note: Links to docs will display an error until the docs builds have been completed.
|
meta-cla
Bot
added
the
CLA Signed
This PR needs a
|
msluszniak
added a commit
to software-mansion-labs/executorch
that referenced
this pull request
Aug 30, 2026
vulkan_graph_serialize.py: the flatc JSON dialect has no spelling for Python's Infinity/NaN, so a graph carrying a non-finite float failed to serialize. Emit inf/-inf the way flatc parses them, and reject NaN explicitly rather than writing something flatc cannot read back. utils.py: OpRepSets used num_tensors_in_node() == 1 to decide that meta["val"] is a bare FakeTensor, but that helper counts tensors rather than nesting, so an op declared to return Tensor[] that happens to produce exactly one lands there with a one-element list and crashes the partitioner. Unwrap it. Backports of upstream pytorch/executorch#22307 and #22308.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #22305.
The Vulkan graph is serialized by dumping it to JSON with Python's
jsonmodule and handing that toflatc. The two disagree on how to spell the non-finite floats: Python emitsInfinity/-Infinity/NaN, andflatcaccepts none of them. Any model whose graph carries a non-finite scalar therefore fails to lower, with the-inffill value of a transformer attention mask being the common source.all-MiniLM-L6-v2and CLIP ViT-B/32 both partition cleanly and then die with:which names a byte offset inside a deleted temporary file and points at nothing in the model.
flatcdoes acceptinfand-inf(checked against flatc 24.3.25;nan,NaN,infinityare all rejected), so the infinities round-trip exactly. The decompile direction needs the inverse rewrite, sinceflatcwrites bareinftokens thatjson.loadwill not parse; string literals are stepped over so a shader or key name containing "inf" is left alone. FlatBuffers JSON has no spelling for NaN at all, so that case raises with a message naming the problem rather than emitting JSONflatccannot read.Test plan
Three new cases in
backends/vulkan/test/test_serialization.py: an exact round-trip of-inf/inf/ a finite value, the NaN error, and a string literal containing "inf" surviving the decode rewrite untouched.Verified end to end that
sentence-transformers/all-MiniLM-L6-v2now lowers throughVulkanPartitioner(14 delegate blobs, 3/3 runs atPYTHONHASHSEED=0), where before it failed 3/3.cc @SS-JIA @manuelcandales @digantdesai @cbilgin