[Relax][Frontend][TFLite] Add TFLite Resource Variable and Static Hashtable Import Support#19639
Conversation
There was a problem hiding this comment.
Code Review
This pull request adds support for resource variables and static hashtables in the Relax TFLite frontend. It introduces tracking for resource and hashtable values, supports several new operators (such as VAR_HANDLE, ASSIGN_VARIABLE, READ_VARIABLE, HASHTABLE, HASHTABLE_IMPORT, HASHTABLE_FIND, and HASHTABLE_SIZE), and updates CALL_ONCE handling to process non-empty initialization subgraphs. Comprehensive unit tests are also added. The review feedback highlights several robust improvements: wrapping optional TFLite imports in try-except blocks to prevent import errors on older environments, explicitly validating that hashtable keys and values are constant tensors, and correctly handling 0D (scalar) query keys in HASHTABLE_FIND.
c5f070d to
81aa1be
Compare
|
cc @tlopex |
tlopex
left a comment
There was a problem hiding this comment.
Thanks for working on this. I think the hashtable part does not currently match TFLite's builtin HASHTABLE semantics.
The TFLite kernels only support int64 -> string or string -> int64 tables. HASHTABLE_IMPORT also requires the key and value tensors to have the same shape, and HASHTABLE_FIND resizes the output to the query key tensor shape.
This PR's positive tests build INT32 -> INT32 tables and the converter supports imported values with shape [num_keys, ...], returning output shaped like query_shape + value_shape. That is not the contract accepted by the TFLite runtime. Conversely, actual valid TFLite hashtable models involving STRING tensors cannot be imported here because the frontend tensor dtype helpers do not support TensorType.STRING.
So this can accept synthetic/malformed hashtable flatbuffers while not supporting the real TFLite hashtable subset it claims to add. I think the converter should either match TFLite's kernel constraints and add positive tests for valid int64/string and string/int64 tables, or reject HASHTABLE_* until those semantics are supported. The current INT32 and vector-value tests should not be the positive coverage for this operator family.
8e01581 to
43d5042
Compare
|
Hi @tlopex, thanks for the detailed review. I've removed the incorrect The hashtable handling is now limited to TFLite's actual kernel constraints:
So this PR no longer models hashtables as a generic tensor map. It keeps the resource-variable support plus the safe hashtable metadata/size subset, and leaves real string lookup lowering for a future PR after string tensor support exists. Please take another look when you have time. |
|
Update PR description |
Summary
This PR adds incremental Relax TFLite frontend support for the resource
variable initialization subset:
VAR_HANDLEASSIGN_VARIABLEREAD_VARIABLEIt builds on the TFLite control-flow / multi-subgraph support from #19616,
especially
CALL_ONCE. TFLite commonly represents initialization through aCALL_ONCEinit subgraph, then uses resource handles from the main subgraph toread initialized variables. This PR supports that constrained initialization
pattern without introducing general mutable runtime state into Relax.
The PR also adds explicit frontend guards for the TFLite builtin hashtable
operators:
HASHTABLEHASHTABLE_IMPORTHASHTABLE_FINDHASHTABLE_SIZEThese operators are intentionally left unsupported for now. TFLite builtin
hashtable kernels are not generic tensor maps: their runtime implementations
cover the
int64 -> stringandstring -> int64table variants, and correctimport requires proper
TensorType.STRINGsupport. Rejecting the operators issafer than lowering a synthetic numeric table semantics that TFLite does not
actually implement.
Design
Shared Initialization State
The frontend now keeps resource initialization data in shared conversion state:
conversion_state["resource_values"]conversion_state["in_call_once_init"]This state is shared by the main graph converter and the
CALL_ONCEinitsubgraph converter. Each converter instance still keeps its own local
self.resource_handlesmap, keyed by TFLite tensor name.Resource variables use
container + shared_namefromVarHandleOptionswhenpresent, falling back to the handle tensor name. This keeps tensor-name bindings
scoped to each subgraph while allowing init subgraphs and the main graph to
agree on the same logical resource.
CALL_ONCE Init Subgraphs
CALL_ONCEnow accepts a non-empty init subgraph when all operators are in thesupported initialization subset:
VAR_HANDLEASSIGN_VARIABLEThe init subgraph still must have no inputs and no outputs. The converter first
checks every operator against the allowlist, then converts the init subgraph
with a fresh
ExprTableand shared conversion state.The init subconverter deliberately shares the parent
BlockBuilder. This issafe for the current subset because all supported init operators update importer
state and return
None; they do not emit Relax bindings. A comment documentsthat this should be revisited if future
CALL_ONCEinit operators emit Relaxexpressions.
Resource Variables
VAR_HANDLEis declarative. It registers the output resource tensor in thecurrent converter's local
resource_handlesmap and returnsNone.ASSIGN_VARIABLEis accepted only while converting a supportedCALL_ONCEinitsubgraph. It resolves the resource handle through the init converter's local
handle map and stores the assigned tensor expression in shared
conversion_state["resource_values"].READ_VARIABLEresolves the main graph resource handle and returns theinitialized expression from shared state. If the resource has not been
initialized by a supported
CALL_ONCEpath, the frontend raisesOpNotImplemented.This supports the common static-initialization inference pattern while avoiding
incorrect lowering for runtime mutation.
Hashtable Operators
HASHTABLEregisters the table handle and validates the dtype pair againstTFLite kernel constraints (
int64/stringorstring/int64).HASHTABLE_IMPORTin a supportedCALL_ONCEinit subgraph captures staticmetadata (table size, key/value dtypes) but does not store actual string data,
because Relax does not yet support
TensorType.STRING.HASHTABLE_SIZEreturns a scalar Relax constant for statically importedtables.
HASHTABLE_FINDis rejected withOpNotImplementedbecause Relax cannotrepresent TFLite string tensors or the runtime lookup semantics.
Operator Support
VAR_HANDLEVarHandleOptionsCALL_ONCEinit subgraphsASSIGN_VARIABLEAssignVariableOptionsCALL_ONCEinit subgraphs onlyREAD_VARIABLEReadVariableOptionsHASHTABLEHashtableOptionsint64/stringorstring/int64pair, rejects other combinationsHASHTABLE_IMPORTHashtableImportOptionsCALL_ONCEinit subgraphs only, constant key/value shape validationHASHTABLE_FINDHashtableFindOptionsTensorType.STRINGsupport in RelaxHASHTABLE_SIZEHashtableSizeOptions[size]int64 for statically imported tablesSafety Checks
ASSIGN_VARIABLEoutsideCALL_ONCEinitialization raisesOpNotImplemented.READ_VARIABLEwithout supported initialization raisesOpNotImplemented.CALL_ONCEinit subgraphs with inputs or outputs remain unsupported.CALL_ONCEinit subgraphs containing operators outside the resource-variableinitialization allowlist remain unsupported.
OpNotImplementeduntil thefrontend can model their real int64/string table semantics.
Not Included
ASSIGN_VARIABLEmutation in the main graph.returns.
container/shared_namematching pattern.TensorType.STRINGimport support.Tests
The tests manually build minimal TFLite flatbuffers and compare imported Relax
IR with
tvm.ir.assert_structural_equal. Unsupported patterns usepytest.raises.test_resource_variable_call_once_init_readCALL_ONCEinit subgraph withVAR_HANDLE + ASSIGN_VARIABLE, then main graphREAD_VARIABLEtest_assign_variable_main_subgraph_unsupportedASSIGN_VARIABLEguardtest_read_variable_uninitialized_unsupportedREAD_VARIABLEwithout supported initialization guardtest_hashtable_call_once_import_find_unsupportedtest_hashtable_call_once_import_size_unsupportedtest_hashtable_import_main_subgraph_unsupportedHASHTABLE_IMPORTremains unsupportedtest_hashtable_size_uninitialized_unsupportedHASHTABLE_SIZEremains unsupportedLocal validation:
python -m py_compile \ python/tvm/relax/frontend/tflite/tflite_frontend.py \ tests/python/relax/test_frontend_tflite.py python -m ruff format --check \ python/tvm/relax/frontend/tflite/tflite_frontend.py \ tests/python/relax/test_frontend_tflite.py python -m ruff check \ python/tvm/relax/frontend/tflite/tflite_frontend.py \ tests/python/relax/test_frontend_tflite.py python -m pytest \ tests/python/relax/test_frontend_tflite.py \ -k "resource_variable or read_variable_uninitialized or hashtable" -q python -m pytest \ tests/python/relax/test_frontend_tflite.py -qResult: