Skip to content

ORT raises node "does not have type information set by parent node" for initializers declared in outer graph #24880

New issue
New issue
Open
#25068
Open
ORT raises node "does not have type information set by parent node" for initializers declared in outer graph#24880
#25068
Assignees
copilot-swe-agent
Labels
core runtimeissues related to core runtimemodel:transformerissues related to a transformer model: BERT, GPT2, Hugging Face, Longformer, T5, etc.
@justinchuby

Description

@justinchuby
justinchuby
opened on May 28, 2025 · edited by justinchuby
Contributor

Describe the issue

Checked in

onnxruntime/onnxruntime/core/graph/graph.cc

Line 2843 in f8c1393

if (is_outer_scope_nodearg(input_def->Name()))
and manifested in

onnxruntime/onnxruntime/python/tools/transformers/convert_generation.py

Lines 1077 to 1081 in f8c1393

# Add type info, otherwise ORT will raise error: "input arg (*) does not have type information set by parent node."
for initializer in moved_initializers:
shape = onnx.numpy_helper.to_array(initializer).shape
value_info = onnx.helper.make_tensor_value_info(initializer.name, initializer.data_type, shape)
graph.value_info.append(value_info)

is_outer_scope_nodearg doesn't seem to return true when the node input is an initializer defined in the outer graph (w/o having a value_info proto in the subgraph).

To reproduce

Produce a node with a subgraph that references an outer graph, and do not create a value_info for the initializer in the subgraph.

The node I observed was from

Urgency

No response

Platform

Linux

OS Version

N/A

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.22

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

Activity

justinchuby
added
core runtimeissues related to core runtime
on May 28, 2025
justinchuby

justinchuby commented on May 28, 2025

@justinchuby
justinchuby
on May 28, 2025
ContributorAuthor

cc @skottmckay

justinchuby
mentioned this on May 28, 2025
skottmckay

skottmckay commented on May 28, 2025

@skottmckay
skottmckay
on May 28, 2025
Contributor

When programmatically building a subgraph you need to manually specify outer scope node args given they won't be known until the subgraph is added to its parent graph. That is done by calling AddOuterScopeNodeArg, and that is what is_outer_scope_nodearg is checking. That is expected to return false when the subgraph is within a parent graph as the type info should come from the real values in the parent graph.

Graph::InferAndVerifySubgraphTypes adds the type info. Any values consumed within the subgraph should be in the implicit inputs for the node that contains the subgraph.

onnxruntime/onnxruntime/core/graph/graph.cc

Lines 2764 to 2787 in f8c1393

// Apply any current input type/shape information to the Nodes in the subgraph that are implicitly
// consuming NodeArg's from this scope or higher.
// The NodeArg's that implicit_input_defs point to would have any type/shape inferencing applied to them
// by now. As the subgraph is referring to the outer scope NodeArg, we simply replace any information in
// the subgraph with the details from the outer scope NodeArg.
const auto& implicit_input_defs = node.GetDefinitions().implicit_input_defs;
for (const auto* implicit_node_arg : implicit_input_defs) {
auto subgraph_nodearg = subgraph.GetNodeArg(implicit_node_arg->Name());
// the implicit input defs may be for a nested subgraph, so it won't necessarily match here.
// if that is the case, we will update the type/shape information when we descend into the
// nested subgraph later.
if (!subgraph_nodearg)
continue;
status = subgraph_nodearg->UpdateTypeAndShape(*implicit_node_arg, true, options.override_types, subgraph.logger_);
if (!status.IsOK()) {
return ORT_MAKE_STATUS(ONNXRUNTIME, FAIL, "Node:", node.Name(), " ", status.ErrorMessage());
}
// all values above us should have a type by now due to ONNX requirements.
if (subgraph_nodearg->Type() == nullptr)
return ORT_MAKE_STATUS(ONNXRUNTIME, FAIL, "Subgraph input missing type.");
}

github-actions
added
model:transformerissues related to a transformer model: BERT, GPT2, Hugging Face, Longformer, T5, etc.
on May 28, 2025
justinchuby
assigned
Copilot
on Jun 14, 2025
Copilot
linked a pull request that will close this issue on Jun 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

core runtimeissues related to core runtimemodel:transformerissues related to a transformer model: BERT, GPT2, Hugging Face, Longformer, T5, etc.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    Participants

    @skottmckay@justinchuby

    Issue actions
      ORT raises node "does not have type information set by parent node" for initializers declared in outer graph · Issue #24880 · microsoft/onnxruntime