[Bug] Grafting Fails Due to Entity ID Type Mismatch (`ID!` vs. `Bytes!`)

[0x19dG87]

Summary

Subgraph syncing fails during the grafting process because of a type mismatch for the ID attribute of an entity between the source (graft) and the destination (new subgraph). Specifically, the destination schema defines the attribute as Bytes!, while the source schema defines it as ID!. This causes the data copy operation to fail immediately with a store error, preventing the new subgraph from ever syncing.

Steps to Reproduce

1. Start indexing subgraph QmWZnioK8S9HL45kQqKFd67NxeqM1uEoVhYrrbcPDxuQSV and its graft QmQfYe5Ygg9A3mAiuBZYj5a64bDKLF4gF6sezfhgxKvb9y.
2. Wait graft to reach copy point
3. Check in graphman copying process graphman copy list started.
4. Open graph node logs and wait for error. You can restart subgraph in graphman to catch error (graphman restart QmWZnioK8S9HL45kQqKFd67NxeqM1uEoVhYrrbcPDxuQSV).

Observed Behavior

The subgraph fails during the data copy phase of the graft with a store error. The relevant logs show the following exception:

The attribute User.id has type Bytes!, but its type in the source is ID!

Relevant Logs:

Dec 01 21:08:53.909 INFO Initializing graft by copying data from sgd91 to sgd108, runner_index: 28, sgd: 108, subgraph_id: QmWZnioK8S9HL45kQqKFd67NxeqM1uEoVhYrrbcPDxuQSV, component: SubgraphInstanceManager
...
Dec 01 21:08:55.953 INFO Starting to copy `User` entities from "sgd91"."user" to "sgd108"."user", dst: sgd108, runner_index: 28, sgd: 108, subgraph_id: QmWZnioK8S9HL45kQqKFd67NxeqM1uEoVhYrrbcPDxuQSV, component: SubgraphInstanceManager
...
Dec 01 21:08:55.956 ERRO copy worker had an error: store error: The attribute User.id has type Bytes!, but its type in the source is ID!, dst: sgd108, runner_index: 28, sgd: 108, subgraph_id: QmWZnioK8S9HL45kQqKFd67NxeqM1uEoVhYrrbcPDxuQSV, component: SubgraphInstanceManager
Dec 01 21:08:55.958 ERRO Failed to start subgraph, code: SubgraphStartFailure, error: store error: The attribute User.id has type Bytes!, but its type in the source is ID!, runner_index: 28, sgd: 108, subgraph_id: QmWZnioK8S9HL45kQqKFd67NxeqM1uEoVhYrrbcPDxuQSV, component: SubgraphInstanceManager

The subgraph is now in a "never synced" state without progressing or failing deterministically.

graphman status:

deployment           | QmWZnioK8S9HL45kQqKFd67NxeqM1uEoVhYrrbcPDxuQSV
action               | sgd91 -> sgd108 (base)
started              | 2025-11-26T22:57:28+00:00
progress             | 0.00% done, 16/61680650

Expected Behavior

1. Ideal (Preventative): The system should prevent the deployment of a subgraph if its schema contains incompatible type changes for entities that are present in the specified graft deployment.
2. Acceptable (Recovery/Conversion): If deployment is allowed, the grafting process should either successfully convert the data types during the copy (e.g., automatically convert ID! which can contains hex strings into Bytes!) or safely stop the process with a deterministic error that allows to generate valid POI.

Potential Solutions and Next Steps

This issue has implications for slashing risk if indexers are stalled on broken deployments. Two main approaches are suggested to resolve this:

1. Schema Validation (Preventative): Implement a check during deployment to ensure that the schema of the new subgraph is fully compatible with the schema of the graft target for all common entities. If a type conflict like ID! vs. Bytes! is detected, the deployment should be rejected. This is the safest approach to prevent future issues.
2. Type Conversion (Runtime Fix): Implement logic within the grafting process to attempt type conversion for known, common mismatches like ID to Bytes (or vice-versa), especially for values that resemble hex addresses (0x...). This would address currently deployed subgraphs that are stuck, though it is a more complex implementation.

Immediate Need: We need a way to recover/stop the currently stalled subgraphs or implement a fix to handle this type of schema mismatch during grafting.

Relevant log output

IPFS hash

QmWZnioK8S9HL45kQqKFd67NxeqM1uEoVhYrrbcPDxuQSV

Subgraph name or link to explorer

https://thegraph.com/explorer/subgraphs/7pXNLCc12pRM3bBPUAP9ZoEvkgUCjaBe9QC3DV9L2qzE?v=2&view=Indexers&chain=arbitrum-one

Some information to help us out

• Tick this box if this bug is caused by a regression found in the latest release.
• Tick this box if this bug is specific to the hosted service.
• I have searched the issue tracker to make sure this issue is not a duplicate.

OS information

None