to_executorch segfaults when constant tensors are in device memory

[shoumikhin]

Serializing a program with CUDA parameters segfaults the emitter

to_executorch() crashes with a segmentation fault when a program's constant
tensors live in device memory. The emitter reads a constant's storage through a
host pointer cast, with no check of where that storage actually is:

# exir/emit/_emitter.py, _tensor_spec_to_evalue
buffer_data = bytes(
    ctypes.cast(
        typing.cast(torch.UntypedStorage, spec.storage).data_ptr(),
        ctypes.POINTER(spec_array_type),
    ).contents
)

data_ptr() on a CUDA storage is a device address, so ctypes.cast reads device
memory from the host.

Reproducer, on a machine with a CUDA build of torch:

import torch
from torch.export import export
from executorch.exir import to_edge_transform_and_lower

model = torch.nn.Sequential(torch.nn.Linear(4, 3), torch.nn.ReLU()).eval().cuda()
example = torch.ones(2, 4).cuda()
program = to_edge_transform_and_lower(export(model, (example,))).to_executorch()

The faulthandler traceback points straight at the cast:

Fatal Python error: Segmentation fault
Current thread ...:
  File ".../executorch/exir/emit/_emitter.py", line 460 in _tensor_spec_to_evalue

The same script with host parameters succeeds, which is the workaround: keep
parameters on the host and let the partitioner see the device only through the
example input.

Suggested fix: copy the storage to the host before serializing when it is not
already there, or fail with a clear message instead of reading device memory
from the host.

Found while running an end-to-end delegate export on Linux aarch64. Not specific
to any one backend, since the crash is in the shared serialization path.

[AxelNoun]

Hello @shoumikhin happy to pick this up if it's not already in flight; the fix looks small and well-scoped.

Root cause: _tensor_spec_to_evalue dereferences spec.storage.data_ptr() host-side via ctypes.cast(...).contents, but for any non-CPU storage (CUDA, and the same class of problem for MPS/XPU) data_ptr() is a device address, so the host read faults. Not CUDA-specific.

Proposed fix (emit-site, minimal):

storage = typing.cast(torch.UntypedStorage, spec.storage)
if storage.device.type != "cpu":
    storage = storage.cpu()  # transient host copy, bounded by the largest constant
buffer_data = bytes(
    ctypes.cast(storage.data_ptr(), ctypes.POINTER(spec_array_type)).contents
)

Copy-to-host is lossless here: the .pte constant segment is raw host bytes regardless of the exporting device, and the runtime/delegate re-places them at load. So I'd lean toward copying silently rather than raising, raising would force every device-side export workflow to manually .cpu() the module first. Can add a debug-level log on the copy.

Two questions before opening the PR:

1. Fix at the read site only (safety net), or add a normalization pass in to_executorch() that moves all constant storages to CPU up front and keeps the emitter allocation-free? Happy to do both (pass + assert at emit).

2. Are there other raw data_ptr() host reads in the serialization path (e.g. the external/named-data constants flow) worth patching in the same PR? I'll grep, but flagging in case you know offhand.

Tests: the repro from the issue gated on torch.cuda.is_available() (parametrized over available accelerators), plus a byte-equality check that the same weights exported from CPU vs device produce an identical program buffer.

[Gasoonjia]

Hi @shoumikhin thanks for your question but i don't think

model = torch.nn.Sequential(torch.nn.Linear(4, 3), torch.nn.ReLU()).eval().cuda()
example = torch.ones(2, 4).cuda()
program = to_edge_transform_and_lower(export(model, (example,))).to_executorch()

is a legal way to export:

1. We didn't design for exporting from cuda model and cuda input for our regular path
2. This code snippet is trying to lower a cuda model onto portable library, which we can't support. We can try it on cuda backend and it works:

(et0) [gasoonjia@devvm38968.vll0 ~]$ python
Python 3.11.15 (main, Jun 11 2026, 15:20:16) [GCC 14.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> from torch.export import export
>>> from executorch.exir import EdgeCompileConfig, to_edge_transform_and_lower
W0803 00:01:15.075000 2749909 site-packages/torch/utils/_pytree.py:630] <enum 'KernelPreference'> is an Enum subclass and is now natively supported by torch.compile as an opaque value type. Calling register_constant() on Enum subclasses is deprecated and will be an error in a future release.
>>> from executorch.backends.cuda.cuda_backend import CudaBackend
>>> from executorch.backends.cuda.cuda_partitioner import CudaPartitioner
>>> model = torch.nn.Sequential(
...     torch.nn.Linear(4, 3),
...     torch.nn.ReLU(),
... ).eval().cuda()
>>> 
>>> example = torch.ones(2, 4).cuda()
>>> 
>>> partitioner = CudaPartitioner([
...     CudaBackend.generate_method_name_compile_spec("forward")
... ])
<stdin>:1: ExperimentalWarning: This API and all of cuda backend related functionality are experimental.
>>> 
>>> program = to_edge_transform_and_lower(
...     export(model, (example,)),
...     partitioner=[partitioner],
...     compile_config=EdgeCompileConfig(_check_ir_validity=False),
... ).to_executorch()
/home/gasoonjia/.conda/envs/et0/lib/python3.11/copyreg.py:105: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
  return cls.__new__(cls, *args)
/home/gasoonjia/.conda/envs/et0/lib/python3.11/copyreg.py:105: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
  return cls.__new__(cls, *args)
/home/gasoonjia/.conda/envs/et0/lib/python3.11/copyreg.py:105: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
  return cls.__new__(cls, *args)
/home/gasoonjia/.conda/envs/et0/lib/python3.11/copyreg.py:105: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
  return cls.__new__(cls, *args)
/home/gasoonjia/.conda/envs/et0/lib/python3.11/site-packages/torch/_inductor/compile_fx.py:320: UserWarning: TensorFloat32 tensor cores for float32 matrix multiplication available but not enabled. Consider setting `torch.set_float32_matmul_precision('high')` for better performance.
  warnings.warn(
/home/gasoonjia/.conda/envs/et0/lib/python3.11/copyreg.py:105: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
  return cls.__new__(cls, *args)
>>> 
>>> print(len(program.buffer))
396192

and also hi @AxelNoun thank you for your enthusiasm! It is so nice having your support! Let me have a better understanding of shoumikhin's concern and we can further collberate on this issue!

[shoumikhin]

Agreed that the snippet in my report is not a supported path; lowering a CUDA module onto the portable library is user error. My ask is only about how it fails: today it is a bare segfault (exit 139) with no Python exception, no message naming the device, and no stack. The cause is a host read of a device pointer in the emitter, where _tensor_spec_to_evalue does ctypes.cast(spec.storage.data_ptr(), ...) on CUDA storage. A small guard that raises InternalError when spec.storage.device.type != "cpu" would turn the crash into an actionable message without changing any supported behavior.

Separately, I could not reproduce the partitioner side-by-side on my hardware, but for an unrelated reason: it stops earlier in Inductor with NoValidChoicesError (a Triton GEMM template gap for this device), not in the emitter.