Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

torch.gather - Cannot call numel() on tensor with symbolic sizes/strides #122773

Open
rahulvijayaraghavan opened this issue Mar 27, 2024 · 0 comments
Open

torch.gather - Cannot call numel() on tensor with symbolic sizes/strides #122773

rahulvijayaraghavan opened this issue Mar 27, 2024 · 0 comments
Labels
module: dynamic shapes oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@rahulvijayaraghavan
Copy link

rahulvijayaraghavan commented Mar 27, 2024 •
edited by pytorch-bot bot

馃悰 Describe the bug

Following error is observed for torch.gather op out variant:
Cannot call numel() on tensor with symbolic sizes/strides

Error logs

---------------------------------------------------------------------------
TorchRuntimeError                         Traceback (most recent call last)
[<ipython-input-5-b4bd9d5a4e80>](https://localhost:8080/#) in <cell line: 14>()
     12 inp2 = torch.randn(1, 3)
     13 ind2 = torch.randint(2, (1, 1))
---> 14 opt_model(inp2, ind2, out)
     15 print(out)
     16 # Cannot call numel() on tensor with symbolic sizes/strides

31 frames
[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/eval_frame.py](https://localhost:8080/#) in _fn(*args, **kwargs)
    407             prior = set_eval_frame(callback)
    408             try:
--> 409                 return fn(*args, **kwargs)
    410             finally:
    411                 set_eval_frame(prior)

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/convert_frame.py](https://localhost:8080/#) in catch_errors(frame, cache_entry, frame_state)
    937         with compile_lock, _disable_current_modes():
    938             # skip=1: skip this frame
--> 939             return callback(frame, cache_entry, hooks, frame_state, skip=1)
    940 
    941     catch_errors._torchdynamo_orig_callable = callback  # type: ignore[attr-defined]

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/convert_frame.py](https://localhost:8080/#) in _convert_frame(frame, cache_entry, hooks, frame_state, skip)
    800         counters["frames"]["total"] += 1
    801         try:
--> 802             result = inner_convert(
    803                 frame, cache_entry, hooks, frame_state, skip=skip + 1
    804             )

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/convert_frame.py](https://localhost:8080/#) in _convert_frame_assert(frame, cache_entry, hooks, frame_state, skip)
    398         )
    399 
--> 400         return _compile(
    401             frame.f_code,
    402             frame.f_globals,

[/usr/lib/python3.10/contextlib.py](https://localhost:8080/#) in inner(*args, **kwds)
     77         def inner(*args, **kwds):
     78             with self._recreate_cm():
---> 79                 return func(*args, **kwds)
     80         return inner
     81 

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/convert_frame.py](https://localhost:8080/#) in _compile(code, globals, locals, builtins, compiler_fn, one_graph, export, export_constraints, hooks, cache_size, frame, frame_state, compile_id, skip)
    684         fail_user_frame_lineno: Optional[int] = None
    685         try:
--> 686             guarded_code = compile_inner(code, one_graph, hooks, transform)
    687             return guarded_code
    688         except (

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/utils.py](https://localhost:8080/#) in time_wrapper(*args, **kwargs)
    263             with torch.profiler.record_function(f"{key} (dynamo_timed)"):
    264                 t0 = time.time()
--> 265                 r = func(*args, **kwargs)
    266                 time_spent = time.time() - t0
    267             compilation_time_metrics[key].append(time_spent)

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/convert_frame.py](https://localhost:8080/#) in compile_inner(code, one_graph, hooks, transform)
    539             CompileContext.get().attempt = attempt
    540             try:
--> 541                 out_code = transform_code_object(code, transform)
    542                 break
    543             except exc.RestartAnalysis as e:

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/bytecode_transformation.py](https://localhost:8080/#) in transform_code_object(code, transformations, safe)
   1034     propagate_line_nums(instructions)
   1035 
-> 1036     transformations(instructions, code_options)
   1037     return clean_and_assemble_instructions(instructions, keys, code_options)[1]
   1038 

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/convert_frame.py](https://localhost:8080/#) in _fn(*args, **kwargs)
    163         cleanup = setup_compile_debug()
    164         try:
--> 165             return fn(*args, **kwargs)
    166         finally:
    167             cleanup.close()

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/convert_frame.py](https://localhost:8080/#) in transform(instructions, code_options)
    501         try:
    502             with tracing(tracer.output.tracing_context), tracer.set_current_tx():
--> 503                 tracer.run()
    504         except exc.UnspecializeRestartAnalysis:
    505             speculation_log.clear()

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py](https://localhost:8080/#) in run(self)
   2114 
   2115     def run(self):
-> 2116         super().run()
   2117 
   2118     def match_nested_cell(self, name, cell):

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py](https://localhost:8080/#) in run(self)
    837             try:
    838                 self.output.push_tx(self)
--> 839                 while self.step():
    840                     pass
    841             except BackendCompilerFailed:

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py](https://localhost:8080/#) in step(self)
    751 
    752         try:
--> 753             self.dispatch_table[inst.opcode](self, inst)
    754             return not self.output.should_exit
    755         except ReturnValueOp:

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py](https://localhost:8080/#) in wrapper(self, inst)
    479                 return handle_graph_break(self, inst, speculation.reason)
    480             try:
--> 481                 return inner_fn(self, inst)
    482             except Unsupported as excp:
    483                 if self.generic_context_manager_depth > 0:

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py](https://localhost:8080/#) in CALL_FUNCTION_KW(self, inst)
   1253         kwargs = dict(zip(argnames, kwargs_list))
   1254         assert len(kwargs) == len(argnames)
-> 1255         self.call_function(fn, args, kwargs)
   1256 
   1257     def LOAD_METHOD_SUPER(self, inst):

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/symbolic_convert.py](https://localhost:8080/#) in call_function(self, fn, args, kwargs)
    691         if inner_fn and callable(inner_fn) and is_forbidden(inner_fn):
    692             raise AssertionError(f"Attempt to trace forbidden callable {inner_fn}")
--> 693         self.push(fn.call_function(self, args, kwargs))
    694 
    695     def inline_user_function_return(self, fn, args, kwargs):

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/variables/torch.py](https://localhost:8080/#) in call_function(self, tx, args, kwargs)
    743                     fn_ = getattr(torch, torch_sym_op)
    744 
--> 745             tensor_variable = wrap_fx_proxy(
    746                 tx=tx,
    747                 proxy=tx.output.create_proxy(

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/variables/builder.py](https://localhost:8080/#) in wrap_fx_proxy(tx, proxy, example_value, subclass_type, **options)
   1339     }
   1340     if subclass_type is None:
-> 1341         return wrap_fx_proxy_cls(target_cls=TensorVariable, **kwargs)
   1342     else:
   1343         result = wrap_fx_proxy_cls(target_cls=TensorWithTFOverrideVariable, **kwargs)

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/variables/builder.py](https://localhost:8080/#) in wrap_fx_proxy_cls(target_cls, tx, proxy, example_value, subclass_type, **options)
   1424         # only allow_non_graph_fake in this instance because we handle the non-fake
   1425         # cases properly below.
-> 1426         example_value = get_fake_value(proxy.node, tx, allow_non_graph_fake=True)
   1427 
   1428     # Handle recursive calls here

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/utils.py](https://localhost:8080/#) in get_fake_value(node, tx, allow_non_graph_fake)
   1744             unimplemented(f"TypeError {node.target}: {cause}")
   1745 
-> 1746         raise TorchRuntimeError(str(e)).with_traceback(e.__traceback__) from None
   1747 
   1748     if not allow_non_graph_fake:

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/utils.py](https://localhost:8080/#) in get_fake_value(node, tx, allow_non_graph_fake)
   1676     try:
   1677         with tx.fake_mode, enable_python_dispatcher():
-> 1678             ret_val = wrap_fake_exception(
   1679                 lambda: run_node(tx.output, node, args, kwargs, nnmodule)
   1680             )

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/utils.py](https://localhost:8080/#) in wrap_fake_exception(fn)
   1207 def wrap_fake_exception(fn):
   1208     try:
-> 1209         return fn()
   1210     except UnsupportedFakeTensorException as e:
   1211         from .exc import unimplemented

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/utils.py](https://localhost:8080/#) in <lambda>()
   1677         with tx.fake_mode, enable_python_dispatcher():
   1678             ret_val = wrap_fake_exception(
-> 1679                 lambda: run_node(tx.output, node, args, kwargs, nnmodule)
   1680             )
   1681     except Unsupported:

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/utils.py](https://localhost:8080/#) in run_node(tracer, node, args, kwargs, nnmodule)
   1812             unimplemented(make_error_message(e), from_exc=e)
   1813         except Exception as e:
-> 1814             raise RuntimeError(make_error_message(e)).with_traceback(
   1815                 e.__traceback__
   1816             ) from e

[/usr/local/lib/python3.10/dist-packages/torch/_dynamo/utils.py](https://localhost:8080/#) in run_node(tracer, node, args, kwargs, nnmodule)
   1794         try:
   1795             if op == "call_function":
-> 1796                 return node.target(*args, **kwargs)
   1797             elif op == "call_method":
   1798                 return getattr(args[0], node.target)(*args[1:], **kwargs)

[/usr/local/lib/python3.10/dist-packages/torch/utils/_stats.py](https://localhost:8080/#) in wrapper(*args, **kwargs)
     18             simple_call_counter[fn.__qualname__] = 0
     19         simple_call_counter[fn.__qualname__] = simple_call_counter[fn.__qualname__] + 1
---> 20         return fn(*args, **kwargs)
     21     return wrapper

[/usr/local/lib/python3.10/dist-packages/torch/_subclasses/fake_tensor.py](https://localhost:8080/#) in __torch_dispatch__(self, func, types, args, kwargs)
    860         ), func
    861         try:
--> 862             return self.dispatch(func, types, args, kwargs)
    863         except TypeError:
    864             log.exception("fake tensor raised TypeError")

[/usr/local/lib/python3.10/dist-packages/torch/_subclasses/fake_tensor.py](https://localhost:8080/#) in dispatch(self, func, types, args, kwargs)
   1205 
   1206         if self.cache_enabled:
-> 1207             return self._cached_dispatch_impl(func, types, args, kwargs)
   1208         else:
   1209             return self._dispatch_impl(func, types, args, kwargs)

[/usr/local/lib/python3.10/dist-packages/torch/_subclasses/fake_tensor.py](https://localhost:8080/#) in _cached_dispatch_impl(self, func, types, args, kwargs)
    938 
    939         if output is unassigned:
--> 940             output = self._dispatch_impl(func, types, args, kwargs)
    941 
    942         return output

[/usr/local/lib/python3.10/dist-packages/torch/_subclasses/fake_tensor.py](https://localhost:8080/#) in _dispatch_impl(self, func, types, args, kwargs)
   1422         try:
   1423             with in_kernel_invocation_manager(self):
-> 1424                 r = func(*args, **kwargs)
   1425         except NotImplementedError as not_implemented_error:
   1426             return maybe_run_unsafe_fallback(not_implemented_error)

[/usr/local/lib/python3.10/dist-packages/torch/_ops.py](https://localhost:8080/#) in __call__(self_, *args, **kwargs)
    598         # use `self_` to avoid naming collide with aten ops arguments that
    599         # are named "self". This way, all the aten ops can be called by kwargs.
--> 600         return self_._op(*args, **kwargs)
    601 
    602     def __hash__(self):

TorchRuntimeError: Failed running call_function <built-in method gather of type object at 0x7d5191f98460>(*(FakeTensor(..., size=(1, s0)), 1, FakeTensor(..., size=(1, 1), dtype=torch.int64)), **{'sparse_grad': False, 'out': FakeTensor(..., size=(1, 1))}):
Cannot call numel() on tensor with symbolic sizes/strides
Exception raised from throw_cannot_call_with_symbolic at ../c10/core/TensorImpl.cpp:298 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x57 (0x7d5192e8e3b7 in /usr/local/lib/python3.10/dist-packages/torch/lib/libc10.so)
frame #1: c10::TensorImpl::throw_cannot_call_with_symbolic(char const*) const + 0x9c (0x7d5192e3ca1c in /usr/local/lib/python3.10/dist-packages/torch/lib/libc10.so)
frame #2: <unknown function> + 0x626ff (0x7d5192e696ff in /usr/local/lib/python3.10/dist-packages/torch/lib/libc10.so)
frame #3: at::get_overlap_status(c10::TensorImpl const*, c10::TensorImpl const*) + 0x50a (0x7d516e32069a in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #4: at::assert_no_overlap(c10::TensorImpl*, c10::TensorImpl*) + 0x9 (0x7d516e320a19 in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #5: at::meta::structured_gather::meta(at::Tensor const&, long, at::Tensor const&, bool) + 0xd1 (0x7d516e9cc1a1 in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #6: <unknown function> + 0x34ae86e (0x7d516fe4486e in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #7: <unknown function> + 0x352e8fc (0x7d516fec48fc in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #8: <unknown function> + 0x7bbdbe (0x7d5191608dbe in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_python.so)
frame #9: <unknown function> + 0x1b7f827 (0x7d516e515827 in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #10: c10::Dispatcher::callBoxed(c10::OperatorHandle const&, std::vector<c10::IValue, std::allocator<c10::IValue> >*) const + 0x238 (0x7d5191aae7a8 in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_python.so)
frame #11: torch::jit::invokeOperatorFromPython(std::vector<std::shared_ptr<torch::jit::Operator>, std::allocator<std::shared_ptr<torch::jit::Operator> > > const&, pybind11::args, pybind11::kwargs const&, std::optional<c10::DispatchKey>) + 0x1c1 (0x7d5191844d81 in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_python.so)
frame #12: torch::jit::_get_operation_for_overload_or_packet(std::vector<std::shared_ptr<torch::jit::Operator>, std::allocator<std::shared_ptr<torch::jit::Operator> > > const&, c10::Symbol, pybind11::args, pybind11::kwargs const&, bool, std::optional<c10::DispatchKey>) + 0x531 (0x7d5191845461 in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_python.so)
frame #13: <unknown function> + 0x8e1140 (0x7d519172e140 in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_python.so)
frame #14: <unknown function> + 0x48bc84 (0x7d51912d8c84 in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_python.so)
frame #15: <unknown function> + 0x15a10e (0x57ac2291210e in /usr/bin/python3)
frame #16: PyObject_Call + 0xbb (0x57ac2292142b in /usr/bin/python3)
frame #17: _PyEval_EvalFrameDefault + 0x2a27 (0x57ac228fd5d7 in /usr/bin/python3)
frame #18: _PyFunction_Vectorcall + 0x7c (0x57ac229129fc in /usr/bin/python3)
frame #19: _PyObject_FastCallDictTstate + 0x16d (0x57ac22907cbd in /usr/bin/python3)
frame #20: _PyObject_Call_Prepend + 0x5c (0x57ac2291d86c in /usr/bin/python3)
frame #21: <unknown function> + 0x280700 (0x57ac22a38700 in /usr/bin/python3)
frame #22: PyObject_Call + 0xbb (0x57ac2292142b in /usr/bin/python3)
frame #23: _PyEval_EvalFrameDefault + 0x2a27 (0x57ac228fd5d7 in /usr/bin/python3)
frame #24: _PyFunction_Vectorcall + 0x7c (0x57ac229129fc in /usr/bin/python3)
frame #25: _PyEval_EvalFrameDefault + 0x8ac (0x57ac228fb45c in /usr/bin/python3)
frame #26: _PyFunction_Vectorcall + 0x7c (0x57ac229129fc in /usr/bin/python3)
frame #27: _PyEval_EvalFrameDefault + 0x8ac (0x57ac228fb45c in /usr/bin/python3)
frame #28: _PyFunction_Vectorcall + 0x7c (0x57ac229129fc in /usr/bin/python3)
frame #29: _PyEval_EvalFrameDefault + 0x8ac (0x57ac228fb45c in /usr/bin/python3)
frame #30: _PyFunction_Vectorcall + 0x7c (0x57ac229129fc in /usr/bin/python3)
frame #31: _PyEval_EvalFrameDefault + 0x2a27 (0x57ac228fd5d7 in /usr/bin/python3)
frame #32: <unknown function> + 0x16893e (0x57ac2292093e in /usr/bin/python3)
frame #33: <unknown function> + 0x15e3f9 (0x57ac229163f9 in /usr/bin/python3)
frame #34: PyObject_CallMethod + 0xc8 (0x57ac229eea48 in /usr/bin/python3)
frame #35: torch::handle_torch_function_no_python_arg_parser(c10::ArrayRef<_object*>, _object*, _object*, char const*, _object*, char const*, torch::TorchFunctionName) + 0x3de (0x7d5191a80f8e in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_python.so)
frame #36: <unknown function> + 0x7b798c (0x7d519160498c in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_python.so)
frame #37: <unknown function> + 0x1b7ff07 (0x7d516e515f07 in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #38: <unknown function> + 0x7bbdbe (0x7d5191608dbe in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_python.so)
frame #39: <unknown function> + 0x1b7f827 (0x7d516e515827 in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #40: at::_ops::gather_out::redispatch(c10::DispatchKeySet, at::Tensor const&, long, at::Tensor const&, bool, at::Tensor&) + 0x19b (0x7d516f42678b in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #41: <unknown function> + 0x4fa4f2e (0x7d517193af2e in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #42: <unknown function> + 0x7bbdbe (0x7d5191608dbe in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_python.so)
frame #43: <unknown function> + 0x1b7f827 (0x7d516e515827 in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #44: at::_ops::gather_out::redispatch(c10::DispatchKeySet, at::Tensor const&, long, at::Tensor const&, bool, at::Tensor&) + 0x19b (0x7d516f42678b in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #45: <unknown function> + 0x48668d5 (0x7d51711fc8d5 in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #46: <unknown function> + 0x4866b3e (0x7d51711fcb3e in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #47: <unknown function> + 0x7bbdbe (0x7d5191608dbe in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_python.so)
frame #48: <unknown function> + 0x1b7f827 (0x7d516e515827 in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #49: <unknown function> + 0x1b7fd17 (0x7d516e515d17 in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #50: <unknown function> + 0x7bbdbe (0x7d5191608dbe in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_python.so)
frame #51: <unknown function> + 0x1b7f827 (0x7d516e515827 in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #52: at::_ops::gather_out::call(at::Tensor const&, long, at::Tensor const&, bool, at::Tensor&) + 0x3bb (0x7d516f4b5a2b in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_cpu.so)
frame #53: <unknown function> + 0x66e960 (0x7d51914bb960 in /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch_python.so)
frame #54: <unknown function> + 0x15a10e (0x57ac2291210e in /usr/bin/python3)
frame #55: PyObject_Call + 0xbb (0x57ac2292142b in /usr/bin/python3)
frame #56: _PyEval_EvalFrameDefault + 0x2a27 (0x57ac228fd5d7 in /usr/bin/python3)
frame #57: _PyFunction_Vectorcall + 0x7c (0x57ac229129fc in /usr/bin/python3)
frame #58: _PyEval_EvalFrameDefault + 0x6bd (0x57ac228fb26d in /usr/bin/python3)
frame #59: _PyFunction_Vectorcall + 0x7c (0x57ac229129fc in /usr/bin/python3)
frame #60: _PyEval_EvalFrameDefault + 0x6bd (0x57ac228fb26d in /usr/bin/python3)
frame #61: _PyFunction_Vectorcall + 0x7c (0x57ac229129fc in /usr/bin/python3)
frame #62: _PyEval_EvalFrameDefault + 0x6bd (0x57ac228fb26d in /usr/bin/python3)
frame #63: _PyFunction_Vectorcall + 0x7c (0x57ac229129fc in /usr/bin/python3)


from user code:
   File "<ipython-input-5-b4bd9d5a4e80>", line 2, in gather_func
    torch.gather(input, 1, index, sparse_grad=False, out=out)

Set TORCH_LOGS="+dynamo" and TORCHDYNAMO_VERBOSE=1 for more information


You can suppress this exception and fall back to eager by setting:
    import torch._dynamo
    torch._dynamo.config.suppress_errors = True

Minified repro

def gather_func(input, index, out):
  torch.gather(input, 1, index, sparse_grad=False, out=out)

out = torch.empty(1, 1)

opt_model = torch.compile(gather_func)

inp1 = torch.randn(1, 2)
ind1 = torch.randint(2, (1, 1))
opt_model(inp1, ind1, out)

inp2 = torch.randn(1, 3)
ind2 = torch.randint(2, (1, 1))
opt_model(inp2, ind2, out)

Versions

PyTorch version: 2.4.0.dev20240326+cpu
Is debug build: False
CUDA used to build PyTorch: Could not collect
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.3 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: 14.0.0-1ubuntu1.1
CMake version: version 3.27.9
Libc version: glibc-2.35

Python version: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] (64-bit runtime)
Python platform: Linux-6.1.58+-x86_64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: 12.2.140
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: Could not collect
Nvidia driver version: Could not collect
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.9.6
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.9.6
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.9.6
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.9.6
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.9.6
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.9.6
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.9.6
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                       x86_64
CPU op-mode(s):                     32-bit, 64-bit
Address sizes:                      46 bits physical, 48 bits virtual
Byte Order:                         Little Endian
CPU(s):                             2
On-line CPU(s) list:                0,1
Vendor ID:                          GenuineIntel
Model name:                         Intel(R) Xeon(R) CPU @ 2.20GHz
CPU family:                         6
Model:                              79
Thread(s) per core:                 2
Core(s) per socket:                 1
Socket(s):                          1
Stepping:                           0
BogoMIPS:                           4399.99
Flags:                              fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm rdseed adx smap xsaveopt arat md_clear arch_capabilities
Hypervisor vendor:                  KVM
Virtualization type:                full
L1d cache:                          32 KiB (1 instance)
L1i cache:                          32 KiB (1 instance)
L2 cache:                           256 KiB (1 instance)
L3 cache:                           55 MiB (1 instance)
NUMA node(s):                       1
NUMA node0 CPU(s):                  0,1
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit:        Not affected
Vulnerability L1tf:                 Mitigation; PTE Inversion
Vulnerability Mds:                  Vulnerable; SMT Host state unknown
Vulnerability Meltdown:             Vulnerable
Vulnerability Mmio stale data:      Vulnerable
Vulnerability Retbleed:             Vulnerable
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass:    Vulnerable
Vulnerability Spectre v1:           Vulnerable: __user pointer sanitization and usercopy barriers only; no swapgs barriers
Vulnerability Spectre v2:           Vulnerable, IBPB: disabled, STIBP: disabled, PBRSB-eIBRS: Not affected
Vulnerability Srbds:                Not affected
Vulnerability Tsx async abort:      Vulnerable

Versions of relevant libraries:
[pip3] numpy==1.25.2
[pip3] torch==2.4.0.dev20240326+cpu
[pip3] torchaudio==2.2.1+cu121
[pip3] torchdata==0.7.1
[pip3] torchsummary==1.5.1
[pip3] torchtext==0.17.1
[pip3] torchvision==0.17.1+cu121
[pip3] triton==2.2.0
[conda] Could not collect

cc @ezyang @msaroufim @bdhirsh @anijain2305 @zou3519 @chauhang
@yf225 yf225 added the triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module label Mar 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
module: dynamic shapes oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ezyang @yf225 @rahulvijayaraghavan