Unknown layout while initializing tensors

[sssrijan-amazon]

🐛 Describe the bug

I am using the C++ Distributions of PyTorch 2.2.0 and following the setup and example from here

I am getting the following exception when running the code -

#include <torch/torch.h>
#include <iostream>
int main() {
    std::cout << torch::rand({2, 3});
}

terminate called after throwing an instance of 'c10::Error'
  what():  Unknown layout
Exception raised from operator<< at ../c10/core/Layout.h:69 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x57 (0x7fa49bb4c817 in /home/sssrijan/srijan-p38/lib/python3.8/site-packages/torch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, char const*) + 0x68 (0x7fa49bafd108 in /home/sssrijan/srijan-p38/lib/python3.8/site-packages/torch/lib/libc10.so)
frame #2: <unknown function> + 0x18e09ba (0x7fa48614c9ba in /home/sssrijan/srijan-p38/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #3: <unknown function> + 0x18e639a (0x7fa48615239a in /home/sssrijan/srijan-p38/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #4: <unknown function> + 0x2baef34 (0x7fa48741af34 in /home/sssrijan/srijan-p38/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #5: at::_ops::rand::call(c10::ArrayRef<c10::SymInt>, std::optional<c10::ScalarType>, std::optional<c10::Layout>, std::optional<c10::Device>, std::optional<bool>) + 0x19f (0x7fa4870daa8f in /home/sssrijan/srijan-p38/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #6: ./main() [0x4071cd]
frame #7: ./main() [0x4073b4]
frame #8: ./main() [0x403abb]
frame #9: __libc_start_main + 0xea (0x7fa483f4813a in /lib64/libc.so.6)
frame #10: ./main() [0x40377a]

Tried to use other constructors of tensor and still it fails

torch::tensor(1)

terminate called after throwing an instance of 'c10::Error'
  what():  Unknown layout
Exception raised from operator<< at ../c10/core/Layout.h:69 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x57 (0x7fbee6548817 in /home/sssrijan/srijan-p38/lib/python3.8/site-packages/torch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, char const*) + 0x68 (0x7fbee64f9108 in /home/sssrijan/srijan-p38/lib/python3.8/site-packages/torch/lib/libc10.so)
frame #2: <unknown function> + 0x18e09ba (0x7fbed0b489ba in /home/sssrijan/srijan-p38/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #3: <unknown function> + 0x18e639a (0x7fbed0b4e39a in /home/sssrijan/srijan-p38/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #4: <unknown function> + 0x2bb203f (0x7fbed1e1a03f in /home/sssrijan/srijan-p38/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #5: at::_ops::scalar_tensor::call(c10::Scalar const&, std::optional<c10::ScalarType>, std::optional<c10::Layout>, std::optional<c10::Device>, std::optional<bool>) + 0x193 (0x7fbed1ad5743 in /home/sssrijan/srijan-p38/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so)
frame #6: ./main() [0x4085e7]
frame #7: ./main() [0x408a32]
frame #8: ./main() [0x4090da]
frame #9: ./main() [0x404202]
frame #10: __libc_start_main + 0xea (0x7fbece94413a in /lib64/libc.so.6)
frame #11: ./main() [0x403dba]

Similar setup works with torch< 2.2 but not with torch==2.2.0

Versions

Collecting environment information...
PyTorch version: 2.2.0+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Amazon Linux 2 (x86_64)
GCC version: (GCC) 7.3.1 20180712 (Red Hat 7.3.1-17)
Clang version: Could not collect
CMake version: version 2.8.12.2
Libc version: glibc-2.26

Python version: 3.8.18 (default, Feb 15 2024, 20:18:37) [GCC 7.3.1 20180712 (Red Hat 7.3.1-17)] (64-bit runtime)
Python platform: Linux-5.10.209-175.812.amzn2int.x86_64-x86_64-with-glibc2.2.5
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 96
On-line CPU(s) list: 0-95
Thread(s) per core: 2
Core(s) per socket: 24
Socket(s): 2
NUMA node(s): 2
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
Stepping: 7
CPU MHz: 3100.049
BogoMIPS: 4999.99
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 1024K
L3 cache: 36608K
NUMA node0 CPU(s): 0-23,48-71
NUMA node1 CPU(s): 24-47,72-95
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves ida arat pku ospke

Versions of relevant libraries:
[pip3] torch==2.2.0+cpu
[conda] Could not collect

cc @ezyang @gchanan @zou3519 @kadeng @seemethere @malfet @osalpekar @atalman @jbschlosser

[michael-person]

I am having the same trouble.

[malfet]

Hi priority to get a repro the issue and validate the claim that it worked in 2.1, but broken in 2.2

[michael-person]

I have no issue on torch 2.1

[atalman]

Worked for me using package:

https://download.pytorch.org/libtorch/cpu/libtorch-shared-with-deps-2.2.0%2Bcpu.zip
and nightly:
https://download.pytorch.org/libtorch/nightly/cpu/libtorch-shared-with-deps-latest.zip

as suggested in https://pytorch.org/get-started/locally/

cmake -DCMAKE_PREFIX_PATH=/data/home/atalman/temp33/libt/libtorch ..
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Torch: /data/home/atalman/temp33/libt/libtorch/lib/libtorch.so  
-- Configuring done
-- Generating done
-- Build files have been written to: /data/home/atalman/temp33/libt/example/build
(jan22) atalman@ip-10-200-84-248:~/temp33/libt/example/build$ cmake --build . --config Release
[ 50%] Building CXX object CMakeFiles/example.dir/example.cpp.o
[100%] Linking CXX executable example
[100%] Built target example
(jan22) atalman@ip-10-200-84-248:~/temp33/libt/example/build$ ls
CMakeCache.txt  CMakeFiles  Makefile  cmake_install.cmake  example
(jan22) atalman@ip-10-200-84-248:~/temp33/libt/example/build$ ls -las
total 412
  4 drwxrwxr-x 3 atalman atalman   6144 Feb 16 18:23 .
  4 drwxrwxr-x 3 atalman atalman   6144 Feb 16 18:21 ..
 16 -rw-rw-r-- 1 atalman atalman  15430 Feb 16 18:23 CMakeCache.txt
  4 drwxrwxr-x 6 atalman atalman   6144 Feb 16 18:23 CMakeFiles
  8 -rw-rw-r-- 1 atalman atalman   5431 Feb 16 18:23 Makefile
  4 -rw-rw-r-- 1 atalman atalman   1652 Feb 16 18:23 cmake_install.cmake
372 -rwxrwxr-x 1 atalman atalman 380576 Feb 16 18:23 example
(jan22) atalman@ip-10-200-84-248:~/temp33/libt/example/build$ ./example 
 0.6368  0.8858  0.9782
 0.3194  0.6896  0.4702
[ CPUFloatType{2,3} ]

[atalman]

@sssrijan-amazon @michael-person could you please confirm the package url you used in your test ?
For prod we have following packages:

Download here (Pre-cxx11 ABI):
https://download.pytorch.org/libtorch/cpu/libtorch-shared-with-deps-2.2.0%2Bcpu.zip
Download here (cxx11 ABI):
https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-2.2.0%2Bcpu.zip

[malfet]

I suspect key here is gcc version, which is gcc-7 as reported by @sssrijan-amazon , but starting with PyTorch-2.2 we are building everything with gcc-9, as the first compiler version with stable C++17 ABI, and I suspect this what might be happening thrre.

Trying to find a version of AmazonLinux with gcc-7 compiler installed by default

[huydhn]

I'm on one of the runner, gcc-7 is indeed the default on AmazonLinux 2

sudo yum install gcc-c++

============================================================================================================================================================================================================================================================================================================================
 Package                                                                       Arch                                                                  Version                                                                                Repository                                                                 Size
============================================================================================================================================================================================================================================================================================================================
Installing:
 gcc                                                                           x86_64                                                                7.3.1-17.amzn2                                                                         amzn2-core                                                                 22 M
 gcc-c++                                                                       x86_64                                                                7.3.1-17.amzn2                                                                         amzn2-core                                                                 13 M
Installing for dependencies:
 cpp                                                                           x86_64                                                                7.3.1-17.amzn2                                                                         amzn2-core                                                                9.2 M
 glibc-devel                                                                   x86_64                                                                2.26-63.amzn2.0.1                                                                      amzn2-core                                                                995 k
 glibc-headers                                                                 x86_64                                                                2.26-63.amzn2.0.1                                                                      amzn2-core                                                                516 k
 kernel-headers                                                                x86_64                                                                4.14.336-255.557.amzn2                                                                 amzn2-core                                                                1.2 M
 libatomic                                                                     x86_64                                                                7.3.1-17.amzn2                                                                         amzn2-core                                                                 46 k
 libcilkrts                                                                    x86_64                                                                7.3.1-17.amzn2                                                                         amzn2-core                                                                 85 k
 libitm                                                                        x86_64                                                                7.3.1-17.amzn2                                                                         amzn2-core                                                                 85 k
 libmpc                                                                        x86_64                                                                1.0.1-3.amzn2.0.2                                                                      amzn2-core                                                                 52 k
 libmpx                                                                        x86_64                                                                7.3.1-17.amzn2                                                                         amzn2-core                                                                 52 k
 libquadmath                                                                   x86_64                                                                7.3.1-17.amzn2                                                                         amzn2-core                                                                189 k
 libsanitizer                                                                  x86_64                                                                7.3.1-17.amzn2                                                                         amzn2-core                                                                642 k
 mpfr                                                                          x86_64                                                                3.1.1-4.amzn2.0.2                                                                      amzn2-core                                                                208 k

Transaction Summary
============================================================================================================================================================================================================================================================================================================================
Install  2 Packages (+12 Dependent packages)

and the crash can be reproduced with gcc-7

$ CC=/usr/bin/gcc-7 CXX=/usr/bin/g++-7 cmake -DCMAKE_PREFIX_PATH=/tmp/debug-120020/libtorch ..
$ cmake --build . --config Release
$ ./example-app
terminate called after throwing an instance of 'c10::Error'
  what():  Unknown layout
Exception raised from operator<< at ../c10/core/Layout.h:70 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x57 (0x7fd51880a757 in /tmp/debug-120020/libtorch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, char const*) + 0x68 (0x7fd5187baf48 in /tmp/debug-120020/libtorch/lib/libc10.so)
frame #2: <unknown function> + 0x190747a (0x7fd502c9d47a in /tmp/debug-120020/libtorch/lib/libtorch_cpu.so)
frame #3: <unknown function> + 0x190cd1a (0x7fd502ca2d1a in /tmp/debug-120020/libtorch/lib/libtorch_cpu.so)
frame #4: <unknown function> + 0x2bfdee4 (0x7fd503f93ee4 in /tmp/debug-120020/libtorch/lib/libtorch_cpu.so)
frame #5: at::_ops::rand::call(c10::ArrayRef<c10::SymInt>, std::optional<c10::ScalarType>, std::optional<c10::Layout>, std::optional<c10::Device>, std::optional<bool>) + 0x19f (0x7fd503c49c7f in /tmp/debug-120020/libtorch/lib/libtorch_cpu.so)
frame #6: <unknown function> + 0x8665 (0x5557d4584665 in ./example-app)
frame #7: <unknown function> + 0x88c1 (0x5557d45848c1 in ./example-app)
frame #8: <unknown function> + 0x4755 (0x5557d4580755 in ./example-app)
frame #9: __libc_start_main + 0xf3 (0x7fd500fb5083 in /lib/x86_64-linux-gnu/libc.so.6)
frame #10: <unknown function> + 0x439e (0x5557d458039e in ./example-app)

Aborted (core dumped)

So the fix would be to upgrade gcc to 9 or above.

[michael-person]

Ah, that would do it. I am on a Amazon Linux 2 AMI w/ gcc-7

[malfet]

Ok, let me add a check to one of the header to raise an explicit error if GCC major is less than 9
Removing high priority, as this BC breaking change was already documented in 2.2 release notes, but wasn't clearly articulated as a link-time requirement as well

[malfet]

@michael-person aren't newer toolchain available to you?

[michael-person]

@malfet they probably are but unfortunately are not easily accessible for me atm. I'm not sure how @sssrijan-amazon feels but at least for me, just knowing that my environment need to be updated is enough of a solution

[sssrijan-amazon]

Thanks a lot for the updates. I will try out with the correct gcc versions.

[michael-person]

So I was able to find GCC 10.5.0 and recompiled the example code and am still getting the "unknown layout". I'm not sure how much this matters but I realized that I was also on an old Python version (3.8.8 compiled against GCC 7.3) and tried updating to a newer Python version (3.9.18 compiled against GCC 10.5) and am still seeing the "unknown layout" error. The only other thing that sticks out to me is that I am using Cuda 12.2 but PyTorch 2.2 was compiled against Cuda 12.1, would that maybe cause an issue?

[PaliC]

Validated we no longer support gcc < 9 for release 2.2.2

/home/ec2-user/libtorch/include/c10/util/C++17.h:13:2: error: #error "You're trying to build PyTorch with a too old version of GCC. We need GCC 9 or later."
 #error \
  ^~~~~