Accessing the device attribute of bias terms of TransformerEncoderLayer initialized with bias = False causes Attribute error

[beekill95]

🐛 Describe the bug

When performing a forward pass on TransformerEncoderLayer initialized with an even number of heads (2, 4, 6, ...) and without bias (bias=False) in eval mode (model.eval()), I face with an error AttributeError: 'NoneType' object has no attribute 'device'.

Here is the stack trace:

../lib/python3.8/site-packages/torch/nn/modules/module.py:1518: in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
../lib/python3.8/site-packages/torch/nn/modules/module.py:1527: in _call_impl
    return forward_call(*args, **kwargs)
../lib/python3.8/site-packages/torch/nn/modules/transformer.py:678: in forward
    elif not all((x.device.type in _supported_device_type) for x in tensor_args):

Upon looking at the relevant code piece, I think the following lines will create the error because the bias terms are None and this line accesses the device attribute of None objects and thus the error.

Versions

Collecting environment information...
PyTorch version: 2.1.2+cpu
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.2 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: Could not collect
Libc version: glibc-2.35

Python version: 3.8.18 | packaged by conda-forge | (default, Oct 10 2023, 15:44:36)  [GCC 12.3.0] (64-bit runtime)
Python platform: Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with-glibc2.10
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Address sizes:                   39 bits physical, 48 bits virtual
Byte Order:                      Little Endian
CPU(s):                          8
On-line CPU(s) list:             0-7
Vendor ID:                       GenuineIntel
Model name:                      11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
CPU family:                      6
Model:                           140
Thread(s) per core:              2
Core(s) per socket:              4
Socket(s):                       1
Stepping:                        1
BogoMIPS:                        5606.41
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx512vbmi umip avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid movdiri movdir64b fsrm avx512_vp2intersect md_clear flush_l1d arch_capabilities
Virtualization:                  VT-x
Hypervisor vendor:               Microsoft
Virtualization type:             full
L1d cache:                       192 KiB (4 instances)
L1i cache:                       128 KiB (4 instances)
L2 cache:                        5 MiB (4 instances)
L3 cache:                        12 MiB (1 instance)
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1:        Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:        Mitigation; Enhanced IBRS, IBPB conditional, RSB filling
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected

Versions of relevant libraries:
[pip3] mypy==0.971
[pip3] mypy-extensions==1.0.0
[pip3] numpy==1.23.5
[pip3] torch==2.1.2+cpu
[pip3] torch_geometric==2.4.0
[pip3] torch-training-loop==0.1.3
[pip3] torcheval==0.0.7
[pip3] torchvision==0.16.2
[conda] numpy                     1.23.5                   pypi_0    pypi
[conda] torch                     2.1.2+cpu                pypi_0    pypi
[conda] torch-geometric           2.4.0                    pypi_0    pypi
[conda] torch-training-loop       0.1.3                    pypi_0    pypi
[conda] torcheval                 0.0.7                    pypi_0    pypi
[conda] torchvision               0.16.2                   pypi_0    pypi

cc @jbschlosser @bhosmer @cpuhrsch @erichan1 @drisspg @mikaylagawarecki

[beekill95]

These lines may be troublesome as well because they access the bias terms, which might or might not exist depending on whether the layer initialized with bias or not.

[ayush0x00]

Hey @beekill95 can you provide code snippet where you are getting the error?

[tringwald]

Seems to be a duplicate of #116385.

[beekill95]

I just checked that the problem is fixed with the latest torch version. Feel free to close this issue.