Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Could not debug remote code with dataloaders which has num_workers>0 #818

Closed
1 task done
quancs opened this issue Dec 19, 2021 · 4 comments
Closed
1 task done

Could not debug remote code with dataloaders which has num_workers>0 #818

quancs opened this issue Dec 19, 2021 · 4 comments
Labels
waiting for response

Comments

@quancs
Copy link

quancs commented Dec 19, 2021

VS Code version

1.63.2

Extension version

v2021.12.1559732655

OS type

Windows

OS version

local win11, remote centos 7

Python distribution

Anaconda

Python version

3.8.5

Language server

Default

Expected behaviour

No error no matter debug or run

Actual behaviour

Error when debug in vscode: double free or corruption. No error when just run python boring.py

Steps to reproduce

  1. save boring.py on remote machine
import torch
from torch.utils.data import DataLoader, Dataset


class RandomDataset(Dataset):

    def __init__(self, size, length):
        self.len = length
        self.data = torch.randn(length, size)

    def __getitem__(self, index):
        return self.data[index]

    def __len__(self):
        return self.len


if __name__ == "__main__":
    train_data = DataLoader(RandomDataset(32, 64), batch_size=2, num_workers=2)

    for x in train_data:
        print(1)
  1. debug boring.py in vscode with remote mode

Logs

用户属于 'pythonaa' 实验组
用户属于 'pythonTensorboardExperiment' 实验组
用户属于 'PythonPyTorchProfiler' 实验组
用户属于 'pythonDeprecatePythonPath' 实验组
> conda info --json
> /home/quancs/miniconda3/bin/python ~/.vscode-server/extensions/ms-python.python-2021.12.1559732655/pythonFiles/interpreterInfo.py
Python 解释器路径: /home/quancs/miniconda3/bin/python
> /home/quancs/miniconda3/envs/metrics/bin/python ~/.vscode-server/extensions/ms-python.python-2021.12.1559732655/pythonFiles/interpreterInfo.py
> /home/quancs/miniconda3/envs/torch/bin/python ~/.vscode-server/extensions/ms-python.python-2021.12.1559732655/pythonFiles/interpreterInfo.py
> /bin/python ~/.vscode-server/extensions/ms-python.python-2021.12.1559732655/pythonFiles/interpreterInfo.py
> ". /home/quancs/miniconda3/bin/activate && conda activate base && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python /mnt/home/quancs/.vscode-server/extensions/ms-python.python-2021.12.1559732655/pythonFiles/printEnvVariables.py"
> ". /home/quancs/miniconda3/bin/activate && conda activate base && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python /mnt/home/quancs/.vscode-server/extensions/ms-python.python-2021.12.1559732655/pythonFiles/printEnvVariables.py"
> ". /home/quancs/miniconda3/bin/activate && conda activate base && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python /mnt/home/quancs/.vscode-server/extensions/ms-python.python-2021.12.1559732655/pythonFiles/printEnvVariables.py"
> ". /home/quancs/miniconda3/bin/activate && conda activate base && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python /mnt/home/quancs/.vscode-server/extensions/ms-python.python-2021.12.1559732655/pythonFiles/printEnvVariables.py"
> ". /home/quancs/miniconda3/bin/activate && conda activate base && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python /mnt/home/quancs/.vscode-server/extensions/ms-python.python-2021.12.1559732655/pythonFiles/printEnvVariables.py"
> ". /home/quancs/miniconda3/bin/activate && conda activate base && echo 'e8b39361-0157-4923-80e1-22d70d46dee6' && python /mnt/home/quancs/.vscode-server/extensions/ms-python.python-2021.12.1559732655/pythonFiles/printEnvVariables.py"
> /usr/bin/python ~/.vscode-server/extensions/ms-python.python-2021.12.1559732655/pythonFiles/interpreterInfo.py
正在启动 Pylance 语言服务器。
> /home/quancs/miniconda3/bin/python -m mypy --follow-imports=silent --ignore-missing-imports --show-column-numbers --no-pretty ~/projects/NBSS_pmt/boring.py
cwd: ~/projects/NBSS_pmt
##########Linting Output - mypy##########
Success: no issues found in 1 source file

Code of Conduct

  • I agree to follow this project's Code of Conduct
@brettcannon brettcannon changed the title Dec 20, 2021
@brettcannon
Copy link
Member

Can you provide your launch.json and the output of the debugging failure?

@quancs
Copy link
Author

quancs commented Dec 21, 2021

yes. @brettcannon

launch.json

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "justMyCode": true,
        },
    ]
}

Console output:

(base) ➜  NBSS_pmt git:(master) ✗  cd /mnt/home/quancs/projects/NBSS_pmt ; /usr/bin/env /home/quancs/miniconda3/bin/python /mnt/home/quancs/.vscode-server/extensions/ms-python.python-2021.12.1559732655/pythonFiles/lib/python/debugpy/launcher 38695 -- /mnt/home/quancs/projects/NBSS_pmt/boring.py 
*** Error in `/home/quancs/miniconda3/bin/python': double free or corruption (!prev): 0x000055f3db4fea60 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7340f)[0x2b4dc3f6540f]
/lib64/libc.so.6(+0x78c7e)[0x2b4dc3f6ac7e]
/lib64/libc.so.6(+0x79957)[0x2b4dc3f6b957]
/lib64/ld-linux-x86-64.so.2(_dl_deallocate_tls+0x58)[0x2b4dc3ac25a8]
/lib64/libpthread.so.0(+0x7237)[0x2b4dc3cdb237]
/lib64/libpthread.so.0(+0x734f)[0x2b4dc3cdb34f]
/lib64/libpthread.so.0(pthread_join+0xdb)[0x2b4dc3cdd69b]
/home/quancs/miniconda3/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so(pthreadpool_destroy+0x8b)[0x2b4e8cf2a37b]
/home/quancs/miniconda3/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so(+0x251b827)[0x2b4e8aba4827]
/home/quancs/miniconda3/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so(_ZN2at15set_num_threadsEi+0x38)[0x2b4e8957cf28]
/home/quancs/miniconda3/lib/python3.8/site-packages/torch/lib/libtorch_python.so(+0x99e3e6)[0x2b4e155583e6]
/home/quancs/miniconda3/bin/python(+0x140825)[0x55f3d9ebb825]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0xffa68)[0x55f3d9e7aa68]
/home/quancs/miniconda3/bin/python(_PyFunction_Vectorcall+0x10b)[0x55f3d9f0686b]
/home/quancs/miniconda3/bin/python(PyVectorcall_Call+0x71)[0x55f3d9eb8041]
/home/quancs/miniconda3/bin/python(_PyEval_EvalFrameDefault+0x1fdb)[0x55f3d9f3d99b]
/home/quancs/miniconda3/bin/python(_PyFunction_Vectorcall+0x10b)[0x55f3d9f0686b]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0x100066)[0x55f3d9e7b066]
/home/quancs/miniconda3/bin/python(_PyEval_EvalCodeWithName+0x2d2)[0x55f3d9f05a92]
/home/quancs/miniconda3/bin/python(+0x18bd20)[0x55f3d9f06d20]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0x1001de)[0x55f3d9e7b1de]
/home/quancs/miniconda3/bin/python(_PyFunction_Vectorcall+0x10b)[0x55f3d9f0686b]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0x100066)[0x55f3d9e7b066]
/home/quancs/miniconda3/bin/python(_PyObject_FastCallDict+0x38d)[0x55f3d9f0760d]
/home/quancs/miniconda3/bin/python(_PyObject_Call_Prepend+0x63)[0x55f3d9f07733]
/home/quancs/miniconda3/bin/python(+0x18c8ca)[0x55f3d9f078ca]
/home/quancs/miniconda3/bin/python(_PyObject_MakeTpCall+0x1a4)[0x55f3d9eb87d4]
/home/quancs/miniconda3/bin/python(+0xffee1)[0x55f3d9e7aee1]
/home/quancs/miniconda3/bin/python(_PyFunction_Vectorcall+0x10b)[0x55f3d9f0686b]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0xffa68)[0x55f3d9e7aa68]
/home/quancs/miniconda3/bin/python(_PyFunction_Vectorcall+0x10b)[0x55f3d9f0686b]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0xffa68)[0x55f3d9e7aa68]
/home/quancs/miniconda3/bin/python(_PyFunction_Vectorcall+0x10b)[0x55f3d9f0686b]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0x100066)[0x55f3d9e7b066]
/home/quancs/miniconda3/bin/python(_PyEval_EvalCodeWithName+0x659)[0x55f3d9f05e19]
/home/quancs/miniconda3/bin/python(_PyObject_FastCallDict+0x20c)[0x55f3d9f0748c]
/home/quancs/miniconda3/bin/python(_PyObject_Call_Prepend+0x63)[0x55f3d9f07733]
/home/quancs/miniconda3/bin/python(+0x18c8ca)[0x55f3d9f078ca]
/home/quancs/miniconda3/bin/python(_PyObject_MakeTpCall+0x1a4)[0x55f3d9eb87d4]
/home/quancs/miniconda3/bin/python(+0xffee1)[0x55f3d9e7aee1]
/home/quancs/miniconda3/bin/python(_PyFunction_Vectorcall+0x10b)[0x55f3d9f0686b]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0x100066)[0x55f3d9e7b066]
/home/quancs/miniconda3/bin/python(_PyFunction_Vectorcall+0x10b)[0x55f3d9f0686b]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0x13d9ef)[0x55f3d9eb89ef]
/home/quancs/miniconda3/bin/python(+0x20958a)[0x55f3d9f8458a]
/home/quancs/miniconda3/bin/python(PyObject_GetIter+0x16)[0x55f3d9ebe376]
/home/quancs/miniconda3/bin/python(_PyEval_EvalFrameDefault+0x165a)[0x55f3d9f3d01a]
/home/quancs/miniconda3/bin/python(_PyEval_EvalCodeWithName+0x2d2)[0x55f3d9f05a92]
/home/quancs/miniconda3/bin/python(PyEval_EvalCodeEx+0x44)[0x55f3d9f06754]
/home/quancs/miniconda3/bin/python(PyEval_EvalCode+0x1c)[0x55f3d9f94edc]
/home/quancs/miniconda3/bin/python(+0x24f083)[0x55f3d9fca083]
/home/quancs/miniconda3/bin/python(+0x140699)[0x55f3d9ebb699]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0xffee1)[0x55f3d9e7aee1]
======= Memory map: ========
2b4dc3ab1000-2b4dc3ad1000 r-xp 00000000 08:04 13680862                   /usr/lib64/ld-2.18.so
2b4dc3ad1000-2b4dc3ad2000 rw-p 00000000 00:00 0 
2b4dc3ad2000-2b4dc3ad9000 r--s 00000000 08:04 14024983                   /usr/lib64/gconv/gconv-modules.cache
2b4dc3ad9000-2b4dc3adc000 r--p 00000000 fd:00 118098547                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_struct.cpython-38-x86_64-linux-gnu.so
2b4dc3adc000-2b4dc3ae2000 r-xp 00003000 fd:00 118098547                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_struct.cpython-38-x86_64-linux-gnu.so
2b4dc3ae2000-2b4dc3ae6000 r--p 00009000 fd:00 118098547                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_struct.cpython-38-x86_64-linux-gnu.so
2b4dc3ae6000-2b4dc3ae7000 r--p 0000c000 fd:00 118098547                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_struct.cpython-38-x86_64-linux-gnu.so
2b4dc3ae7000-2b4dc3ae8000 rw-p 0000d000 fd:00 118098547                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_struct.cpython-38-x86_64-linux-gnu.so
2b4dc3ae8000-2b4dc3aec000 r--p 00000000 fd:00 118098562                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so
2b4dc3aec000-2b4dc3af4000 r-xp 00004000 fd:00 118098562                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so
2b4dc3af4000-2b4dc3af8000 r--p 0000c000 fd:00 118098562                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so
2b4dc3af8000-2b4dc3af9000 r--p 0000f000 fd:00 118098562                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so
2b4dc3af9000-2b4dc3afa000 rw-p 00010000 fd:00 118098562                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so
2b4dc3afa000-2b4dc3afb000 rw-s 00000000 00:13 252171475                  /dev/shm/sem.N38K6v (deleted)
2b4dc3afb000-2b4dc3afc000 rw-s 00000000 00:13 252171476                  /dev/shm/sem.1j3uZn (deleted)
2b4dc3afc000-2b4dc3afd000 rw-s 00000000 00:13 252171477                  /dev/shm/sem.ZiTgSf (deleted)
2b4dc3afd000-2b4dc3afe000 rw-s 00000000 00:13 252171478                  /dev/shm/sem.Tyy5K7 (deleted)
2b4dc3afe000-2b4dc3aff000 r--p 00000000 fd:00 118098526                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_heapq.cpython-38-x86_64-linux-gnu.so
2b4dc3aff000-2b4dc3b02000 r-xp 00001000 fd:00 118098526                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_heapq.cpython-38-x86_64-linux-gnu.so
2b4dc3b02000-2b4dc3b05000 r--p 00004000 fd:00 118098526                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_heapq.cpython-38-x86_64-linux-gnu.so
2b4dc3b05000-2b4dc3b06000 r--p 00006000 fd:00 118098526                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_heapq.cpython-38-x86_64-linux-gnu.so
2b4dc3b06000-2b4dc3b07000 rw-p 00007000 fd:00 118098526                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_heapq.cpython-38-x86_64-linux-gnu.so
2b4dc3b07000-2b4dc3b08000 r--p 00000000 fd:00 118098533                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_opcode.cpython-38-x86_64-linux-gnu.so
2b4dc3b08000-2b4dc3b09000 r-xp 00001000 fd:00 118098533                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_opcode.cpython-38-x86_64-linux-gnu.so
2b4dc3b09000-2b4dc3b0a000 r--p 00002000 fd:00 118098533                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_opcode.cpython-38-x86_64-linux-gnu.so
2b4dc3b0a000-2b4dc3b0b000 r--p 00002000 fd:00 118098533                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_opcode.cpython-38-x86_64-linux-gnu.so
2b4dc3b0b000-2b4dc3b0c000 rw-p 00003000 fd:00 118098533                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_opcode.cpython-38-x86_64-linux-gnu.so
2b4dc3b0c000-2b4dc3b0e000 r--p 00000000 fd:00 118098537                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_queue.cpython-38-x86_64-linux-gnu.so
2b4dc3b0e000-2b4dc3b0f000 r-xp 00002000 fd:00 118098537                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_queue.cpython-38-x86_64-linux-gnu.so
2b4dc3b0f000-2b4dc3b10000 r--p 00003000 fd:00 118098537                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_queue.cpython-38-x86_64-linux-gnu.so
2b4dc3b10000-2b4dc3b11000 r--p 00003000 fd:00 118098537                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_queue.cpython-38-x86_64-linux-gnu.so
2b4dc3b11000-2b4dc3b12000 rw-p 00004000 fd:00 118098537                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_queue.cpython-38-x86_64-linux-gnu.so
2b4dc3b12000-2b4dc3b13000 rw-s 00000000 00:13 252171479                  /dev/shm/sem.eYHWDZ (deleted)
2b4dc3b13000-2b4dc3b14000 rw-s 00000000 00:13 252171480                  /dev/shm/sem.LooPwR (deleted)
2b4dc3b14000-2b4dc3c98000 rw-p 00000000 00:00 0 
2b4dc3c98000-2b4dc3c9a000 r--p 00000000 fd:00 118098527                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_json.cpython-38-x86_64-linux-gnu.so
2b4dc3c9a000-2b4dc3caa000 r-xp 00002000 fd:00 118098527                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_json.cpython-38-x86_64-linux-gnu.so
2b4dc3caa000-2b4dc3cac000 r--p 00012000 fd:00 118098527                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_json.cpython-38-x86_64-linux-gnu.so
2b4dc3cac000-2b4dc3cad000 r--p 00013000 fd:00 118098527                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_json.cpython-38-x86_64-linux-gnu.so
2b4dc3cad000-2b4dc3cae000 rw-p 00014000 fd:00 118098527                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_json.cpython-38-x86_64-linux-gnu.so
2b4dc3cae000-2b4dc3cb3000 r--p 00000000 fd:00 118098543                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_socket.cpython-38-x86_64-linux-gnu.so
2b4dc3cb3000-2b4dc3cbb000 r-xp 00005000 fd:00 118098543                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_socket.cpython-38-x86_64-linux-gnu.so
2b4dc3cbb000-2b4dc3cc3000 r--p 0000d000 fd:00 118098543                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_socket.cpython-38-x86_64-linux-gnu.so
2b4dc3cc3000-2b4dc3cc4000 r--p 00014000 fd:00 118098543                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_socket.cpython-38-x86_64-linux-gnu.so
2b4dc3cc4000-2b4dc3cc5000 rw-p 00015000 fd:00 118098543                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_socket.cpython-38-x86_64-linux-gnu.so
2b4dc3cc5000-2b4dc3cc8000 r--p 00000000 fd:00 118098570                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/select.cpython-38-x86_64-linux-gnu.so
2b4dc3cc8000-2b4dc3ccb000 r-xp 00003000 fd:00 118098570                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/select.cpython-38-x86_64-linux-gnu.so
2b4dc3ccb000-2b4dc3ccd000 r--p 00006000 fd:00 118098570                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/select.cpython-38-x86_64-linux-gnu.so
2b4dc3ccd000-2b4dc3cce000 ---p 00008000 fd:00 118098570                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/select.cpython-38-x86_64-linux-gnu.so
2b4dc3cce000-2b4dc3ccf000 r--p 00008000 fd:00 118098570                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/select.cpython-38-x86_64-linux-gnu.so
2b4dc3ccf000-2b4dc3cd0000 rw-p 00009000 fd:00 118098570                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/select.cpython-38-x86_64-linux-gnu.so
2b4dc3cd0000-2b4dc3cd1000 rw-s 00000000 00:13 252171481                  /dev/shm/sem.hFXIpJ (deleted)
2b4dc3cd1000-2b4dc3cd2000 r--p 00020000 08:04 13680862                   /usr/lib64/ld-2.18.so
2b4dc3cd2000-2b4dc3cd3000 rw-p 00021000 08:04 13680862                   /usr/lib64/ld-2.18.so
2b4dc3cd3000-2b4dc3cd4000 rw-p 00000000 00:00 0 
2b4dc3cd4000-2b4dc3ced000 r-xp 00000000 08:04 13666408                   /usr/lib64/libpthread-2.18.so
2b4dc3ced000-2b4dc3eec000 ---p 00019000 08:04 13666408                   /usr/lib64/libpthread-2.18.so
2b4dc3eec000-2b4dc3eed000 r--p 00018000 08:04 13666408                   /usr/lib64/libpthread-2.18.so
2b4dc3eed000-2b4dc3eee000 rw-p 00019000 08:04 13666408                   /usr/lib64/libpthread-2.18.so
2b4dc3eee000-2b4dc3ef2000 rw-p 00000000 00:00 0 
2b4dc3ef2000-2b4dc4094000 r-xp 00000000 08:04 13636609                   /usr/lib64/libc-2.18.so
2b4dc4094000-2b4dc4294000 ---p 001a2000 08:04 13636609                   /usr/lib64/libc-2.18.so
2b4dc4294000-2b4dc4298000 r--p 001a2000 08:04 13636609                   /usr/lib64/libc-2.18.so
2b4dc4298000-2b4dc429a000 rw-p 001a6000 08:04 13636609                   /usr/lib64/libc-2.18.so
2b4dc429a000-2b4dc429e000 rw-p 00000000 00:00 0 
2b4dc429e000-2b4dc42a0000 r-xp 00000000 08:04 13666386                   /usr/lib64/libdl-2.18.so
2b4dc42a0000-2b4dc44a0000 ---p 00002000 08:04 13666386                   /usr/lib64/libdl-2.18.so
2b4dc44a0000-2b4dc44a1000 r--p 00002000 08:04 13666386                   /usr/lib64/libdl-2.18.so
2b4dc44a1000-2b4dc44a2000 rw-p 00003000 08:04 13666386                   /usr/lib64/libdl-2.18.so
2b4dc44a2000-2b4dc44a4000 r-xp 00000000 08:04 13680860                   /usr/lib64/libutil-2.18.so
2b4dc44a4000-2b4dc46a3000 ---p 00002000 08:04 13680860                   /usr/lib64/libutil-2.18.so
2b4dc46a3000-2b4dc46a4000 r--p 00001000 08:04 13680860                   /usr/lib64/libutil-2.18.so
2b4dc46a4000-2b4dc46a5000 rw-p 00002000 08:04 13680860                   /usr/lib64/libutil-2.18.so
2b4dc46a5000-2b4dc46ac000 r-xp 00000000 08:04 13680483                   /usr/lib64/librt-2.18.so
2b4dc46ac000-2b4dc48ab000 ---p 00007000 08:04 13680483                   /usr/lib64/librt-2.18.so
2b4dc48ab000-2b4dc48ac000 r--p 00006000 08:04 13680483                   /usr/lib64/librt-2.18.so
2b4dc48ac000-2b4dc48ad000 rw-p 00007000 08:04 13680483                   /usr/lib64/librt-2.18.so
2b4dc48ad000-2b4dc49ae000 r-xp 00000000 08:04 13666363                   /usr/lib64/libm-2.18.so
2b4dc49ae000-2b4dc4bad000 ---p 00101000 08:04 13666363                   /usr/lib64/libm-2.18.so
2b4dc4bad000-2b4dc4bae000 r--p 00100000 08:04 13666363                   /usr/lib64/libm-2.18.so
2b4dc4bae000-2b4dc4baf000 rw-p 00101000 08:04 13666363                   /usr/lib64/libm-2.18.so
2b4dc4baf000-2b4dcb0f2000 r--p 00000000 08:04 13639059                   /usr/lib/locale/locale-archive
2b4dcb0f2000-2b4dcb2b2000 rw-p 00000000 00:00 0 
2b4dcb2b2000-2b4dcb2b4000 r--p 00000000 fd:00 118098560                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/fcntl.cpython-38-x86_64-linux-gnu.so
2b4dcb2b4000-2b4dcb2b6000 r-xp 00002000 fd:00 118098560                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/fcntl.cpython-38-x86_64-linux-gnu.so
2b4dcb2b6000-2b4dcb2b8000 r--p 00004000 fd:00 118098560                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/fcntl.cpython-38-x86_64-linux-gnu.so
2b4dcb2b8000-2b4dcb2b9000 r--p 00005000 fd:00 118098560                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/fcntl.cpython-38-x86_64-linux-gnu.so
2b4dcb2b9000-2b4dcb2ba000 rw-p 00006000 fd:00 118098560                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/fcntl.cpython-38-x86_64-linux-gnu.so
2b4dcb2ba000-2b4dcb2bb000 rw-s 00000000 00:13 252171482                  /dev/shm/sem.A2iEiB (deleted)
2b4dcb2bb000-2b4dcb2fb000 rw-p 00000000 00:00 0 
2b4dcb2fb000-2b4dcb2fe000 r--p 00000000 fd:00 118098574                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/unicodedata.cpython-38-x86_64-linux-gnu.so
2b4dcb2fe000-2b4dcb304000 r-xp 00003000 fd:00 118098574                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/unicodedata.cpython-38-x86_64-linux-gnu.so
2b4dcb304000-2b4dcb406000 r--p 00009000 fd:00 118098574                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/unicodedata.cpython-38-x86_64-linux-gnu.so
2b4dcb406000-2b4dcb407000 r--p 0010a000 fd:00 118098574                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/unicodedata.cpython-38-x86_64-linux-gnu.so
2b4dcb407000-2b4dcb408000 rw-p 0010b000 fd:00 118098574                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/unicodedata.cpython-38-x86_64-linux-gnu.so
2b4dcb408000-2b4dcb488000 rw-p 00000000 00:00 0 
2b4dcb488000-2b4dcb48a000 r--p 00000000 fd:00 118098558                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/binascii.cpython-38-x86_64-linux-gnu.so
2b4dcb48a000-2b4dcb48f000 r-xp 00002000 fd:00 118098558                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/binascii.cpython-38-x86_64-linux-gnu.so
2b4dcb48f000-2b4dcb491000 r--p 00007000 fd:00 118098558                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/binascii.cpython-38-x86_64-linux-gnu.so
2b4dcb491000-2b4dcb492000 ---p 00009000 fd:00 118098558                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/binascii.cpython-38-x86_64-linux-gnu.so
2b4dcb492000-2b4dcb493000 r--p 00009000 fd:00 118098558                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/binascii.cpython-38-x86_64-linux-gnu.so
2b4dcb493000-2b4dcb494000 rw-p 0000a000 fd:00 118098558                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/binascii.cpython-38-x86_64-linux-gnu.so
2b4dcb494000-2b4dcb497000 r--p 00000000 fd:00 118628888                  /mnt/home/quancs/miniconda3/lib/libz.so.1.2.11
2b4dcb497000-2b4dcb4ab000 r-xp 00003000 fd:00 118628888                  /mnt/home/quancs/miniconda3/lib/libz.so.1.2.11
2b4dcb4ab000-2b4dcb4b2000 r--p 00017000 fd:00 118628888                  /mnt/home/quancs/miniconda3/lib/libz.so.1.2.11
2b4dcb4b2000-2b4dcb4b3000 r--p 0001d000 fd:00 118628888                  /mnt/home/quancs/miniconda3/lib/libz.so.1.2.11
2b4dcb4b3000-2b4dcb4b4000 rw-p 0001e000 fd:00 118628888                  /mnt/home/quancs/miniconda3/lib/libz.so.1.2.11
2b4dcb4b4000-2b4dcb4f4000 rw-p 00000000 00:00 0 
2b4dcb4f4000-2b4dcb4f9000 r--p 00000000 fd:00 118098522                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_datetime.cpython-38-x86_64-linux-gnu.so
2b4dcb4f9000-2b4dcb50a000 r-xp 00005000 fd:00 118098522                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_datetime.cpython-38-x86_64-linux-gnu.so
2b4dcb50a000-2b4dcb510000 r--p 00016000 fd:00 118098522                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_datetime.cpython-38-x86_64-linux-gnu.so
2b4dcb510000-2b4dcb511000 r--p 0001b000 fd:00 118098522                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_datetime.cpython-38-x86_64-linux-gnu.so
2b4dcb511000-2b4dcb513000 rw-p 0001c000 fd:00 118098522                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_datetime.cpython-38-x86_64-linux-gnu.so
2b4dcb513000-2b4dcb51a000 r--p 00000000 fd:00 118098523                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_decimal.cpython-38-x86_64-linux-gnu.so
2b4dcb51a000-2b4dcb54e000 r-xp 00007000 fd:00 118098523                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_decimal.cpython-38-x86_64-linux-gnu.so
2b4dcb54e000-2b4dcb55e000 r--p 0003b000 fd:00 118098523                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_decimal.cpython-38-x86_64-linux-gnu.so
2b4dcb55e000-2b4dcb55f000 r--p 0004a000 fd:00 118098523                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_decimal.cpython-38-x86_64-linux-gnu.so
2b4dcb55f000-2b4dcb562000 rw-p 0004b000 fd:00 118098523                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_decimal.cpython-38-x86_64-linux-gnu.so
2b4dcb562000-2b4dcb5a2000 rw-p 00000000 00:00 0 
2b4dcb5a2000-2b4dcb5a3000 r--p 00000000 fd:00 118098506                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_bisect.cpython-38-x86_64-linux-gnu.so
2b4dcb5a3000-2b4dcb5a4000 r-xp 00001000 fd:00 118098506                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_bisect.cpython-38-x86_64-linux-gnu.so
2b4dcb5a4000-2b4dcb5a5000 r--p 00002000 fd:00 118098506                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_bisect.cpython-38-x86_64-linux-gnu.so
2b4dcb5a5000-2b4dcb5a6000 r--p 00002000 fd:00 118098506                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_bisect.cpython-38-x86_64-linux-gnu.so
2b4dcb5a6000-2b4dcb5a7000 rw-p 00003000 fd:00 118098506                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_bisect.cpython-38-x86_64-linux-gnu.so
2b4dcb5a7000-2b4dcb5a8000 r--p 00000000 fd:00 118098542                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_sha512.cpython-38-x86_64-linux-gnu.so
2b4dcb5a8000-2b4dcb5b1000 r-xp 00001000 fd:00 118098542                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_sha512.cpython-38-x86_64-linux-gnu.so
2b4dcb5b1000-2b4dcb5b2000 r--p 0000a000 fd:00 118098542                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_sha512.cpython-38-x86_64-linux-gnu.so
2b4dcb5b2000-2b4dcb5b3000 r--p 0000a000 fd:00 118098542                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_sha512.cpython-38-x86_64-linux-gnu.so
2b4dcb5b3000-2b4dcb5b4000 rw-p 0000b000 fd:00 118098542                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_sha512.cpython-38-x86_64-linux-gnu.so
2b4dcb5b4000-2b4dcb5b6000 r--p 00000000 fd:00 118098538                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_random.cpython-38-x86_64-linux-gnu.so
2b4dcb5b6000-2b4dcb5b8000 r-xp 00002000 fd:00 118098538                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_random.cpython-38-x86_64-linux-gnu.so
2b4dcb5b8000-2b4dcb5b9000 r--p 00004000 fd:00 118098538                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_random.cpython-38-x86_64-linux-gnu.so
2b4dcb5b9000-2b4dcb5ba000 r--p 00004000 fd:00 118098538                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_random.cpython-38-x86_64-linux-gnu.so
2b4dcb5ba000-2b4dcb5bb000 rw-p 00005000 fd:00 118098538                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_random.cpython-38-x86_64-linux-gnu.so
2b4dcb5bb000-2b4dcb63b000 rw-p 00000000 00:00 0 
2b4dcb63b000-2b4dcb64b000 r--p 00000000 fd:00 118098545                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_ssl.cpython-38-x86_64-linux-gnu.so
2b4dcb64b000-2b4dcb654000 r-xp 00010000 fd:00 118098545                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_ssl.cpython-38-x86_64-linux-gnu.so
2b4dcb654000-2b4dcb660000 r--p 00019000 fd:00 118098545                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_ssl.cpython-38-x86_64-linux-gnu.so
2b4dcb660000-2b4dcb661000 ---p 00025000 fd:00 118098545                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_ssl.cpython-38-x86_64-linux-gnu.so
2b4dcb661000-2b4dcb662000 r--p 00025000 fd:00 118098545                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_ssl.cpython-38-x86_64-linux-gnu.so
2b4dcb662000-2b4dcb669000 rw-p 00026000 fd:00 118098545                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_ssl.cpython-38-x86_64-linux-gnu.so
2b4dcb669000-2b4dcb688000 r--p 00000000 fd:00 118113625                  /mnt/home/quancs/miniconda3/lib/libssl.so.1.1
2b4dcb688000-2b4dcb6d3000 r-xp 0001f000 fd:00 118113625                  /mnt/home/quancs/miniconda3/lib/libssl.so.1.1
2b4dcb6d3000-2b4dcb6ec000 r--p 0006a000 fd:00 118113625                  /mnt/home/quancs/miniconda3/lib/libssl.so.1.1
2b4dcb6ec000-2b4dcb6ed000 ---p 00083000 fd:00 118113625                  /mnt/home/quancs/miniconda3/lib/libssl.so.1.1
2b4dcb6ed000-2b4dcb6f6000 r--p 00083000 fd:00 118113625                  /mnt/home/quancs/miniconda3/lib/libssl.so.1.1
2b4dcb6f6000-2b4dcb6fa000 rw-p 0008c000 fd:00 118113625                  /mnt/home/quancs/miniconda3/lib/libssl.so.1.1
2b4dcb6fa000-2b4dcb775000 r--p 00000000 fd:00 115214149                  /mnt/home/quancs/miniconda3/lib/libcrypto.so.1.1
2b4dcb775000-2b4dcb909000 r-xp 0007b000 fd:00 115214149                  /mnt/home/quancs/miniconda3/lib/libcrypto.so.1.1
2b4dcb909000-2b4dcb996000 r--p 0020f000 fd:00 115214149                  /mnt/home/quancs/miniconda3/lib/libcrypto.so.1.1
2b4dcb996000-2b4dcb9c1000 r--p 0029b000 fd:00 115214149                  /mnt/home/quancs/miniconda3/lib/libcrypto.so.1.1
2b4dcb9c1000-2b4dcb9c3000 rw-p 002c6000 fd:00 115214149                  /mnt/home/quancs/miniconda3/lib/libcrypto.so.1.1
2b4dcb9c3000-2b4dcba07000 rw-p 00000000 00:00 0 
2b4dcba07000-2b4dcba0e000 r--p 00000000 fd:00 118098567                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/pyexpat.cpython-38-x86_64-linux-gnu.so
2b4dcba0e000-2b4dcba3b000 r-xp 00007000 fd:00 118098567                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/pyexpat.cpython-38-x86_64-linux-gnu.so
2b4dcba3b000-2b4dcba46000 r--p 00034000 fd:00 118098567                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/pyexpat.cpython-38-x86_64-linux-gnu.so
2b4dcba46000-2b4dcba49000 r--p 0003e000 fd:00 118098567                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/pyexpat.cpython-38-x86_64-linux-gnu.so
2b4dcba49000-2b4dcba4b000 rw-p 00041000 fd:00 118098567                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/pyexpat.cpython-38-x86_64-linux-gnu.so
2b4dcba4b000-2b4dcba8b000 rw-p 00000000 00:00 0 
2b4dcba8b000-2b4dcba8d000 r--p 00000000 fd:00 118098576                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/zlib.cpython-38-x86_64-linux-gnu.so
2b4dcba8d000-2b4dcba90000 r-xp 00002000 fd:00 118098576                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/zlib.cpython-38-x86_64-linux-gnu.so
2b4dcba90000-2b4dcba93000 r--p 00005000 fd:00 118098576                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/zlib.cpython-38-x86_64-linux-gnu.so
2b4dcba93000-2b4dcba94000 r--p 00007000 fd:00 118098576                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/zlib.cpython-38-x86_64-linux-gnu.so
2b4dcba94000-2b4dcba95000 rw-p 00008000 fd:00 118098576                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/zlib.cpython-38-x86_64-linux-gnu.so
2b4dcba95000-2b4dcbb15000 rw-p 00000000 00:00 0 
2b4dcbb15000-2b4dcbb18000 r--p 00000000 fd:00 118098508                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_bz2.cpython-38-x86_64-linux-gnu.so
2b4dcbb18000-2b4dcbb28000 r-xp 00003000 fd:00 118098508                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_bz2.cpython-38-x86_64-linux-gnu.so
2b4dcbb28000-2b4dcbb2b000 r--p 00013000 fd:00 118098508                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_bz2.cpython-38-x86_64-linux-gnu.so
2b4dcbb2b000-2b4dcbb2c000 r--p 00015000 fd:00 118098508                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_bz2.cpython-38-x86_64-linux-gnu.so
2b4dcbb2c000-2b4dcbb2e000 rw-p 00016000 fd:00 118098508                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_bz2.cpython-38-x86_64-linux-gnu.so
2b4dcbb2e000-2b4dcbb31000 r--p 00000000 fd:00 118098529                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_lzma.cpython-38-x86_64-linux-gnu.so
2b4dcbb31000-2b4dcbb35000 r-xp 00003000 fd:00 118098529                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_lzma.cpython-38-x86_64-linux-gnu.so
2b4dcbb35000-2b4dcbb38000 r--p 00007000 fd:00 118098529                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_lzma.cpython-38-x86_64-linux-gnu.so
2b4dcbb38000-2b4dcbb39000 r--p 00009000 fd:00 118098529                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_lzma.cpython-38-x86_64-linux-gnu.so
2b4dcbb39000-2b4dcbb3a000 rw-p 0000a000 fd:00 118098529                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_lzma.cpython-38-x86_64-linux-gnu.so
2b4dcbb3a000-2b4dcbb3e000 r--p 00000000 fd:00 118097249                  /mnt/home/quancs/miniconda3/lib/liblzma.so.5.2.5
2b4dcbb3e000-2b4dcbb55000 r-xp 00004000 fd:00 118097249                  /mnt/home/quancs/miniconda3/lib/liblzma.so.5.2.5
2b4dcbb55000-2b4dcbb60000 r--p 0001b000 fd:00 118097249                  /mnt/home/quancs/miniconda3/lib/liblzma.so.5.2.5
2b4dcbb60000-2b4dcbb61000 ---p 00026000 fd:00 118097249                  /mnt/home/quancs/miniconda3/lib/liblzma.so.5.2.5
2b4dcbb61000-2b4dcbb62000 r--p 00026000 fd:00 118097249                  /mnt/home/quancs/miniconda3/lib/liblzma.so.5.2.5
2b4dcbb62000-2b4dcbb63000 rw-p 00027000 fd:00 118097249                  /mnt/home/quancs/miniconda3/lib/liblzma.so.5.2.5
2b4dcbb63000-2b4dcbb65000 r--p 00000000 fd:00 118098561                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/grp.cpython-38-x86_64-linux-gnu.so
2b4dcbb65000-2b4dcbb66000 r-xp 00002000 fd:00 118098561                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/grp.cpython-38-x86_64-linux-gnu.so
2b4dcbb66000-2b4dcbb67000 r--p 00003000 fd:00 118098561                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/grp.cpython-38-x86_64-linux-gnu.so
2b4dcbb67000-2b4dcbb68000 r--p 00003000 fd:00 118098561                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/grp.cpython-38-x86_64-linux-gnu.so
2b4dcbb68000-2b4dcbb69000 rw-p 00004000 fd:00 118098561                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/grp.cpython-38-x86_64-linux-gnu.so
2b4dcbb69000-2b4dcbc69000 rw-p 00000000 00:00 0 
2b4dcbc69000-2b4dcbc6a000 r--p 00000000 fd:00 118239435                  /mnt/home/quancs/miniconda3/lib/python3.8/site-packages/mkl/_mklinit.cpython-38-x86_64-linux-gnu.so
2b4dcbc6a000-2b4dcbc6b000 r-xp 00001000 fd:00 118239435                  /mnt/home/quancs/miniconda3/lib/python3.8/site-packages/mkl/_mklinit.cpython-38-x86_64-linux-gnu.so
2b4dcbc6b000-2b4dcbc6c000 r--p 00002000 fd:00 118239435                  /mnt/home/quancs/miniconda3/lib/python3.8/site-packages/mkl/_mklinit.cpython-38-x86_64-linux-gnu.so
2b4dcbc6c000-2b4dcbc6d000 r--p 00002000 fd:00 118239435                  /mnt/home/quancs/miniconda3/lib/python3.8/site-packages/mkl/_mklinit.cpython-38-x86_64-linux-gnu.so
2b4dcbc6d000-2b4dcbc6e000 rw-p 00003000 fd:00 118239435                  /mnt/home/quancs/miniconda3/lib/python3.8/site-packages/mkl/_mklinit.cpython-38-x86_64-linux-gnu.so
2b4dcbc6e000-2b4dcbc70000 r--p 00000000 fd:00 118242066                  /mnt/home/quancs/miniconda3/lib/python3.8/site-packages/numpy/linalg/lapack_lite.cpython-38-x86_64-linux-gnu.so
2b4dcbc70000-2b4dcbc72000 r-xp 00002000 fd:00 118242066                  /mnt/home/quancs/miniconda3/lib/python3.8/site-packages/numpy/linalg/lapack_lite.cpython-38-x86_64-linux-gnu.so
2b4dcbc72000-2b4dcbc73000 r--p 00004000 fd:00 118242066                  /mnt/home/quancs/miniconda3/lib/python3.8/site-packages/numpy/linalg/lapack_lite.cpython-38-x86_64-linux-gnu.so
2b4dcbc73000-2b4dcbc74000 r--p 00004000 fd:00 118242066                  /mnt/home/quancs/miniconda3/lib/python3.8/site-packages/numpy/linalg/lapack_lite.cpython-38-x86_64-linux-gnu.so
2b4dcbc74000-2b4dcbc75000 rw-p 00005000 fd:00 118242066                  /mnt/home/quancs/miniconda3/lib/python3.8/site-packages/numpy/linalg/lapack_lite.cpython-38-x86_64-linux-gnu.so
2b4dcbc75000-2b4dcbc76000 r--p 00000000 fd:00 118098515                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_contextvars.cpython-38-x86_64-linux-gnu.so
2b4dcbc76000-2b4dcbc77000 r-xp 00001000 fd:00 118098515                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_contextvars.cpython-38-x86_64-linux-gnu.so
2b4dcbc77000-2b4dcbc78000 r--p 00002000 fd:00 118098515                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_contextvars.cpython-38-x86_64-linux-gnu.so
2b4dcbc78000-2b4dcbc79000 r--p 00002000 fd:00 118098515                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_contextvars.cpython-38-x86_64-linux-gnu.so
2b4dcbc79000-2b4dcbc7a000 rw-p 00003000 fd:00 118098515                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_contextvars.cpython-38-x86_64-linux-gnu.so
2b4dcbc7a000-2b4dcbfba000 rw-p 00000000 00:00 0 
2b4dcbfba000-2b4dcbfbd000 r--p 00000000 fd:00 118098573                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/termios.cpython-38-x86_64-linux-gnu.so
2b4dcbfbd000-2b4dcbfbe000 r-xp 00003000 fd:00 118098573                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/termios.cpython-38-x86_64-linux-gnu.so
2b4dcbfbe000-2b4dcbfc0000 r--p 00004000 fd:00 118098573                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/termios.cpython-38-x86_64-linux-gnu.so
2b4dcbfc0000-2b4dcbfc1000 r--p 00005000 fd:00 118098573                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/termios.cpython-38-x86_64-linux-gnu.so
2b4dcbfc1000-2b4dcbfc3000 rw-p 00006000 fd:00 118098573                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/termios.cpython-38-x86_64-linux-gnu.so
2b4dcbfc3000-2b4dcbfcb000 r--p 00000000 fd:00 118098518                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so
2b4dcbfcb000-2b4dcbfda000 r-xp 00008000 fd:00 118098518                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so
2b4dcbfda000-2b4dcbfe0000 r--p 00017000 fd:00 118098518                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so
2b4dcbfe0000-2b4dcbfe1000 ---p 0001d000 fd:00 118098518                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so
2b4dcbfe1000-2b4dcbfe2000 r--p 0001d000 fd:00 118098518                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so
2b4dcbfe2000-2b4dcbfe6000 rw-p 0001e000 fd:00 118098518                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so
2b4dcbfe6000-2b4dcbfe8000 r--p 00000000 fd:00 118097216                  /mnt/home/quancs/miniconda3/lib/libffi.so.7.1.0
2b4dcbfe8000-2b4dcbfee000 r-xp 00002000 fd:00 118097216                  /mnt/home/quancs/miniconda3/lib/libffi.so.7.1.0
2b4dcbfee000-2b4dcbfef000 r--p 00008000 fd:00 118097216                  /mnt/home/quancs/miniconda3/lib/libffi.so.7.1.0
2b4dcbfef000-2b4dcbff0000 ---p 00009000 fd:00 118097216                  /mnt/home/quancs/miniconda3/lib/libffi.so.7.1.0
2b4dcbff0000-2b4dcbff1000 r--p 00009000 fd:00 118097216                  /mnt/home/quancs/miniconda3/lib/libffi.so.7.1.0
2b4dcbff1000-2b4dcbff2000 rw-p 0000a000 fd:00 118097216                  /mnt/home/quancs/miniconda3/lib/libffi.so.7.1.0
2b4dcbff2000-2b4dcc032000 rw-p 00000000 00:00 0 *** Error in `/home/quancs/miniconda3/bin/python': double free or corruption (!prev): 0x000055f3db4fea60 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7340f)[0x2b4dc3f6540f]
/lib64/libc.so.6(+0x78c7e)[0x2b4dc3f6ac7e]
/lib64/libc.so.6(+0x79957)[0x2b4dc3f6b957]
/lib64/ld-linux-x86-64.so.2(_dl_deallocate_tls+0x58)[0x2b4dc3ac25a8]
/lib64/libpthread.so.0(+0x7237)[0x2b4dc3cdb237]
/lib64/libpthread.so.0(+0x734f)[0x2b4dc3cdb34f]
/lib64/libpthread.so.0(pthread_join+0xdb)[0x2b4dc3cdd69b]
/home/quancs/miniconda3/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so(pthreadpool_destroy+0x8b)[0x2b4e8cf2a37b]
/home/quancs/miniconda3/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so(+0x251b827)[0x2b4e8aba4827]
/home/quancs/miniconda3/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so(_ZN2at15set_num_threadsEi+0x38)[0x2b4e8957cf28]
/home/quancs/miniconda3/lib/python3.8/site-packages/torch/lib/libtorch_python.so(+0x99e3e6)[0x2b4e155583e6]
/home/quancs/miniconda3/bin/python(+0x140825)[0x55f3d9ebb825]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0xffa68)[0x55f3d9e7aa68]
/home/quancs/miniconda3/bin/python(_PyFunction_Vectorcall+0x10b)[0x55f3d9f0686b]
/home/quancs/miniconda3/bin/python(PyVectorcall_Call+0x71)[0x55f3d9eb8041]
/home/quancs/miniconda3/bin/python(_PyEval_EvalFrameDefault+0x1fdb)[0x55f3d9f3d99b]
/home/quancs/miniconda3/bin/python(_PyFunction_Vectorcall+0x10b)[0x55f3d9f0686b]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0x100066)[0x55f3d9e7b066]
/home/quancs/miniconda3/bin/python(_PyEval_EvalCodeWithName+0x2d2)[0x55f3d9f05a92]
/home/quancs/miniconda3/bin/python(+0x18bd20)[0x55f3d9f06d20]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0x1001de)[0x55f3d9e7b1de]
/home/quancs/miniconda3/bin/python(_PyFunction_Vectorcall+0x10b)[0x55f3d9f0686b]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0x100066)[0x55f3d9e7b066]
/home/quancs/miniconda3/bin/python(_PyObject_FastCallDict+0x38d)[0x55f3d9f0760d]
/home/quancs/miniconda3/bin/python(_PyObject_Call_Prepend+0x63)[0x55f3d9f07733]
/home/quancs/miniconda3/bin/python(+0x18c8ca)[0x55f3d9f078ca]
/home/quancs/miniconda3/bin/python(_PyObject_MakeTpCall+0x1a4)[0x55f3d9eb87d4]
/home/quancs/miniconda3/bin/python(+0xffee1)[0x55f3d9e7aee1]
/home/quancs/miniconda3/bin/python(_PyFunction_Vectorcall+0x10b)[0x55f3d9f0686b]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0xffa68)[0x55f3d9e7aa68]
/home/quancs/miniconda3/bin/python(_PyFunction_Vectorcall+0x10b)[0x55f3d9f0686b]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0xffa68)[0x55f3d9e7aa68]
/home/quancs/miniconda3/bin/python(_PyFunction_Vectorcall+0x10b)[0x55f3d9f0686b]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0x100066)[0x55f3d9e7b066]
/home/quancs/miniconda3/bin/python(_PyEval_EvalCodeWithName+0x659)[0x55f3d9f05e19]
/home/quancs/miniconda3/bin/python(_PyObject_FastCallDict+0x20c)[0x55f3d9f0748c]
/home/quancs/miniconda3/bin/python(_PyObject_Call_Prepend+0x63)[0x55f3d9f07733]
/home/quancs/miniconda3/bin/python(+0x18c8ca)[0x55f3d9f078ca]
/home/quancs/miniconda3/bin/python(_PyObject_MakeTpCall+0x1a4)[0x55f3d9eb87d4]
/home/quancs/miniconda3/bin/python(+0xffee1)[0x55f3d9e7aee1]
/home/quancs/miniconda3/bin/python(_PyFunction_Vectorcall+0x10b)[0x55f3d9f0686b]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0x100066)[0x55f3d9e7b066]
/home/quancs/miniconda3/bin/python(_PyFunction_Vectorcall+0x10b)[0x55f3d9f0686b]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0x13d9ef)[0x55f3d9eb89ef]
/home/quancs/miniconda3/bin/python(+0x20958a)[0x55f3d9f8458a]
/home/quancs/miniconda3/bin/python(PyObject_GetIter+0x16)[0x55f3d9ebe376]
/home/quancs/miniconda3/bin/python(_PyEval_EvalFrameDefault+0x165a)[0x55f3d9f3d01a]
/home/quancs/miniconda3/bin/python(_PyEval_EvalCodeWithName+0x2d2)[0x55f3d9f05a92]
/home/quancs/miniconda3/bin/python(PyEval_EvalCodeEx+0x44)[0x55f3d9f06754]
/home/quancs/miniconda3/bin/python(PyEval_EvalCode+0x1c)[0x55f3d9f94edc]
/home/quancs/miniconda3/bin/python(+0x24f083)[0x55f3d9fca083]
/home/quancs/miniconda3/bin/python(+0x140699)[0x55f3d9ebb699]
/home/quancs/miniconda3/bin/python(+0x7df36)[0x55f3d9df8f36]
/home/quancs/miniconda3/bin/python(+0xffee1)[0x55f3d9e7aee1]
======= Memory map: ========
2b4dc3ab1000-2b4dc3ad1000 r-xp 00000000 08:04 13680862                   /usr/lib64/ld-2.18.so
2b4dc3ad1000-2b4dc3ad2000 rw-p 00000000 00:00 0 
2b4dc3ad2000-2b4dc3ad9000 r--s 00000000 08:04 14024983                   /usr/lib64/gconv/gconv-modules.cache
2b4dc3ad9000-2b4dc3adc000 r--p 00000000 fd:00 118098547                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_struct.cpython-38-x86_64-linux-gnu.so
2b4dc3adc000-2b4dc3ae2000 r-xp 00003000 fd:00 118098547                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_struct.cpython-38-x86_64-linux-gnu.so
2b4dc3ae2000-2b4dc3ae6000 r--p 00009000 fd:00 118098547                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_struct.cpython-38-x86_64-linux-gnu.so
2b4dc3ae6000-2b4dc3ae7000 r--p 0000c000 fd:00 118098547                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_struct.cpython-38-x86_64-linux-gnu.so
2b4dc3ae7000-2b4dc3ae8000 rw-p 0000d000 fd:00 118098547                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_struct.cpython-38-x86_64-linux-gnu.so
2b4dc3ae8000-2b4dc3aec000 r--p 00000000 fd:00 118098562                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so
2b4dc3aec000-2b4dc3af4000 r-xp 00004000 fd:00 118098562                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so
2b4dc3af4000-2b4dc3af8000 r--p 0000c000 fd:00 118098562                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so
2b4dc3af8000-2b4dc3af9000 r--p 0000f000 fd:00 118098562                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so
2b4dc3af9000-2b4dc3afa000 rw-p 00010000 fd:00 118098562                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so
2b4dc3afa000-2b4dc3afb000 rw-s 00000000 00:13 252171475                  /dev/shm/sem.N38K6v (deleted)
2b4dc3afb000-2b4dc3afc000 rw-s 00000000 00:13 252171476                  /dev/shm/sem.1j3uZn (deleted)
2b4dc3afc000-2b4dc3afd000 rw-s 00000000 00:13 252171477                  /dev/shm/sem.ZiTgSf (deleted)
2b4dc3afd000-2b4dc3afe000 rw-s 00000000 00:13 252171478                  /dev/shm/sem.Tyy5K7 (deleted)
2b4dc3afe000-2b4dc3aff000 r--p 00000000 fd:00 118098526                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_heapq.cpython-38-x86_64-linux-gnu.so
2b4dc3aff000-2b4dc3b02000 r-xp 00001000 fd:00 118098526                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_heapq.cpython-38-x86_64-linux-gnu.so
2b4dc3b02000-2b4dc3b05000 r--p 00004000 fd:00 118098526                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_heapq.cpython-38-x86_64-linux-gnu.so
2b4dc3b05000-2b4dc3b06000 r--p 00006000 fd:00 118098526                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_heapq.cpython-38-x86_64-linux-gnu.so
2b4dc3b06000-2b4dc3b07000 rw-p 00007000 fd:00 118098526                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_heapq.cpython-38-x86_64-linux-gnu.so
2b4dc3b07000-2b4dc3b08000 r--p 00000000 fd:00 118098533                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_opcode.cpython-38-x86_64-linux-gnu.so
2b4dc3b08000-2b4dc3b09000 r-xp 00001000 fd:00 118098533                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_opcode.cpython-38-x86_64-linux-gnu.so
2b4dc3b09000-2b4dc3b0a000 r--p 00002000 fd:00 118098533                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_opcode.cpython-38-x86_64-linux-gnu.so
2b4dc3b0a000-2b4dc3b0b000 r--p 00002000 fd:00 118098533                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_opcode.cpython-38-x86_64-linux-gnu.so
2b4dc3b0b000-2b4dc3b0c000 rw-p 00003000 fd:00 118098533                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_opcode.cpython-38-x86_64-linux-gnu.so
2b4dc3b0c000-2b4dc3b0e000 r--p 00000000 fd:00 118098537                  /mnt/home/quancs/miniconda3/lib/python3.8/lib-dynload/_quTraceback (most recent call last):
  File "/home/quancs/miniconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 990, in _try_get_data
    data = self._data_queue.get(timeout=timeout)
  File "/home/quancs/miniconda3/lib/python3.8/multiprocessing/queues.py", line 107, in get
    if not self._poll(timeout):
  File "/home/quancs/miniconda3/lib/python3.8/multiprocessing/connection.py", line 257, in poll
    return self._poll(timeout)
  File "/home/quancs/miniconda3/lib/python3.8/multiprocessing/connection.py", line 424, in _poll
    r = wait([self], timeout)
  File "/home/quancs/miniconda3/lib/python3.8/multiprocessing/connection.py", line 931, in wait
    ready = selector.select(timeout)
  File "/home/quancs/miniconda3/lib/python3.8/selectors.py", line 415, in select
    fd_event_list = self._selector.poll(timeout)
  File "/home/quancs/miniconda3/lib/python3.8/site-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler
    _error_if_any_worker_fails()
RuntimeError: DataLoader worker (pid 38727) is killed by signal: Aborted. 

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/quancs/miniconda3/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/quancs/miniconda3/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/mnt/home/quancs/.vscode-server/extensions/ms-python.python-2021.12.1559732655/pythonFiles/lib/python/debugpy/__main__.py", line 45, in <module>
    cli.main()
  File "/mnt/home/quancs/.vscode-server/extensions/ms-python.python-2021.12.1559732655/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 444, in main
    run()
  File "/mnt/home/quancs/.vscode-server/extensions/ms-python.python-2021.12.1559732655/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 285, in run_file
    runpy.run_path(target_as_str, run_name=compat.force_str("__main__"))
  File "/home/quancs/miniconda3/lib/python3.8/runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/quancs/miniconda3/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/quancs/miniconda3/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/mnt/home/quancs/projects/NBSS_pmt/boring.py", line 21, in <module>
    for x in train_data:
  File "/home/quancs/miniconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
    data = self._next_data()
  File "/home/quancs/miniconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1186, in _next_data
    idx, data = self._get_data()
  File "/home/quancs/miniconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1152, in _get_data
    success, data = self._try_get_data()
  File "/home/quancs/miniconda3/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1003, in _try_get_data
    raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str)) from e
RuntimeError: DataLoader worker (pid(s) 38727) exited unexpectedly

@ghost ghost assigned karthiknadig Dec 21, 2021
@karthiknadig karthiknadig transferred this issue from microsoft/vscode-python Jan 13, 2022
@fabioz
Copy link
Collaborator

fabioz commented Apr 1, 2022

@quancs I couldn't reproduce this as is. When you ran and had the crash, did you add a breakpoint in any particular line?

Also, it may be worth trying with a newer version of pytorch (since the error trace points to some error inside libtorch_cpu.so).

@int19h
Copy link
Collaborator

int19h commented Jun 22, 2022

Please re-open with more details as outlined above if this issue still occurs.

@int19h int19h closed this as completed Jun 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
waiting for response
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@brettcannon @fabioz @int19h @karthiknadig @quancs