The program hangs after using the debugger

[kgboyko]

Type: Bug

The program hangs after using the debugger

After setting a breakpoint in a Jupiter notebook cell and then running the program, the program hangs in a subprocess.

Steps to Reproduce:

1. Kaggle Docker GPU v160 is used (I also tested it on other versions v158, v159).
2. ! pip install -U ipykernel
   ! pip install vllm==0.9.1 # (Tested on versions 0.7.2, 0.8.5.post1, 0.9.1)
3. Create a Jupiter notebook with the following code. Runtime environment: Python-3.11.11
4. Set a breakpoint at the last line, then run the application in debug mode (Required NVidia GPU Card >= 3xxx series).
   VS Code version: Code 1.101.0 (dfaf44141ea9deb3b4096f7cd6d24e00c147a4b1, 2025-06-11T15:00:50.123Z)
   OS version: Windows_NT x64 10.0.26100
   Modes:
   Remote OS version: Linux x64 6.8.0-60-generic
   Remote OS version: Linux x64 6.8.0-60-generic

System Info

Item	Value
CPUs	Intel(R) Core(TM) Ultra 5 125H (18 x 2995)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: enabled_on direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_graphite: disabled_off video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled webgpu: enabled webnn: disabled_off
Load (avg)	undefined
Memory (System)	23.47GB (12.40GB free)
Process Argv	--crash-reporter-id dc65549e-5a3d-4f34-a3ab-d2ec4803df70
Screen Reader	no
VM	0%

Item	Value
Remote	SSH: Zoo
OS	Linux x64 6.8.0-60-generic
CPUs	13th Gen Intel(R) Core(TM) i5-13600K (20 x 1006)
Memory (System)	125.54GB (119.80GB free)
VM	0%

Item	Value
Remote	Container gcr.io/kaggle-gpu-images/python:v160-cu125-pytorch-2.7.0_vllm_091 (kgl_gpu_160_1) @ Zoo
OS	Linux x64 6.8.0-60-generic
CPUs	13th Gen Intel(R) Core(TM) i5-13600K (20 x 998)
Memory (System)	125.54GB (119.80GB free)
VM	0%

Extensions (23)

Extension	Author (truncated)	Version
jupyter-keymap	ms-	1.1.2
remote-containers	ms-	0.417.0
remote-ssh	ms-	0.120.0
remote-ssh-edit	ms-	0.87.0
vscode-remote-extensionpack	ms-	0.26.0
remote-explorer	ms-	0.5.0
remote-server	ms-	1.5.2
docker	doc	0.10.0
vscode-containers	ms-	2.0.3
vscode-docker	ms-	2.0.0
debugpy	ms-	2025.8.0
python	ms-	2025.6.1
vscode-pylance	ms-	2025.5.1
datawrangler	ms-	1.22.0
jupyter	ms-	2025.4.1
jupyter-keymap	ms-	1.1.2
jupyter-renderers	ms-	1.1.0
vscode-jupyter-cell-tags	ms-	0.1.9
vscode-jupyter-powertoys	ms-	0.1.1
vscode-jupyter-slideshow	ms-	0.1.6
markdown-preview-enhanced	shd	0.8.18
intellicode-api-usage-examples	Vis	0.2.9
vscodeintellicode	Vis	1.3.2

A/B Experiments

vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscod805:30301674
binariesv615:30325510
c4g48928:30535728
azure-dev_surveyone:30548225
962ge761:30959799
h48ei257:31000450
pythontbext0:30879054
cppperfnew:31000557
dwnewjupytercf:31046870
pythonrstrctxt:31112756
nativeloc1:31192215
5fd0e150:31155592
dwcopilot:31170013
6074i472:31201624
dwoutputs:31242946
customenabled:31248079
hdaa2157:31222309
copilot_t_ci:31222730
e5gg6876:31282496
pythoneinst12:31285622
bgtreat:31268568
4gafe986:31271826
c7cif404:31314491
996jf627:31283433
pythonrdcb7:31303018
usemplatestapi:31297334
0aa6g176:31307128
7bj51361:31289155
747dc170:31275177
pylancecolor:31314202
aj953862:31281341
generatesymbolt:31295002
convertfstringf:31295003
gendocf:31295004
pylancequickfixf:31319675
0g0a1943:31327026

import os
import transformers; print('Transformers version:', transformers.__version__)
import torch; print('Torch version:', torch.__version__)
import vllm; print('vLLM version:', vllm.__version__)
from vllm import LLM

container_date = os.environ.get('BUILD_DATE', '').split('-')[0]
print(f"Kaggle Docker BUILD_DATE={container_date}")

!cat /etc/os-release | grep -oP "PRETTY_NAME=\"\K([^\"]*)" && uname -r
!free -h
!nv_version="$(nvidia-smi --query-gpu=driver_version --format=csv,noheader)" && echo "My NVIDIA driver version is '${nv_version}'."
!ls -l /usr/local | grep cuda

is_debug = True

os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
os.environ["TRITON_PTXAS_PATH"] = "/usr/local/cuda/bin/ptxas"

if is_debug:
    os.environ['OMP_NUM_THREADS'] = '1'
    os.environ['MKL_NUM_THREADS'] = '1'
    os.environ['POLARS_ALLOW_FORKING_THREAD'] = '1'
    os.environ['CUDA_LAUNCH_BLOCKING'] = "1"
    os.environ['TORCH_USE_CUDA_DSA'] = "1"
    os.environ['OPENBLAS_NUM_THREADS'] = '1'
    os.environ["NUM_INTER_THREADS"] = "1"
    os.environ["NUM_INTRA_THREADS"] = "1"
    os.environ["XLA_FLAGS"] = ("--xla_cpu_multi_thread_eigen=false "
                               "intra_op_parallelism_threads=1")

os.environ["TOKENIZERS_PARALLELISM"] = "false"
os.environ["CUDA_VISIBLE_DEVICES"] = "0"
os.environ["PYTORCH_CUDA_ALLOC_CONF"]='expandable_segments:True'

llm = LLM(
    model="Qwen/Qwen3-0.6B",
    max_num_seqs=1,
    max_model_len=1024,
    trust_remote_code=True,
    enable_prefix_caching = True,
    dtype = torch.half,
    tensor_parallel_size=1,
    gpu_memory_utilization=0.96,
    enforce_eager=True,
    seed=2024,
)
tokenizer = llm.get_tokenizer()
print("LLM Started")

5. When running with a debugger, the following message appears in the log. If running without a debugger, this message does not appear.

77.40s - pydevd: Sending message related to process being replaced timed-out after 5 seconds

6. When you press Ctrl+C, this dump appears. It is clear that it hangs in some subprocess.

---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
/tmp/ipykernel_2264/2697110242.py in <cell line: 0>()
     34 os.environ["PYTORCH_CUDA_ALLOC_CONF"]='expandable_segments:True'
     35 
---> 36 llm = LLM(
     37     model="Qwen/Qwen3-0.6B",
     38     max_num_seqs=1,

/usr/local/lib/python3.11/dist-packages/vllm/entrypoints/llm.py in __init__(self, model, task, tokenizer, tokenizer_mode, skip_tokenizer_init, trust_remote_code, allowed_local_media_path, tensor_parallel_size, dtype, quantization, revision, tokenizer_revision, seed, gpu_memory_utilization, swap_space, cpu_offload_gb, enforce_eager, max_seq_len_to_capture, disable_custom_all_reduce, disable_async_output_proc, hf_token, hf_overrides, mm_processor_kwargs, override_pooler_config, compilation_config, **kwargs)
    241 
    242         # Create the Engine (autoselects V0 vs V1)
--> 243         self.llm_engine = LLMEngine.from_engine_args(
    244             engine_args=engine_args, usage_context=UsageContext.LLM_CLASS)
    245         self.engine_class = type(self.llm_engine)

/usr/local/lib/python3.11/dist-packages/vllm/engine/llm_engine.py in from_engine_args(cls, engine_args, usage_context, stat_loggers)
    492         """Creates an LLM engine from the engine arguments."""
    493         # Create the engine configs.
--> 494         vllm_config = engine_args.create_engine_config(usage_context)
    495 
    496         engine_cls = cls

/usr/local/lib/python3.11/dist-packages/vllm/engine/arg_utils.py in create_engine_config(self, usage_context)
   1016 
   1017         device_config = DeviceConfig(device=current_platform.device_type)
-> 1018         model_config = self.create_model_config()
   1019 
   1020         # * If VLLM_USE_V1 is unset, we enable V1 for "supported features"

/usr/local/lib/python3.11/dist-packages/vllm/engine/arg_utils.py in create_model_config(self)
    908             self.load_format = LoadFormat.RUNAI_STREAMER
    909 
--> 910         return ModelConfig(
    911             model=self.model,
    912             hf_config_path=self.hf_config_path,

    [... skipping hidden 1 frame]

/usr/local/lib/python3.11/dist-packages/vllm/config.py in __post_init__(self)
    546             self.model, hf_token=self.hf_token, revision=self.revision)
    547 
--> 548         supported_tasks, task = self._resolve_task(self.task)
    549         self.supported_tasks = supported_tasks
    550         self.task = task

/usr/local/lib/python3.11/dist-packages/vllm/config.py in _resolve_task(self, task_option)
    796             # NOTE: Listed from highest to lowest priority,
    797             # in case the model supports multiple of them
--> 798             "transcription": registry.is_transcription_model(architectures),
    799             "generate": registry.is_text_generation_model(architectures),
    800             "pooling": registry.is_pooling_model(architectures),

/usr/local/lib/python3.11/dist-packages/vllm/model_executor/models/registry.py in is_transcription_model(self, architectures)
    556         architectures: Union[str, list[str]],
    557     ) -> bool:
--> 558         model_cls, _ = self.inspect_model_cls(architectures)
    559         return model_cls.supports_transcription
    560 

/usr/local/lib/python3.11/dist-packages/vllm/model_executor/models/registry.py in inspect_model_cls(self, architectures)
    470 
    471         for arch in architectures:
--> 472             model_info = self._try_inspect_model_cls(arch)
    473             if model_info is not None:
    474                 return (model_info, arch)

/usr/local/lib/python3.11/dist-packages/vllm/model_executor/models/registry.py in _try_inspect_model_cls(self, model_arch)
    443             return None
    444 
--> 445         return _try_inspect_model_cls(model_arch, self.models[model_arch])
    446 
    447     def _normalize_archs(

/usr/local/lib/python3.11/dist-packages/vllm/model_executor/models/registry.py in _try_inspect_model_cls(model_arch, model)
    363 ) -> Optional[_ModelInfo]:
    364     try:
--> 365         return model.inspect_model_cls()
    366     except Exception:
    367         logger.exception("Error in inspecting model architecture '%s'",

/usr/local/lib/python3.11/dist-packages/vllm/model_executor/models/registry.py in inspect_model_cls(self)
    334     # Performed in another process to avoid initializing CUDA
    335     def inspect_model_cls(self) -> _ModelInfo:
--> 336         return _run_in_subprocess(
    337             lambda: _ModelInfo.from_model_cls(self.load_model_cls()))
    338 

/usr/local/lib/python3.11/dist-packages/vllm/model_executor/models/registry.py in _run_in_subprocess(fn)
    590         # cannot use `sys.executable __file__` here because the script
    591         # contains relative imports
--> 592         returned = subprocess.run(_SUBPROCESS_COMMAND,
    593                                   input=input_bytes,
    594                                   capture_output=True)

/usr/lib/python3.11/subprocess.py in run(input, capture_output, timeout, check, *popenargs, **kwargs)
    548     with Popen(*popenargs, **kwargs) as process:
    549         try:
--> 550             stdout, stderr = process.communicate(input, timeout=timeout)
    551         except TimeoutExpired as exc:
    552             process.kill()

/usr/lib/python3.11/subprocess.py in communicate(self, input, timeout)
   1207 
   1208             try:
-> 1209                 stdout, stderr = self._communicate(input, endtime, timeout)
   1210             except KeyboardInterrupt:
   1211                 # https://bugs.python.org/issue25942

/usr/lib/python3.11/subprocess.py in _communicate(self, input, endtime, orig_timeout)
   2113                             'failed to raise TimeoutExpired.')
   2114 
-> 2115                     ready = selector.select(timeout)
   2116                     self._check_timeout(endtime, orig_timeout, stdout, stderr)
   2117 

/usr/lib/python3.11/selectors.py in select(self, timeout)
    413         ready = []
    414         try:
--> 415             fd_event_list = self._selector.poll(timeout)
    416         except InterruptedError:
    417             return ready

KeyboardInterrupt:

7. Having previously tried all working versions of VLLM and Docker, I came to the unambiguous conclusion that the problem is in VSCode.

8. Log

Transformers version: 4.51.3
Torch version: 2.7.0+cu126
INFO 06-12 21:02:13 [__init__.py:244] Automatically detected platform cuda.
2025-06-12 21:02:13.189772: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-06-12 21:02:13.197188: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1749762133.205913    4385 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1749762133.208566    4385 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-06-12 21:02:13.217907: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
vLLM version: 0.9.1
Kaggle Docker BUILD_DATE=20250508
Ubuntu 22.04.4 LTS
6.8.0-60-generic
               total        used        free      shared  buff/cache   available
Mem:           125Gi       4.3Gi        17Gi       9.0Mi       103Gi       119Gi
Swap:          8.0Gi       0.0Ki       8.0Gi
My NVIDIA driver version is '560.35.03'.
lrwxrwxrwx 1 root root   22 Jul 10  2024 cuda -> /etc/alternatives/cuda
lrwxrwxrwx 1 root root   25 Jul 10  2024 cuda-12 -> /etc/alternatives/cuda-12
drwxr-xr-x 1 root root 4096 Jul 10  2024 cuda-12.5
INFO 06-12 21:02:23 [config.py:823] This model supports multiple tasks: {'reward', 'generate', 'classify', 'score', 'embed'}. Defaulting to 'generate'.
WARNING 06-12 21:02:23 [config.py:3271] Casting torch.bfloat16 to torch.float16.
INFO 06-12 21:02:23 [config.py:2195] Chunked prefill is enabled with max_num_batched_tokens=8192.
WARNING 06-12 21:02:23 [config.py:2232] max_num_batched_tokens (8192) exceeds max_num_seqs* max_model_len (1024). This may lead to unexpected behavior.
WARNING 06-12 21:02:23 [cuda.py:91] To see benefits of async output processing, enable CUDA graph. Since, enforce-eager is enabled, async output processor cannot be used
WARNING 06-12 21:02:25 [env_override.py:17] NCCL_CUMEM_ENABLE is set to 0, skipping override. This may increase memory overhead with cudagraph+allreduce: https://github.com/NVIDIA/nccl/issues/1234
INFO 06-12 21:02:27 [__init__.py:244] Automatically detected platform cuda.
2025-06-12 21:02:27.403391: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:477] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1749762147.413643   48564 cuda_dnn.cc:8310] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
E0000 00:00:1749762147.416761   48564 cuda_blas.cc:1418] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
INFO 06-12 21:02:30 [core.py:455] Waiting for init message from front-end.
INFO 06-12 21:02:30 [core.py:70] Initializing a V1 LLM engine (v0.9.1) with config: model='Qwen/Qwen3-0.6B', speculative_config=None, tokenizer='Qwen/Qwen3-0.6B', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config={}, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=1024, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, enforce_eager=True, kv_cache_dtype=auto,  device_config=cuda, decoding_config=DecodingConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_backend=''), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=2024, served_model_name=Qwen/Qwen3-0.6B, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=True, use_async_output_proc=False, pooler_config=None, compilation_config={"level":0,"debug_dump_path":"","cache_dir":"","backend":"","custom_ops":[],"splitting_ops":[],"use_inductor":true,"compile_sizes":[],"inductor_compile_config":{"enable_auto_functionalized_v2":false},"inductor_passes":{},"use_cudagraph":true,"cudagraph_num_of_warmups":0,"cudagraph_capture_sizes":[],"cudagraph_copy_inputs":false,"full_cuda_graph":false,"max_capture_size":0,"local_cache_dir":null}
WARNING 06-12 21:02:30 [utils.py:2737] Methods determine_num_available_blocks,device_config,get_cache_block_size_bytes,initialize_cache not implemented in <vllm.v1.worker.gpu_worker.Worker object at 0x710235449c10>
INFO 06-12 21:02:31 [parallel_state.py:1065] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0
WARNING 06-12 21:02:31 [topk_topp_sampler.py:59] FlashInfer is not available. Falling back to the PyTorch-native implementation of top-p & top-k sampling. For the best performance, please install FlashInfer.
INFO 06-12 21:02:31 [gpu_model_runner.py:1595] Starting to load model Qwen/Qwen3-0.6B...
INFO 06-12 21:02:31 [gpu_model_runner.py:1600] Loading model from scratch...
INFO 06-12 21:02:31 [logger.py:59] Using Flash Attention backend on V1 engine.
INFO 06-12 21:02:32 [weight_utils.py:292] Using model weights format ['*.safetensors']
INFO 06-12 21:02:32 [weight_utils.py:345] No model.safetensors.index.json found in remote.
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  1.71it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:00<00:00,  1.71it/s]

INFO 06-12 21:02:33 [default_loader.py:272] Loading weights took 0.60 seconds
INFO 06-12 21:02:33 [gpu_model_runner.py:1624] Model loading took 1.1201 GiB and 1.735851 seconds
INFO 06-12 21:02:34 [gpu_worker.py:227] Available KV cache memory: 21.20 GiB
INFO 06-12 21:02:34 [kv_cache_utils.py:715] GPU KV cache size: 198,448 tokens
INFO 06-12 21:02:34 [kv_cache_utils.py:719] Maximum concurrency for 1,024 tokens per request: 193.80x
INFO 06-12 21:02:34 [core.py:171] init engine (profile, create kv cache, warmup model) took 1.06 seconds
LLM Started

[kgboyko]

There was an assumption that maybe something in Linux Ubuntu was updated. For example, the SSH library was updated. There was a computer with an old Linux that had not been updated for two months. For testing, I took an old container Kaggle Docker GPU v158 and vllm v0.7.2. More than 100 models of different LLMs were tested on this pair and everything worked.

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
  cloud-init
The following packages will be upgraded:
  apt distro-info-data docker-buildx-plugin docker-ce docker-ce-cli docker-ce-rootless-extras docker-compose-plugin grub-efi-amd64 grub-efi-amd64-bin grub-efi-amd64-signed initramfs-tools
  initramfs-tools-bin initramfs-tools-core libapt-pkg6.0 libldap-2.5-0 libldap-common libnvidia-container-tools libnvidia-container1 linux-base nvidia-container-toolkit
  nvidia-container-toolkit-base pci.ids python3-update-manager
23 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
Need to get 103 MB of archives.
After this operation, 2981 kB of additional disk space will be used.
Do you want to continue? [Y/n] n
Abort.

For better repetition of the result, I used previously saved WHL files when installing Python libraries.

! pip install -U ipykernel
! python -m pip install --no-index --find-links=/kaggle/input/whl/vllm/vllm-0-7-2 bitsandbytes vllm triton
! python -m pip install -U --no-dependencies --no-index --find-links=/kaggle/input/whl/flashinfer_python/flashinfer_python-0-2-1-post1-v02 \
    flashinfer-python==0.2.1.post1 -i https://flashinfer.ai/whl/cu121/torch2.5/

I run the example only on the first video card.

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.35.03              Driver Version: 560.35.03      CUDA Version: 12.6     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3090 Ti     Off |   00000000:01:00.0 Off |                  Off |
|  0%   32C    P8             26W /  450W |   23192MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce RTX 3090 Ti     Off |   00000000:05:00.0 Off |                  Off |
|  0%   44C    P8             15W /  450W |       2MiB /  24564MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|

Took the old Qwen LLM

import os
import transformers; print('Transformers version:', transformers.__version__)
import torch; print('Torch version:', torch.__version__)
import vllm; print('vLLM version:', vllm.__version__)
from vllm import LLM

container_date = os.environ.get('BUILD_DATE', '').split('-')[0]
print(f"Kaggle Docker BUILD_DATE={container_date}")

!cat /etc/os-release | grep -oP "PRETTY_NAME=\"\K([^\"]*)" && uname -r
!free -h
!nv_version="$(nvidia-smi --query-gpu=driver_version --format=csv,noheader)" && echo "My NVIDIA driver version is '${nv_version}'."
!ls -l /usr/local | grep cuda

is_debug = True

os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
os.environ["TRITON_PTXAS_PATH"] = "/usr/local/cuda/bin/ptxas"

if is_debug:
    os.environ['OMP_NUM_THREADS'] = '1'
    os.environ['MKL_NUM_THREADS'] = '1'
    os.environ['POLARS_ALLOW_FORKING_THREAD'] = '1'
    os.environ['CUDA_LAUNCH_BLOCKING'] = "1"
    os.environ['TORCH_USE_CUDA_DSA'] = "1"
    os.environ['OPENBLAS_NUM_THREADS'] = '1'
    os.environ["NUM_INTER_THREADS"] = "1"
    os.environ["NUM_INTRA_THREADS"] = "1"
    os.environ["XLA_FLAGS"] = ("--xla_cpu_multi_thread_eigen=false "
                               "intra_op_parallelism_threads=1")

os.environ["TOKENIZERS_PARALLELISM"] = "false"
os.environ["CUDA_VISIBLE_DEVICES"] = "0"
os.environ["PYTORCH_CUDA_ALLOC_CONF"]='expandable_segments:True'

# https://huggingface.co/casperhansen/deepseek-r1-distill-qwen-14b-awq
llm = LLM(
    model="/kaggle/input/models-llm/Deepseek-AI/DeepSeek-R1-Distill/Qwen/Quant/qwen-14b-awq-casperhansen", # "Qwen/Qwen3-0.6B",
    max_num_seqs=1,
    max_model_len=1024,
    trust_remote_code=True,
    enable_prefix_caching = True,
    dtype = torch.half,
    tensor_parallel_size=1,
    gpu_memory_utilization=0.96,
    enforce_eager=True,
    seed=2024,
)
tokenizer = llm.get_tokenizer()
print("LLM Started")

Transformers version: 4.48.3
Torch version: 2.5.1+cu121
2025-06-13 07:06:36.673294: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-06-13 07:06:36.679841: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2025-06-13 07:06:36.687425: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2025-06-13 07:06:36.689663: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-06-13 07:06:36.695956: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-06-13 07:06:37.511089: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
INFO 06-13 07:06:38 __init__.py:190] Automatically detected platform cuda.
vLLM version: 0.7.2
Kaggle Docker BUILD_DATE=20250219
13.74s - pydevd: Sending message related to process being replaced timed-out after 5 seconds
Ubuntu 22.04.3 LTS
5.15.0-136-generic
18.92s - pydevd: Sending message related to process being replaced timed-out after 5 seconds
               total        used        free      shared  buff/cache   available
Mem:            62Gi       3.6Gi        16Gi       3.0Mi        42Gi        58Gi
Swap:          8.0Gi       4.0Mi       8.0Gi
24.07s - pydevd: Sending message related to process being replaced timed-out after 5 seconds
My NVIDIA driver version is '560.35.03
560.35.03'.
29.29s - pydevd: Sending message related to process being replaced timed-out after 5 seconds
lrwxrwxrwx 1 root root   22 Nov 10  2023 cuda -> /etc/alternatives/cuda
lrwxrwxrwx 1 root root   25 Nov 10  2023 cuda-12 -> /etc/alternatives/cuda-12
drwxr-xr-x 1 root root 4096 Nov 10  2023 cuda-12.2
WARNING 06-13 07:06:59 config.py:2386] Casting torch.bfloat16 to torch.float16.

---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)
/tmp/ipykernel_2833/478751866.py in <cell line: 36>()
     34 os.environ["PYTORCH_CUDA_ALLOC_CONF"]='expandable_segments:True'
     35 
---> 36 llm = LLM(
     37     model="/kaggle/input/models-llm/Deepseek-AI/DeepSeek-R1-Distill/Qwen/Quant/qwen-14b-awq-casperhansen", # "Qwen/Qwen3-0.6B",
     38     max_num_seqs=1,

/usr/local/lib/python3.10/dist-packages/vllm/utils.py in inner(*args, **kwargs)
   1049                     )
   1050 
-> 1051             return fn(*args, **kwargs)
   1052 
   1053         return inner  # type: ignore

/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/llm.py in __init__(self, model, tokenizer, tokenizer_mode, skip_tokenizer_init, trust_remote_code, allowed_local_media_path, tensor_parallel_size, dtype, quantization, revision, tokenizer_revision, seed, gpu_memory_utilization, swap_space, cpu_offload_gb, enforce_eager, max_seq_len_to_capture, disable_custom_all_reduce, disable_async_output_proc, hf_overrides, mm_processor_kwargs, task, override_pooler_config, compilation_config, **kwargs)
    240         # to avoid import order issues
    241         self.engine_class = self.get_engine_class()
--> 242         self.llm_engine = self.engine_class.from_engine_args(
    243             engine_args, usage_context=UsageContext.LLM_CLASS)
    244 

/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py in from_engine_args(cls, engine_args, usage_context, stat_loggers)
    479         """Creates an LLM engine from the engine arguments."""
    480         # Create the engine configs.
--> 481         engine_config = engine_args.create_engine_config(usage_context)
    482         executor_class = cls._get_executor_cls(engine_config)
    483         # Create the LLM engine.

/usr/local/lib/python3.10/dist-packages/vllm/engine/arg_utils.py in create_engine_config(self, usage_context)
   1073 
   1074         device_config = DeviceConfig(device=self.device)
-> 1075         model_config = self.create_model_config()
   1076 
   1077         if (model_config.is_multimodal_model and not envs.VLLM_USE_V1

/usr/local/lib/python3.10/dist-packages/vllm/engine/arg_utils.py in create_model_config(self)
    996 
    997     def create_model_config(self) -> ModelConfig:
--> 998         return ModelConfig(
    999             model=self.model,
   1000             task=self.task,

/usr/local/lib/python3.10/dist-packages/vllm/config.py in __init__(self, model, task, tokenizer, tokenizer_mode, trust_remote_code, dtype, seed, allowed_local_media_path, revision, code_revision, rope_scaling, rope_theta, tokenizer_revision, max_model_len, spec_target_max_model_len, quantization, enforce_eager, max_seq_len_to_capture, max_logprobs, disable_sliding_window, skip_tokenizer_init, served_model_name, limit_mm_per_prompt, use_async_output_proc, config_format, hf_overrides, mm_processor_kwargs, disable_mm_preprocessor_cache, override_neuron_config, override_pooler_config, logits_processor_pattern, generation_config, enable_sleep_mode, override_generation_config, model_impl)
    362         self.served_model_name = get_served_model_name(model,
    363                                                        served_model_name)
--> 364         self.multimodal_config = self._init_multimodal_config(
    365             limit_mm_per_prompt)
    366         if not self.skip_tokenizer_init:

/usr/local/lib/python3.10/dist-packages/vllm/config.py in _init_multimodal_config(self, limit_mm_per_prompt)
    422     ) -> Optional["MultiModalConfig"]:
    423         architectures = getattr(self.hf_config, "architectures", [])
--> 424         if ModelRegistry.is_multimodal_model(architectures):
    425             return MultiModalConfig(limit_per_prompt=limit_mm_per_prompt or {})
    426 

/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/registry.py in is_multimodal_model(self, architectures)
    443         architectures: Union[str, List[str]],
    444     ) -> bool:
--> 445         model_cls, _ = self.inspect_model_cls(architectures)
    446         return model_cls.supports_multimodal
    447 

/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/registry.py in inspect_model_cls(self, architectures)
    399 
    400         for arch in architectures:
--> 401             model_info = self._try_inspect_model_cls(arch)
    402             if model_info is not None:
    403                 return (model_info, arch)

/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/registry.py in _try_inspect_model_cls(self, model_arch)
    374             return None
    375 
--> 376         return _try_inspect_model_cls(model_arch, self.models[model_arch])
    377 
    378     def _normalize_archs(

/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/registry.py in _try_inspect_model_cls(model_arch, model)
    302 ) -> Optional[_ModelInfo]:
    303     try:
--> 304         return model.inspect_model_cls()
    305     except Exception:
    306         logger.exception("Error in inspecting model architecture '%s'",

/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/registry.py in inspect_model_cls(self)
    273     # Performed in another process to avoid initializing CUDA
    274     def inspect_model_cls(self) -> _ModelInfo:
--> 275         return _run_in_subprocess(
    276             lambda: _ModelInfo.from_model_cls(self.load_model_cls()))
    277 

/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/registry.py in _run_in_subprocess(fn)
    498         # cannot use `sys.executable __file__` here because the script
    499         # contains relative imports
--> 500         returned = subprocess.run(
    501             [sys.executable, "-m", "vllm.model_executor.models.registry"],
    502             input=input_bytes,

/usr/lib/python3.10/subprocess.py in run(input, capture_output, timeout, check, *popenargs, **kwargs)
    503     with Popen(*popenargs, **kwargs) as process:
    504         try:
--> 505             stdout, stderr = process.communicate(input, timeout=timeout)
    506         except TimeoutExpired as exc:
    507             process.kill()

/usr/lib/python3.10/subprocess.py in communicate(self, input, timeout)
   1152 
   1153             try:
-> 1154                 stdout, stderr = self._communicate(input, endtime, timeout)
   1155             except KeyboardInterrupt:
   1156                 # https://bugs.python.org/issue25942

/usr/lib/python3.10/subprocess.py in _communicate(self, input, endtime, orig_timeout)
   2019                             'failed to raise TimeoutExpired.')
   2020 
-> 2021                     ready = selector.select(timeout)
   2022                     self._check_timeout(endtime, orig_timeout, stdout, stderr)
   2023 

/usr/lib/python3.10/selectors.py in select(self, timeout)
    414         ready = []
    415         try:
--> 416             fd_event_list = self._selector.poll(timeout)
    417         except InterruptedError:
    418             return ready

KeyboardInterrupt:

If you run it without a debugger, the log looks like this.

Transformers version: 4.48.3
Torch version: 2.5.1+cu121
2025-06-13 07:22:48.143368: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-06-13 07:22:48.149609: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2025-06-13 07:22:48.156711: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2025-06-13 07:22:48.158846: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-06-13 07:22:48.164362: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-06-13 07:22:48.651736: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
INFO 06-13 07:22:49 __init__.py:190] Automatically detected platform cuda.
vLLM version: 0.7.2
Kaggle Docker BUILD_DATE=20250219
Ubuntu 22.04.3 LTS
5.15.0-136-generic
               total        used        free      shared  buff/cache   available
Mem:            62Gi       3.5Gi        16Gi       3.0Mi        42Gi        58Gi
Swap:          8.0Gi       4.0Mi       8.0Gi
My NVIDIA driver version is '560.35.03
560.35.03'.
lrwxrwxrwx 1 root root   22 Nov 10  2023 cuda -> /etc/alternatives/cuda
lrwxrwxrwx 1 root root   25 Nov 10  2023 cuda-12 -> /etc/alternatives/cuda-12
drwxr-xr-x 1 root root 4096 Nov 10  2023 cuda-12.2
WARNING 06-13 07:22:49 config.py:2386] Casting torch.bfloat16 to torch.float16.
INFO 06-13 07:22:53 config.py:542] This model supports multiple tasks: {'generate', 'reward', 'embed', 'score', 'classify'}. Defaulting to 'generate'.
INFO 06-13 07:22:54 awq_marlin.py:111] The model is convertible to awq_marlin during runtime. Using awq_marlin kernel.
WARNING 06-13 07:22:54 cuda.py:95] To see benefits of async output processing, enable CUDA graph. Since, enforce-eager is enabled, async output processor cannot be used
WARNING 06-13 07:22:54 config.py:678] Async output processing is not supported on the current platform type cuda.
INFO 06-13 07:22:54 llm_engine.py:234] Initializing a V0 LLM engine (v0.7.2) with config: model='/kaggle/input/models-llm/Deepseek-AI/DeepSeek-R1-Distill/Qwen/Quant/qwen-14b-awq-casperhansen', speculative_config=None, tokenizer='/kaggle/input/models-llm/Deepseek-AI/DeepSeek-R1-Distill/Qwen/Quant/qwen-14b-awq-casperhansen', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=True, dtype=torch.float16, max_seq_len=1024, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=awq_marlin, enforce_eager=True, kv_cache_dtype=auto,  device_config=cuda, decoding_config=DecodingConfig(guided_decoding_backend='xgrammar'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=2024, served_model_name=/kaggle/input/models-llm/Deepseek-AI/DeepSeek-R1-Distill/Qwen/Quant/qwen-14b-awq-casperhansen, num_scheduler_steps=1, multi_step_stream_outputs=True, enable_prefix_caching=True, chunked_prefill_enabled=False, use_async_output_proc=False, disable_mm_preprocessor_cache=False, mm_processor_kwargs=None, pooler_config=None, compilation_config={"splitting_ops":[],"compile_sizes":[],"cudagraph_capture_sizes":[],"max_capture_size":0}, use_cached_outputs=False, 
INFO 06-13 07:22:54 cuda.py:230] Using Flash Attention backend.
INFO 06-13 07:22:55 model_runner.py:1110] Starting to load model /kaggle/input/models-llm/Deepseek-AI/DeepSeek-R1-Distill/Qwen/Quant/qwen-14b-awq-casperhansen...
[W613 07:22:55.769702180 CUDAAllocatorConfig.h:28] Warning: expandable_segments not supported on this platform (function operator())

INFO 06-13 07:22:56 model_runner.py:1115] Loading model weights took 9.3280 GB
INFO 06-13 07:22:58 worker.py:267] Memory profiling takes 1.05 seconds
INFO 06-13 07:22:58 worker.py:267] the current vLLM instance can use total_gpu_memory (23.58GiB) x gpu_memory_utilization (0.96) = 22.63GiB
INFO 06-13 07:22:58 worker.py:267] model weights take 9.33GiB; non_torch_memory takes 0.06GiB; PyTorch activation peak memory takes 0.31GiB; the rest of the memory reserved for KV Cache is 12.94GiB.
INFO 06-13 07:22:58 executor_base.py:110] # CUDA blocks: 4416, # CPU blocks: 1365
INFO 06-13 07:22:58 executor_base.py:115] Maximum concurrency for 1024 tokens per request: 69.00x
INFO 06-13 07:22:59 llm_engine.py:431] init engine (profile, create kv cache, warmup model) took 2.45 seconds
LLM Started

I just can't understand what the problem is.

Previously, there were freezes on various Deep Learning tasks, not only on LLM. VSCode does not like it when many processes are launched, or perhaps the Jupyter notebook does not like them. And when I was debugging, I made a parameter in the program to turn off all libraries that generate processes during debugging. And when debugging, I used only one process. And this always solved the problem of the debugger freezing in the Jupyter notebook on VSCode. Now this does not help, despite the fact that I specifically took the old docker and old libraries that I used in my previous work. It does not work, everything freezes.

Reduced the number of plugins to the smallest possible option and installed the old debugger plugin - did not help.

Extension	Author (truncated)	Version
jupyter-keymap	ms-	1.1.2
remote-containers	ms-	0.417.0
remote-ssh	ms-	0.120.0
remote-ssh-edit	ms-	0.87.0
vscode-remote-extensionpack	ms-	0.26.0
remote-explorer	ms-	0.5.0
remote-server	ms-	1.5.2
vscode-containers	ms-	2.0.3
vscode-docker	ms-	2.0.0
debugpy	ms-	2025.4.1
python	ms-	2025.6.1
jupyter	ms-	2025.5.0
jupyter-keymap	ms-	1.1.2
jupyter-renderers	ms-	1.1.0
vscode-jupyter-cell-tags	ms-	0.1.9
vscode-jupyter-slideshow	ms-	0.1.6

[HombreLluvia]

She also doesn't want to enter the constructor of the LLM() class, although the parameter "justMyCode": false is set.

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [

        {
            "name": "Python Debugger: Current File",
            "type": "debugpy",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "justMyCode": false
        }
    ]
}

But if I convert the program from Jupiter notebook into a regular Python program, then everything starts working, and I can fail with the debugger in the LLM() constructor.

import os
import transformers; print('Transformers version:', transformers.__version__)
import torch; print('Torch version:', torch.__version__)
import vllm; print('vLLM version:', vllm.__version__)
from vllm import LLM

if __name__ == '__main__':
    container_date = os.environ.get('BUILD_DATE', '').split('-')[0]
    print(f"Kaggle Docker BUILD_DATE={container_date}")

    #! cat /etc/os-release | grep -oP "PRETTY_NAME=\"\K([^\"]*)" && uname -r
    #! free -h
    #! nv_version="$(nvidia-smi --query-gpu=driver_version --format=csv,noheader)" && echo "My NVIDIA driver version is '${nv_version}'."
    #! ls -l /usr/local | grep cuda

    is_debug = True

    os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
    os.environ["TRITON_PTXAS_PATH"] = "/usr/local/cuda/bin/ptxas"

    if is_debug:
        os.environ['OMP_NUM_THREADS'] = '1'
        os.environ['MKL_NUM_THREADS'] = '1'
        os.environ['POLARS_ALLOW_FORKING_THREAD'] = '1'
        os.environ['CUDA_LAUNCH_BLOCKING'] = "1"
        os.environ['TORCH_USE_CUDA_DSA'] = "1"
        os.environ['OPENBLAS_NUM_THREADS'] = '1'
        os.environ["NUM_INTER_THREADS"] = "1"
        os.environ["NUM_INTRA_THREADS"] = "1"
        os.environ["XLA_FLAGS"] = ("--xla_cpu_multi_thread_eigen=false "
                                "intra_op_parallelism_threads=1")

    os.environ["TOKENIZERS_PARALLELISM"] = "false"
    os.environ["CUDA_VISIBLE_DEVICES"] = "0"
    os.environ["PYTORCH_CUDA_ALLOC_CONF"]='expandable_segments:True'

    llm = LLM(
        model="/kaggle/input/models-llm/Deepseek-AI/DeepSeek-R1-Distill/Qwen/Quant/qwen-14b-awq-casperhansen", # "Qwen/Qwen3-0.6B",
        max_num_seqs=1,
        max_model_len=8*1024,
        trust_remote_code=True,
        enable_prefix_caching = True,
        dtype = torch.half,
        tensor_parallel_size=1,
        gpu_memory_utilization=0.96,
        enforce_eager=True,
        seed=2024,
    )
    tokenizer = llm.get_tokenizer()
    print("LLM Started")

I tried the option subProcess=false when starting the debugger, but I didn't get any results.