Skip to content

[Modular Converter] Strip @use_kernelized_func(fn) when fn lacks @use_kernel_func_from_hub#46437

Open
kpal002 wants to merge 2 commits into
huggingface:mainfrom
kpal002:fix/modular-converter-strip-orphaned-kernelized-func
Open

[Modular Converter] Strip @use_kernelized_func(fn) when fn lacks @use_kernel_func_from_hub#46437
kpal002 wants to merge 2 commits into
huggingface:mainfrom
kpal002:fix/modular-converter-strip-orphaned-kernelized-func

Conversation

@kpal002
Copy link
Copy Markdown
Contributor

@kpal002 kpal002 commented Jun 5, 2026

What this PR does

Fixes the root cause of the use_kernels=True crash on Qwen3Next and Qwen3.5 (issue #46399).

Root cause

replace_class_node() falls back to the parent class decorators when a modular subclass declares none. For Qwen3NextAttention(Qwen3MoeAttention) and Qwen3_5Attention, this means @use_kernelized_func(apply_rotary_pos_emb) is inherited from Qwen3MoeAttention — but the generated file defines apply_rotary_pos_emb as a partial-RoPE variant without @use_kernel_func_from_hub. When kernelize() runs, attach_hidden_kernels() tries to register that plain function as an nn.Module and raises:

ValueError: Attempted to register a kernel for apply_rotary_pos_emb,
but it was not a torch.nn.Module.

Fix

Add strip_orphaned_kernelized_func_decorators(), called in create_modules() before get_needed_imports().

For each generated file's body dict it:

  1. Collects all module-level function names and identifies which have @use_kernel_func_from_hub
  2. For any class with @use_kernelized_func(fn) where fn is defined locally without @use_kernel_func_from_hub, drops that decorator

Calling it before get_needed_imports means the existing LibCST scope analysis also removes the now-unused use_kernelized_func import automatically — no separate cleanup needed.

What is NOT changed

  • No model files are touched here. The regeneration of modeling_qwen3_next.py and modeling_qwen3_5.py (using this fixed converter) is in a follow-up PR to keep the diff reviewable.
  • No kernel is added for partial RoPE. That belongs in a future PR once kernels-community#919 ships.

PR series for #46399

PR Status Description
#46435 Open Regression tests (currently failing — will pass after this)
This PR Open Converter fix
PR 3 Pending Regenerate modeling_qwen3_next.py and modeling_qwen3_5.py

cc @vasqu @ArthurZucker @MekkCyber @drbh

Before submitting

kuntal-beehive and others added 2 commits June 5, 2026 01:33
…_kernel_func_from_hub

replace_class_node() falls back to the parent class decorators when a modular
subclass declares none. For Qwen3Next and Qwen3.5 this causes @use_kernelized_func(
apply_rotary_pos_emb) to be inherited from Qwen3MoeAttention even though the
generated file defines apply_rotary_pos_emb as a partial-RoPE variant without
@use_kernel_func_from_hub. When kernelize() runs, attach_hidden_kernels() tries to
register that plain function as an nn.Module and raises ValueError.

Fix: add strip_orphaned_kernelized_func_decorators(), called in create_modules()
before get_needed_imports(). It drops @use_kernelized_func(fn) from any class in
the generated body when fn is defined locally without @use_kernel_func_from_hub.
Placing it before get_needed_imports lets the existing scope analysis also remove
the now-unused use_kernelized_func import automatically.

Regression tests: huggingface#46435
Regen of affected model files: follow-up PR
Fixes huggingface#46399
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants