[Docs] Add supported model tables to pretrain_sft advanced tutorial by CyCle1024 · Pull Request #1728 · InternLM/xtuner

CyCle1024 · 2026-04-28T10:25:12Z

Summary

This PR adds comprehensive supported model documentation to the pretrain/sft advanced tutorial, replacing the previous "Coming soon..." placeholder.

Changes

English model doc ()
- Base Config Classes table (5 families)
- Concrete Model Configs table (15 configs)
- Compose Models section with base & concrete tables (10 configs)
- Complete Inheritance Hierarchy tree covering both and branches
Chinese model doc ()
- Fully synchronized with the English version
Skill for auto-sync ()
- : workflow guide for AI to update docs when new model configs are added
- : script to scan and discover all Config classes with inheritance info
- Symlinked from for skill discovery

Supported models covered

Dense: Qwen2 (7B), Qwen3 (0.6B / 4B / 8B)
MoE: DeepSeek-V3, GPT-OSS (21B / 117B), Qwen3-MoE (30B / 235B / FoPE)
VL text backbones: Qwen3 VL Dense (4B / 8B), Qwen3 VL MoE (30B / 235B), Qwen3.5 VL MoE (35B)
Compose models: Qwen3-VL, Qwen3.5-VL, InternVL, InternS1

…ted-models skill - Add model support tables to en/zh pretrain_sft advanced tutorial model.md - Include base configs, concrete configs, compose models and inheritance hierarchy - Add xtuner-sync-supported-models skill under .claude/skills/ with scan script

CyCle1024 · 2026-04-28T10:30:27Z

@claude review

claude · 2026-04-28T10:35:46Z

+| `Qwen3VLTextDense4BConfig` | `Qwen3DenseConfig` | Dense (VL backbone) | ~4B parameters, for multimodal |
+| `Qwen3VLTextDense8BConfig` | `Qwen3DenseConfig` | Dense (VL backbone) | ~8B parameters, for multimodal |
+| `DeepSeekV3Config` | — | MoE | ~671B total / ~37B activated |
+| `GptOss21BA3P6Config` | `GptOssConfig` | MoE | ~21B total / ~3.6B activated |
+| `GptOss117BA5P8Config` | `GptOssConfig` | MoE | ~117B total / ~5.8B activated |
+| `Qwen3MoE30BA3Config` | `Qwen3MoEConfig` | MoE | ~30B total / ~3B activated |


Claude: Critical — incorrect inheritance in table and tree

The "Base Class / Family" for these VL text backbone configs is listed as Qwen3DenseConfig / Qwen3MoEConfig, but the actual direct parents are the concrete configs:

Config Listed parent Actual parent

Qwen3VLTextDense4BConfig Qwen3DenseConfig Qwen3Dense4BConfig

Qwen3VLTextDense8BConfig Qwen3DenseConfig Qwen3Dense8BConfig

Qwen3VLTextMoE30BA3Config Qwen3MoEConfig Qwen3MoE30BA3Config

Qwen3VLTextMoE235BA22Config Qwen3MoEConfig Qwen3MoE235BA22Config

Source references:

xtuner/v1/model/dense/qwen3vl_text.py:96: class Qwen3VLTextDense4BConfig(Qwen3Dense4BConfig)

xtuner/v1/model/moe/qwen3vl_text.py:234: class Qwen3VLTextMoE30BA3Config(Qwen3MoE30BA3Config)

The same error appears in the inheritance hierarchy tree at the bottom of this file (and in the Chinese version, the SKILL.md file).

@claude here we only find the base class of such config, and it can indicate the config family of the config, if you agreed with me, please resolve this conversation.

claude · 2026-04-28T10:35:52Z

+├── TransformerConfig
+│   ├── Dense Models
+│   │   ├── Qwen2DenseConfig
+│   │   │   └── Qwen2Dense7BConfig
+│   │   └── Qwen3DenseConfig
+│   │       ├── Qwen3Dense8BConfig
+│   │       ├── Qwen3Dense4BConfig
+│   │       ├── Qwen3Dense0P6BConfig
+│   │       ├── Qwen3VLTextDense4BConfig
+│   │       └── Qwen3VLTextDense8BConfig
+│   └── MoE Models (via MoEConfig)


Claude: Critical — inheritance tree does not match actual code

The VL text backbone configs are shown as direct children of the family base, but they actually inherit from the concrete scale configs. The corrected tree should be:

Suggested change

├── TransformerConfig

│ ├── Dense Models

│ │ ├── Qwen2DenseConfig

│ │ │ └── Qwen2Dense7BConfig

│ │ └── Qwen3DenseConfig

│ │ ├── Qwen3Dense8BConfig

│ │ ├── Qwen3Dense4BConfig

│ │ ├── Qwen3Dense0P6BConfig

│ │ ├── Qwen3VLTextDense4BConfig

│ │ └── Qwen3VLTextDense8BConfig

│ └── MoE Models (via MoEConfig)

│ ├── Dense Models

│ │ ├── Qwen2DenseConfig

│ │ │ └── Qwen2Dense7BConfig

│ │ └── Qwen3DenseConfig

│ │ ├── Qwen3Dense8BConfig

│ │ │ └── Qwen3VLTextDense8BConfig

│ │ ├── Qwen3Dense4BConfig

│ │ │ └── Qwen3VLTextDense4BConfig

│ │ └── Qwen3Dense0P6BConfig

Similarly for the MoE section below, Qwen3VLTextMoE30BA3Config should be nested under Qwen3MoE30BA3Config, and Qwen3VLTextMoE235BA22Config under Qwen3MoE235BA22Config.

claude · 2026-04-28T10:35:54Z

+}
+
+
+def scan_file(path: Path):


Claude: Warning — missing type hints on function signatures

Per CLAUDE.md: "All new code must include type hints for function signatures (parameters and return types)."

scan_file and main are both missing return type annotations and parameter type hints:

Suggested change

def scan_file(path: Path):

def scan_file(path: Path) -> list[dict[str, str | list[str]]]:

And main should have -> None.

claude · 2026-04-28T10:35:57Z

+    children: dict[str, list[str]] = {}
+    for cfg in all_configs:
+        for p in cfg["parents"]:


Claude: Nit — RELEVANT_BASES set is redundant

The condition p in RELEVANT_BASES or p.endswith("Config") always takes the right branch for every class in the codebase since all relevant parent classes already end with Config. The RELEVANT_BASES set could be removed and the condition simplified to just p.endswith("Config").

claude · 2026-04-28T10:36:13Z

Claude: ## Summary

Docs PR that replaces "Coming soon…" placeholders in the EN/ZH model advanced tutorial with comprehensive supported-model tables, inheritance hierarchy, and a Claude skill + scan script for keeping them in sync.

Issues

Critical

docs/en/pretrain_sft/advanced_tutorial/model.md & inheritance tree: Four VL text backbone configs have incorrect parent classes in both the Concrete Model table and the inheritance hierarchy tree. The same error is repeated in the Chinese model.md and in SKILL.md. Specifically:
- Qwen3VLTextDense4BConfig inherits from Qwen3Dense4BConfig, not Qwen3DenseConfig
- Qwen3VLTextDense8BConfig inherits from Qwen3Dense8BConfig, not Qwen3DenseConfig
- Qwen3VLTextMoE30BA3Config inherits from Qwen3MoE30BA3Config, not Qwen3MoEConfig
- Qwen3VLTextMoE235BA22Config inherits from Qwen3MoE235BA22Config, not Qwen3MoEConfig

Warning

scan_model_configs.py:29: scan_file and main functions missing type hints (required by CLAUDE.md)

Nit

scan_model_configs.py:53-55: RELEVANT_BASES set is redundant given the p.endswith("Config") fallback

Verdict

REQUEST_CHANGES — the inheritance hierarchy errors affect 4 classes across 3 files (EN doc, ZH doc, SKILL.md) and would mislead users about the actual model config structure.

jayhenry · 2026-04-29T13:54:50Z

+## 继承关系
+
+下图展示了 `TrainEngine` 支持的所有配置类的完整继承层级，包括 `TransformerConfig` 和 `BaseComposeConfig` 两大分支。
+


claude Bot reviewed Apr 28, 2026

View reviewed changes

CyCle1024 requested review from HAOCHENYE and jayhenry April 28, 2026 14:26

jayhenry approved these changes Apr 29, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Docs] Add supported model tables to pretrain_sft advanced tutorial#1728

[Docs] Add supported model tables to pretrain_sft advanced tutorial#1728
CyCle1024 wants to merge 1 commit intoInternLM:mainfrom
CyCle1024:docs/add-supported-models-table

CyCle1024 commented Apr 28, 2026

Uh oh!

CyCle1024 commented Apr 28, 2026

Uh oh!

claude Bot Apr 28, 2026

Uh oh!

CyCle1024 Apr 28, 2026

Uh oh!

claude Bot Apr 28, 2026

Uh oh!

claude Bot Apr 28, 2026

Uh oh!

claude Bot Apr 28, 2026

Uh oh!

claude Bot commented Apr 28, 2026

Uh oh!

jayhenry Apr 29, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Config	Listed parent	Actual parent
`Qwen3VLTextDense4BConfig`	`Qwen3DenseConfig`	`Qwen3Dense4BConfig`
`Qwen3VLTextDense8BConfig`	`Qwen3DenseConfig`	`Qwen3Dense8BConfig`
`Qwen3VLTextMoE30BA3Config`	`Qwen3MoEConfig`	`Qwen3MoE30BA3Config`
`Qwen3VLTextMoE235BA22Config`	`Qwen3MoEConfig`	`Qwen3MoE235BA22Config`

	def scan_file(path: Path):
	def scan_file(path: Path) -> list[dict[str, str \| list[str]]]:

		## 继承关系

		下图展示了 `TrainEngine` 支持的所有配置类的完整继承层级，包括 `TransformerConfig` 和 `BaseComposeConfig` 两大分支。

Conversation

CyCle1024 commented Apr 28, 2026

Summary

Changes

Supported models covered

Uh oh!

CyCle1024 commented Apr 28, 2026

Uh oh!

claude Bot Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

CyCle1024 Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot Apr 28, 2026

Choose a reason for hiding this comment

Uh oh!

claude Bot commented Apr 28, 2026

Issues

Critical

Warning

Nit

Verdict

Uh oh!

jayhenry Apr 29, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants