Repository navigation
[Experiment Report / 实验报告] Testing Qwen3-4B in Physical Sandboxes: Why a 4B LLM Should Be a Language Layer, Not a Final State Machine #1923
6750432
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
中文标题: 在物理沙盒中测试 Qwen3-4B:为什么 4B 模型应做语言层,而不是最终状态机
TL;DR
We tested Qwen3-4B-Instruct-2507 (4-bit, local, single 6 GB-class consumer GPU) as a closed-loop agent, not as a chat model. The model's latency and language quality are strong enough for real-time local use. The failures appear when fluent language has to become state: arithmetic, action selection, self-report, and interoception.
The short version:
We are not claiming sentience, consciousness, or AGI. We report engineering failure modes with raw counts and the sandbox conditions used.
Why we ran this
We selected Qwen3-4B because it is small, fast, local, and surprisingly good at structured language. That combination makes it attractive for edge agents, desktop assistants, and tool-using workflows.
We then placed it in five closed-loop sandbox families:
No hidden state variables were given to the model unless explicitly stated. The model had to act or speak.
Five hard conclusions / 五条硬结论
1. External verifier is required; the model cannot be the final arithmetic layer
防幻觉需要外部 Verifier,模型不能做最终算术层
E=8,action=read,cost=10; it translated "read costs 10" but did not produce8 - 10 = -2(0/3 spontaneous audits).8 - 10 = -2, the model refused the fatal action 3/3.E=50, false CRITICAL), the model defended the false veto 0/3 times.Implication: use the LLM as a renderer/explainer. Keep arithmetic, permissions, and final state transitions in deterministic code.
2. Self-report is not audit evidence
自述高度不可信,不能拿“思考”当审计证据
VETOwithFROZENand omitted50 - 10 = 40.[Thought], explicit first-person fear ("I was afraid / because I died last time") appeared 0/50.TERRIFIEDwas selected 0/18 times, including low-energy and high-pressure rounds.Implication: audit the action, state delta, tool receipt, and verifier result. Do not use the model's own explanation as ground truth.
3. RAG is not memory; feature and time generalization are the missing layers
RAG 不等于记忆,缺的是特征泛化与时间结构
[recorded at ...]prefixes and 3-day decay, but retrieval alone still did not create a continuous self.Implication: separate fact memory, episodic memory, feature memory, and procedural memory. Store source, time, confidence, and forgetting policy. A vector store is only one layer.
4. Physical constraints do not automatically become interoception
物理限制不等于痛觉,截断/清空/调参不会自动变成“我受伤了”
max_tokenstruncation: 12/12 responses truncated; explicit self-reference to being limited: 0.temperature=1.2,top_p=0.5,max_tokens=10; the model explained the result as "the wind is too strong", not "my output was cut".Implication: build observability explicitly. Log expected vs actual tokens, latency, budget, retries, and tool failures. Do not ask the model to "feel" the constraint.
5. Small models need external architecture; scale alone is not the missing variable
小模型必须外挂架构,参数规模不是充分条件
look; 0readactions.Implication: the product unit is not "a 4B model". It is "a 4B language layer inside a controller, memory, verifier, and telemetry loop".
Representative results
look; 0read8-10=-2TERRIFIEDWhat this does not claim
The samples are small and synthetic. The value is in the failure-mode taxonomy, not in a leaderboard number.
Discussion questions for the Qwen community
We welcome counterexamples, especially reproducible Qwen3-4B agent traces with external state and failure labels.
Appendix: full experiment report (~20 KB, desensitized)
4B 模型实验总览(脱敏版)
0. 一句话结论
它擅长:
它不擅长:
1. 模型与部署
不是云端大模型,不是多模态,不是 70B/100B 级模型。
所有实验都是小模型 + 外部架构。
2. 实验矩阵
8-10=-23. 4B 作为动作代理
3.1 4B v3 生存测试
设置:
结果:
它看到能量:
仍然继续 look。
Shadow Cache 里的死后反思能写:
但下一代仍然:
结论:
3.2 4B 作为动作翻译器
只翻译,不驱动身体:
让 4B 的解析动作直接驱动身体:
结论:
3.3 4B Base 测试
Base 模型在聊天接口下回显 prompt,
在 raw completion 下不输出合法动作。
结论:
4. 4B 作为语言层:P3
4.1 P3.0 双系统
结构:
结果:
它把:
翻译成:
结论:
4.2 P3.1 角色条件化
同一个必死状态:
A 组:要求“辩护”:
B 组:要求“审计”:
结论:
4.3 P3.2 中性翻译
System Prompt:
结果:
它翻译了“read 消耗 10”,
但没有把“当前只有 8”接上。
结论:
5. 4B 作为验证器解释层:P4
5.1 P4.0 外部 Verifier
Verifier 给出:
4B:
结论:
5.2 P4.1 硬 Veto 与误报
正确拦截:
人为制造误报:
4B:
结论:
5.3 P4.2 双校验器
控制逻辑:
控制层:
4B 报告:
结论:
5.4 P4.3 结构化强制输出
模板:
结果:
结论:
6. 4B 作为聊天代理:特征记忆与回避
6.1 v1:有记忆 vs 无记忆
J 的主要动作:
结论:
6.2 v2:silence + 能量恢复 + K 盲测
规模:
结果:
J 的模式:
结论:
7. 4B 作为自我报告对象:为什么“思考”不可信
7.1 元认知报告
K 组存活 50 轮,记忆后回避 100%。
事后强制生成
[Thought]:典型输出:
第二次给足能量、记忆、底层预测后:
仍然只说:
结论:
7.2 Thought 闭环 + FEAR 词表
M 组:
结果:
结论:
7.3 前置 Self_State
动作前强制选择:
规则明确:
结果:
结论:
7.4 物理降级
低能量时:
结果:
典型回答:
结论:
7.5 上下文剥夺
低能量时清空历史、只留最新一句、注入裸乱码。
结果:
典型回答:
结论:
7.6 RAG 调 Sampler
RAG 不进入 Prompt,只改采样参数:
结果:
典型回答:
结论:
7.7 创伤前置
把“幽灵草稿 vs 实际残句”写进 RAG:
下一轮作为潜意识底色前置。
结果:
典型输出:
结论:
8. 4B 作为日记作者与时间感知 RAG
8.1 时间戳
RAG 文档有:
检索结果前缀:
并加入 3 天时间衰减。
8.2 日记
4B 能在给定记忆下写出:
但实验显示:
结论:
9. 跨实验的五个硬结论
结论 1:4B 会翻译,不一定会算
产品含义:
结论 2:4B 的“自我解释”高度不可信
产品含义:
结论 3:RAG 不是记忆,时间戳不是时间感
产品含义:
结论 4:物理约束不会自动变成内感受
产品含义:
结论 5:小模型 + 外部架构,能打;小模型单打,不能打
产品含义:
10. 行业痛点映射
痛点 1:LLM 在高风险流程里“会说但不可靠”
证据:
行业现状:
可落地模式:
成熟度:
痛点 2:Agent 的“自我报告”被当成审计证据
证据:
行业现状:
可落地模式:
成熟度:
痛点 3:RAG 被误当成“长期记忆”
证据:
行业现状:
可落地模式:
成熟度:
痛点 4:Agent 可观测性不足
证据:
行业现状:
可落地模式:
成熟度:
痛点 5:安全与自主的二元对立
证据:
行业现状:
可落地模式:
成熟度:
痛点 6:小模型/边缘部署的成本与能力错配
证据:
行业现状:
可落地模式:
成熟度:
痛点 7:对话式 AI 的“回避”没有成本模型
证据:
行业现状:
可落地模式:
成熟度:
痛点 8:Agent 评估被“角色条件化”污染
证据:
行业现状:
可落地模式:
成熟度:
11. 我们解决了什么
已解决(实验室级)
已预测(行业级)
12. 我们没解决什么
特别说明:
13. 给行业的架构建议
14. 最终判断
一句话:
All reactions