Case Study: Building a controllable Linux Ops Agent on 6GB VRAM with Qwen3-4B

[GTLordLL]

Hi Qwen Team,

I’m excited to share Cutin Agent (千务小切), an open-source SOP-driven Agent framework specifically optimized for Qwen3-4B in resource-constrained environments (e.g., RTX 3060 6GB VRAM).

While most Agent frameworks rely on high-parameter models (14B+) or cloud APIs for autonomous decision-making, Cutin Agent takes a different path: LLM as a "Digital SOP Executor" rather than a "Free Decision Maker."

🚀 Key Innovations for Qwen3-4B:

1. Thinker + Formatter Dual-Stage Inference: We decoupled reasoning (Temp 0.4) and formatting (Temp 0.0). This approach achieved 100% formatting accuracy on the 4B model, eliminating the common JSON/tool-call hallucination issues in small models.
2. SOP-as-Code Architecture: Using LangGraph, we implemented a 3-node system (SOP Retriever, Tool Caller, Progress Summarizer). The LLM strictly follows structured Markdown SOPs, ensuring 100% predictable behavior in sensitive Linux Ops environments.
3. Expert Tool Loops: Instead of returning raw terminal logs, our 11 built-in tools perform internal "collect-analyze-judge" cycles, returning high-level natural language conclusions. This significantly reduces the reasoning and token load on the 4B model.

📊 Real-world Performance (Full System Health Check):

• Environment: Local Ubuntu 24.04, RTX 3060 6GB (Ollama/Qwen3:4b).
• Workflow: 3 main logic steps (System health probe + Conditional process check + Report generation).
• Result: Successfully completed the multi-step task in 2 rounds (126.77s). The 4B model accurately handled conditional logic (skipping a step when resource usage was low) and terminal state detection (FINISH).

Using FULL_SYSTEM_HEALTH_CHECK as an example, here's how Cutin Agent completes a comprehensive system diagnosis in 2 loop rounds, 127 seconds:

Task: "Run a comprehensive system health check"

## Plan_Steps
1. Simultaneously call get_stystem_cealth (target='all') and check_stystem_stync() to collect system resource metrics (CPU, memory, load, disk) and clock synchronization status in parallel.
2. If the CPU usage exceeds 80% or the memory usage exceeds 90% in the system resource data of step 1, call list_topsprocesses (sort-by='cpu ', limit=10) to identify the list of processes with the highest current resource usage. If all indicators are normal, skip this step.
3. Call generate_summary.rport to summarize all collected data and generate a system health check report. The report should indicate the usage rate of each resource, abnormal processes, and time synchronization status.
4. FINISH。

##Global exception handling
When a tool call in any step returns a 'failed' status, follow the following strategy:
1. If get_System_cealth or check_System_Sync (step 1) fails, mark it as ERROR and terminate, informing the user to check system permissions.
If list_top_decesses (step 2) fails, mark it as ERROR and terminate, informing the user that the process list collection has failed.
If generate_summary.rport (step 3) fails, output the raw data collected in steps 1 and 2 as a substitute report, marked as ERROR.
If the tool returns a failure for two consecutive steps, mark it as ERROR and terminate, output the collected partial data and recommend manual intervention.

[Round 1]
initial_sop_retriever (7.79s)  → Matched SOP: FULL_SYSTEM_HEALTH_CHECK
tool_caller_v2       (17.85s) → Located Step 1: get_system_health + check_system_sync
tool_executor        (0.50s)  → CPU 2.9% | Memory 70.4% | Load normal | Disk 41.6%
progress_summarizer  (23.31s) → Step 1 marked "已完成" (completed), results saved to SOP_PLAN

[Round 2]
tool_caller_v2       (27.64s) → Condition check: CPU<80% AND Memory<90% → Skip Step 2
                                 Located Step 3: generate_summary_report
tool_executor        (12.24s) → Generated detailed health report (tables/trends/recommendations)
progress_summarizer  (24.44s) → Step 3 marked "已完成"
tool_caller_v2       (13.00s) → All steps complete → FINISH

Total: 126.77s | ~3000 tokens/round | Running on RTX 3060 6GB

💡 Feedback for the Team:

Our tests show that Qwen3-4B is remarkably stable at following structured SOPs. However, when tasks scale beyond 6 logical steps, the model occasionally struggles with precise step-tracking in long contexts. We believe that fine-tuning 4B-class models specifically on "Sequential SOP Execution" and "State Persistence" could unlock professional-grade automation on consumer-grade hardware.

Repo: https://github.com/GTLordLL/CutinAgent

I’d love to hear your thoughts on this "Low-cost, High-controllability" approach for local LLMs!