OCR: GLM-OCR model produces garbage output via MLXVLM

[kiki830621]

Problem

macdoc ocr pipeline runs end-to-end without crashes, but the OCR output is meaningless token repetitions instead of recognized text.

Type

bug

Expected

Running macdoc ocr /tmp/ocr-test.png on an image containing "Hello OCR Test 你好世界" should produce readable Markdown text.

Actual

Output is repeated garbage tokens:

刷刷刷刷刷刷一闪ount刷刷刷刷刷一闪ount刷刷刷刷刷刷...

Or with longer max-tokens:

encyencyencyencyency...指指指指指...Span Span Span...

Context

• Model: EZCon/GLM-OCR-8bit-mlx (8-bit quantized GLM-OCR for MLX)
• Pipeline: OCRPipeline → VLMModelFactory.shared.loadContainer() → MLXLMCommon.generate()
• The MLXVLM library has a GlmOcr.swift implementation, so model architecture should be correct
• Possible causes:
  1. The quantized model (EZCon/GLM-OCR-8bit-mlx) itself may be broken
  2. Image preprocessing (CIImage conversion) may not match what the model expects
  3. Chat template / prompt format may be wrong for this model
  4. Need to test with a known-working model (e.g., mlx-community/Qwen2.5-VL-3B-Instruct-4bit)

Impact

macdoc ocr is unusable until this is resolved — the entire OCR feature depends on correct VLM output.

Next Steps

• Test with mlx-community/Qwen2.5-VL-3B-Instruct-4bit to verify the pipeline works with a different model
• If Qwen works, the issue is GLM-OCR specific (model or config)
• If Qwen also fails, the issue is in OCRPipeline's image/prompt handling

[kiki830621]

Diagnosis

Type

bug

Root Cause

Two possible causes (need to isolate):

1. Model-specific issue: `EZCon/GLM-OCR-8bit-mlx` is a third-party 8-bit quantization — not from `mlx-community`. The quantization may have degraded quality. MLXVLM has no pre-registered GLM-OCR model in its `VLMRegistry`, meaning this model hasn't been validated by the Apple team.

2. Pipeline issue unlikely but possible: `OCRPipeline` uses `CIImage(cgImage:)` which is supported by `UserInput.Image`. The prompt "OCR the text in this image." may not match what the model's chat template expects. The `llm-tool` reference uses `ChatSession` for multi-turn, but single-turn should also work.

Evidence

• MLXVLM's `GlmOcr.swift` (1262 lines) has full architecture support including Conv3D patch embed, 2D RoPE, spatial merge via Conv2D downsample, and the GlmOcr-specific merger
• The `Glm46VProcessor` handles image preprocessing internally — no manual image prep needed
• The garbage output pattern (repeated tokens) suggests either weight corruption or wrong tokenizer/chat template pairing

Strategy

• Step 1: Test with a known-working model to isolate pipeline vs model issue:
  ```bash
  hf download mlx-community/Qwen2.5-VL-3B-Instruct-4bit
  macdoc ocr test.png --model mlx-community/Qwen2.5-VL-3B-Instruct-4bit
  ```
• Step 2: If Qwen works → problem is `EZCon/GLM-OCR-8bit-mlx` specific
  • Try `mlx-community` GLM-OCR model if one exists
  • Or re-quantize from the original model
• Step 3: If Qwen also fails → debug `OCRPipeline.processImage` (UserInput, prompt, generate params)
• Step 4: Verify safetensors header is valid (not zero-filled, see OCR: hf CLI xet download produces corrupted (zero-byte) safetensors #68)

Risks

• `EZCon/GLM-OCR-8bit-mlx` may simply be a bad quantization — out of our control
• GLM-OCR as a model may require specific prompt format not handled by MLXVLM's processor