DeepSeek chat completion returns empty/invalid content causing dirtyjson parse failure

[KeepMovingNo1]

When using DeepSeek chat completion API to generate structured slide content for PPT creation, the request returns HTTP 200 OK, but the response content is empty or not valid JSON, which causes a runtime exception during JSON parsing.

This issue breaks the PPT generation workflow and results in a 422 Unprocessable Entity error from the FastAPI endpoint.

2025-12-25 01:23:04,830 - INFO - HTTP Request: POST https://api.deepseek.com/v1/chat/completions "HTTP/1.1 200 OK"
Traceback (most recent call last):
File "/app/servers/fastapi/utils/llm_calls/generate_slide_content.py", line 129, in get_slide_content_from_type_and_outline
response = await client.generate_structured(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/servers/fastapi/services/llm_client.py", line 816, in generate_structured
content = await self._generate_custom_structured(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/servers/fastapi/services/llm_client.py", line 759, in _generate_custom_structured
return await self._generate_openai_structured(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/servers/fastapi/services/llm_client.py", line 558, in _generate_openai_structured
return dict(dirtyjson.loads(content))
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/dirtyjson/init.py", line 94, in loads
return d.decode(search_for_first_object, start_index)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/dirtyjson/loader.py", line 372, in decode
obj = self.scan()
^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/dirtyjson/loader.py", line 207, in scan
raise Error(self.expecting, self.content, self.pos)
dirtyjson.error.Error: Expecting value: line 1 column 1 (char 0)
INFO: 127.0.0.1:52052 - "PATCH /api/v1/ppt/presentation/update HTTP/1.0" 422 Unprocessable Entity
INFO: 127.0.0.1:52056 - "PATCH /api/v1/ppt/presentation/update HTTP/1.0" 422 Unprocessable Entity

[KeepMovingNo1]

In some cases, the system is able to successfully generate and display several slides/pages of the PPT. However, during the same presentation generation process, subsequent slide content generation may fail with an empty or invalid response from the DeepSeek API, eventually causing the entire update request to return 422 Unprocessable Entity.

[614030751]

我也遇到了这个问题请问有办法解决嘛

[KeepMovingNo1]

我也遇到了这个问题请问有办法解决嘛

还没有 感觉是deepseek 返回的结果不符合格式导致，感觉可以将根据内容渲染PPT的流程重试几次？我这边还没尝试

[614030751]

就是deepseek返回的object_schema,而项目要求json_schema没做适配应该是

[wind125]

我在使用本地部署的qwen3-30B也出现了该问题
INFO: 127.0.0.1:34840 - "GET /api/v1/ppt/template-management/summary HTTP/1.0" 200 OK
2026-01-07 09:55:48,602 - INFO - HTTP Request: POST http://xx.xx.xx.xx:18007/v1/chat/completions "HTTP/1.1 200 OK"
Traceback (most recent call last):
File "/app/servers/fastapi/utils/llm_calls/generate_presentation_structure.py", line 112, in generate_presentation_structure
response = await client.generate_structured(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/servers/fastapi/services/llm_client.py", line 823, in generate_structured
content = await self._generate_custom_structured(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/servers/fastapi/services/llm_client.py", line 766, in _generate_custom_structured
return await self._generate_openai_structured(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/servers/fastapi/services/llm_client.py", line 565, in _generate_openai_structured
return dict(dirtyjson.loads(content))
^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/dirtyjson/init.py", line 94, in loads
return d.decode(search_for_first_object, start_index)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/dirtyjson/loader.py", line 372, in decode
obj = self.scan()
^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/dirtyjson/loader.py", line 207, in scan
raise Error(self.expecting, self.content, self.pos)
dirtyjson.error.Error: Expecting value: line 1 column 1 (char 0)
INFO: 127.0.0.1:34846 - "POST /api/v1/ppt/presentation/prepare HTTP/1.0" 500 Internal Server Error

[Charlotte5622]

I have the exact same issue.

[wind125]

我初步认为是自己部署的模型不支持openai接口中的response_format参数，我通过写一个代理服务，将response_format的要求提取到系统提示词中，流程能跑通了

[KeepMovingNo1]

我初步认为是自己部署的模型不支持openai接口中的response_format参数，我通过写一个代理服务，将response_format的要求提取到系统提示词中，流程能跑通了

怎么处理的呢 能发出来瞅瞅吗 我有空也试下 学习下哈哈

[MZerg]

✅ Solution: Custom LLM Proxy for response_format Support

✅ 解决方案：支持自定义 LLM 的 response_format 代理

Problem Summary / 问题概述

When using Presenton with custom OpenAI-compatible LLM endpoints that don't support the response_format parameter (such as local models, self-hosted APIs, or certain cloud providers), presentation generation fails with JSON parsing errors:

当使用不支持 response_format 参数的自定义 OpenAI 兼容 LLM 端点（如本地模型、自托管 API 或某些云服务商）时，演示文稿生成会因 JSON 解析错误而失败：

Failed to generate presentation outlines
LLM API error: Expecting value: line 1 column 1 (char 0)

Or / 或者:

LLM API error: Expecting ',' delimiter: line 3 column 88 (char 113)

Root Causes / 根本原因

Cause 1 / 原因 1: Presenton uses OpenAI's response_format parameter to enforce structured JSON output:

Presenton 使用 OpenAI 的 response_format 参数来强制结构化 JSON 输出：

response_format={
    "type": "json_schema",
    "json_schema": {
        "name": "ResponseSchema",
        "strict": True,
        "schema": {...}
    }
}

Many LLM APIs don't support this parameter and either ignore it or return errors.

许多 LLM API 不支持此参数，要么忽略它，要么返回错误。

Cause 2 / 原因 2: Even when LLMs attempt to follow JSON schema instructions in prompts, they often return responses with formatting issues:

即使 LLM 试图遵循提示词中的 JSON schema 指令，它们通常也会返回带有格式问题的响应：

1. Markdown code blocks / Markdown 代码块: Wrapping JSON in ```json ... ``` / 用 ```json ... ``` 包装 JSON
2. Localized punctuation / 本地化标点符号: Using language-specific quotes (e.g., Chinese "text" instead of English "text") / 使用特定语言的引号（例如中文的 "文本" 而不是英文的 "text"）
3. Unescaped quotes / 未转义的引号: String values containing unescaped " characters that break JSON parsing / 字符串值包含未转义的 " 字符，导致 JSON 解析失败

Complete Solution / 完整解决方案

Create a lightweight HTTP proxy that sits between Presenton and your LLM API to:

创建一个位于 Presenton 和您的 LLM API 之间的轻量级 HTTP 代理，用于：

1. Extract response_format from requests / 提取请求中的 response_format
2. Convert it to natural language instructions / 转换为自然语言指令
3. Inject instructions into the system prompt / 注入指令到系统提示词
4. Clean LLM responses to ensure valid JSON / 清理 LLM 响应以确保有效的 JSON
5. Forward to your actual LLM endpoint / 转发到您的实际 LLM 端点

Complete Proxy Implementation / 完整代理实现

Save this as proxy.py:

将此保存为 proxy.py：

"""
OpenAI-compatible proxy that converts response_format to prompt instructions
and cleans LLM responses for reliable JSON parsing.

OpenAI 兼容代理，将 response_format 转换为提示词指令，
并清理 LLM 响应以实现可靠的 JSON 解析。
"""
import json
import logging
import os
from typing import Optional

import httpx
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse, StreamingResponse

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
app = FastAPI()

# Configuration / 配置
UPSTREAM_LLM_URL = os.environ.get("UPSTREAM_LLM_URL", "http://localhost:11434")
UPSTREAM_API_KEY = os.environ.get("UPSTREAM_API_KEY", "")

# Global shared HTTP client / 全局共享 HTTP 客户端
http_client = httpx.AsyncClient(timeout=httpx.Timeout(300.0, connect=10.0))


def build_format_instruction(response_format: dict) -> str:
    """
    Convert OpenAI response_format to natural language instructions.
    将 OpenAI response_format 转换为自然语言指令。
    """
    if not response_format:
        return ""
    
    fmt_type = response_format.get("type", "")
    
    if fmt_type == "json_object":
        return (
            "\n\n**CRITICAL**: Your entire response must be ONLY a valid JSON object. "
            "Start immediately with { and end with }. "
            "Do not include any explanation, markdown formatting, or other text before or after the JSON."
        )
    
    if fmt_type == "json_schema" and "json_schema" in response_format:
        js = response_format["json_schema"]
        schema = js.get("schema", js)
        name = js.get("name", "ResponseSchema")
        strict = js.get("strict", False)
        
        strict_hint = " Do not include any properties not defined in the schema." if strict else ""
        
        return (
            f"\n\n**CRITICAL**: Your entire response must be ONLY a valid JSON object.{strict_hint} "
            f"Start immediately with {{ and end with }}. "
            f"Do not include any explanation, thinking process, markdown code blocks, or other text before or after the JSON. "
            f"The JSON must conform to this schema (name: {name}):\n"
            f"```json\n{json.dumps(schema, ensure_ascii=False, indent=2)}\n```"
        )
    
    return ""


def clean_markdown_json(content: str) -> str:
    """
    Clean LLM response to ensure valid JSON:
    1. Remove markdown code block wrappers
    2. Replace localized punctuation with standard ASCII
    3. Escape unescaped quotes in JSON string values
    
    清理 LLM 响应以确保有效的 JSON：
    1. 移除 markdown 代码块包装
    2. 将本地化标点替换为标准 ASCII
    3. 转义 JSON 字符串值中的未转义引号
    """
    if not content:
        return content
    
    content = content.strip()
    
    # Step 1: Remove markdown code blocks / 步骤 1：移除 markdown 代码块
    if content.startswith("```json"):
        content = content[7:].lstrip()
    elif content.startswith("```"):
        content = content[3:].lstrip()
    
    if content.endswith("```"):
        content = content[:-3].rstrip()
    
    # Step 2: Replace localized punctuation / 步骤 2：替换本地化标点
    replacements = {
        '\u201c': '"',  # Left double quotation mark / 左双引号
        '\u201d': '"',  # Right double quotation mark / 右双引号
        '\u2018': "'",  # Left single quotation mark / 左单引号
        '\u2019': "'",  # Right single quotation mark / 右单引号
        '\u2013': '-',  # En dash / 短横线
        '\u2014': '-',  # Em dash / 长横线
    }
    for old, new in replacements.items():
        content = content.replace(old, new)
    
    # Step 3: Escape unescaped quotes in JSON string values
    # 步骤 3：转义 JSON 字符串值中的未转义引号
    lines = content.split('\n')
    fixed_lines = []
    
    for line in lines:
        # Match pattern: "key": "value"
        if '": "' in line:
            parts = line.split('": "', 1)
            if len(parts) == 2:
                key_part = parts[0]
                rest = parts[1]
                
                # Find the end of the value / 找到值的结尾
                if rest.endswith('",'):
                    value_part = rest[:-2]
                    trailing = '",'
                elif rest.endswith('"'):
                    value_part = rest[:-1]
                    trailing = '"'
                else:
                    # Not a standard JSON line, keep as-is
                    # 不是标准 JSON 行，保持原样
                    fixed_lines.append(line)
                    continue
                
                # Escape all quotes in the value / 转义值中的所有引号
                value_fixed = value_part.replace('"', '\\"')
                line = f'{key_part}": "{value_fixed}{trailing}'
        
        fixed_lines.append(line)
    
    content = '\n'.join(fixed_lines)
    return content.strip()


@app.get("/health")
async def health():
    """Health check endpoint / 健康检查端点"""
    return {"status": "ok"}


@app.get("/v1/models")
async def list_models(request: Request):
    """Forward model list requests / 转发模型列表请求"""
    auth = request.headers.get("Authorization") or f"Bearer {UPSTREAM_API_KEY}"
    try:
        r = await http_client.get(
            f"{UPSTREAM_LLM_URL}/v1/models",
            headers={"Authorization": auth}
        )
        return JSONResponse(content=r.json(), status_code=r.status_code)
    except Exception as e:
        logger.exception("Error listing models: %s", e)
        return JSONResponse({"error": str(e)}, status_code=502)


@app.post("/v1/chat/completions")
async def chat_completions(request: Request):
    """
    Handle chat completion requests with response_format injection and cleaning.
    处理聊天完成请求，包括 response_format 注入和清理。
    """
    try:
        body = await request.json()
        messages = body.get("messages", [])
        
        # Extract and remove response_format / 提取并移除 response_format
        response_format = body.pop("response_format", None)
        
        # Inject format instructions into system prompt
        # 将格式指令注入到系统提示词
        if response_format:
            instruction = build_format_instruction(response_format)
            if instruction:
                system_found = False
                for msg in messages:
                    if (msg.get("role") or "").lower() == "system":
                        msg["content"] = (msg.get("content") or "") + instruction
                        system_found = True
                        break
                
                if not system_found:
                    messages.insert(0, {"role": "system", "content": instruction})
        
        stream = body.get("stream", False)
        auth = request.headers.get("Authorization") or f"Bearer {UPSTREAM_API_KEY}"
        
        if stream:
            # Handle streaming response / 处理流式响应
            req = http_client.stream(
                "POST",
                f"{UPSTREAM_LLM_URL}/v1/chat/completions",
                json=body,
                headers={"Authorization": auth, "Content-Type": "application/json"}
            )
            
            async def generate():
                async with req as resp:
                    if resp.status_code >= 400:
                        logger.error("Upstream error: %s", resp.status_code)
                        yield b'data: {"error": "upstream error"}\n\n'
                        return
                    
                    async for chunk in resp.aiter_bytes():
                        yield chunk
            
            return StreamingResponse(generate(), media_type="text/event-stream")
        
        else:
            # Handle non-streaming response / 处理非流式响应
            r = await http_client.post(
                f"{UPSTREAM_LLM_URL}/v1/chat/completions",
                json=body,
                headers={"Authorization": auth, "Content-Type": "application/json"}
            )
            
            response_data = r.json()
            
            # Clean LLM response if successful / 如果成功则清理 LLM 响应
            if r.status_code == 200 and "choices" in response_data:
                for choice in response_data.get("choices", []):
                    if "message" in choice and "content" in choice["message"]:
                        original_content = choice["message"]["content"]
                        cleaned_content = clean_markdown_json(original_content)
                        choice["message"]["content"] = cleaned_content
                        
                        logger.info(
                            "Cleaned response: %d → %d chars",
                            len(original_content),
                            len(cleaned_content)
                        )
                        
                        # Validate JSON / 验证 JSON
                        try:
                            json.loads(cleaned_content)
                            logger.info("JSON validation: OK")
                        except json.JSONDecodeError as je:
                            logger.error(
                                "JSON validation failed: %s at line %d col %d",
                                je.msg, je.lineno, je.colno
                            )
            
            return JSONResponse(content=response_data, status_code=r.status_code)
    
    except Exception as e:
        logger.exception("Error processing request: %s", e)
        return JSONResponse(
            {"error": "proxy error", "detail": str(e)},
            status_code=502
        )


if __name__ == "__main__":
    import uvicorn
    uvicorn.run(app, host="0.0.0.0", port=8000)

Requirements / 依赖

Create requirements.txt:

创建 requirements.txt：

fastapi>=0.100.0
uvicorn[standard]>=0.22.0
httpx>=0.24.0

Deployment Options / 部署选项

Option 1: Docker / 选项 1：Docker

Create Dockerfile:

FROM python:3.11-slim

WORKDIR /app
COPY requirements.txt proxy.py ./
RUN pip install --no-cache-dir -r requirements.txt

ENV UPSTREAM_LLM_URL=http://your-llm:8000
ENV UPSTREAM_API_KEY=""

EXPOSE 8000
CMD ["uvicorn", "proxy:app", "--host", "0.0.0.0", "--port", "8000"]

Build and run / 构建并运行:

docker build -t llm-proxy .
docker run -d -p 8000:8000 \
  -e UPSTREAM_LLM_URL=http://your-llm-api:11434 \
  -e UPSTREAM_API_KEY=your-key \
  llm-proxy

Option 2: Direct Python / 选项 2：直接使用 Python

pip install -r requirements.txt

export UPSTREAM_LLM_URL=http://localhost:11434
export UPSTREAM_API_KEY=your-key

python proxy.py

Option 3: Kubernetes / 选项 3：Kubernetes

Create ConfigMap and Deployment:

apiVersion: v1
kind: ConfigMap
metadata:
  name: llm-proxy-code
data:
  requirements.txt: |
    fastapi>=0.100.0
    uvicorn[standard]>=0.22.0
    httpx>=0.24.0
  proxy.py: |
    # [Paste the complete proxy code above]
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: llm-proxy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: llm-proxy
  template:
    metadata:
      labels:
        app: llm-proxy
    spec:
      initContainers:
      - name: install-deps
        image: python:3.11-slim
        command: ["/bin/sh", "-c", "pip install --target /deps -r /app/requirements.txt"]
        volumeMounts:
        - name: code
          mountPath: /app
        - name: deps
          mountPath: /deps
      containers:
      - name: proxy
        image: python:3.11-slim
        command: ["/bin/sh", "-c"]
        args:
          - PYTHONPATH=/deps python -m uvicorn proxy:app --host 0.0.0.0 --port 8000
        workingDir: /app
        env:
        - name: UPSTREAM_LLM_URL
          value: "http://your-llm-service:8000"
        - name: UPSTREAM_API_KEY
          valueFrom:
            secretKeyRef:
              name: llm-credentials
              key: api-key
        ports:
        - containerPort: 8000
        volumeMounts:
        - name: code
          mountPath: /app
        - name: deps
          mountPath: /deps
      volumes:
      - name: code
        configMap:
          name: llm-proxy-code
      - name: deps
        emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
  name: llm-proxy
spec:
  selector:
    app: llm-proxy
  ports:
  - port: 8000
    targetPort: 8000

Configure Presenton / 配置 Presenton

Point Presenton to the proxy instead of your LLM directly:

将 Presenton 指向代理而不是直接指向您的 LLM：

# Environment variables / 环境变量
export CUSTOM_LLM_URL=http://llm-proxy:8000/v1
export CUSTOM_LLM_API_KEY=your-original-key

# Or in Docker Compose / 或在 Docker Compose 中
services:
  presenton:
    environment:
      - CUSTOM_LLM_URL=http://llm-proxy:8000/v1
      - CUSTOM_LLM_API_KEY=your-key

Important / 重要: If you're using HTTP_PROXY environment variable, add the proxy service to NO_PROXY:

如果您使用 HTTP_PROXY 环境变量，请将代理服务添加到 NO_PROXY：

export NO_PROXY=localhost,127.0.0.1,llm-proxy

Testing / 测试

After deployment, verify the proxy is working:

部署后，验证代理是否工作：

# Check proxy health / 检查代理健康状态
curl http://llm-proxy:8000/health

# Check logs / 查看日志
# Docker:
docker logs <container-id>

# Kubernetes:
kubectl logs -l app=llm-proxy --tail=50

# You should see / 应该看到:
# POST /v1/chat/completions upstream_status=200
# Cleaned response: 636 → 624 chars
# JSON validation: OK

Test presentation generation in Presenton. Multi-page presentations should now generate successfully.

在 Presenton 中测试演示文稿生成。多页演示文稿现在应该能够成功生成。

Why This Works / 为什么这有效

Modern LLMs are quite good at following structured output instructions in system prompts, even without API-level enforcement. The challenges are:

现代 LLM 在遵循系统提示词中的结构化输出指令方面表现很好，即使没有 API 级别的强制执行。挑战在于：

1. Formatting: LLMs often add markdown or explanations / 格式化：LLM 经常添加 markdown 或解释
2. Localization: LLMs use local punctuation for content in that language / 本地化：LLM 为该语言的内容使用本地标点符号
3. Escaping: LLMs don't automatically escape special characters in JSON / 转义：LLM 不会自动转义 JSON 中的特殊字符

By handling these at the proxy level, you get:

• ✅ Support for any OpenAI-compatible API
• ✅ No modifications to Presenton
• ✅ Reliable structured output

通过在代理层处理这些问题，您可以获得：

• ✅ 支持任何 OpenAI 兼容 API
• ✅ 无需修改 Presenton
• ✅ 可靠的结构化输出

Troubleshooting / 故障排除

Problem / 问题: Still getting JSON parsing errors / 仍然出现 JSON 解析错误

Solution / 解决方案:

1. Check proxy logs to see the cleaned output / 检查代理日志以查看清理后的输出
2. The clean_markdown_json function may need adjustment for your specific LLM's output format / 可能需要调整 clean_markdown_json 函数以适应您特定 LLM 的输出格式
3. Increase logging to see the exact JSON that's failing / 增加日志记录以查看失败的确切 JSON

Problem / 问题: 503 Service Unavailable errors / 503 服务不可用错误

Solution / 解决方案:
Check NO_PROXY settings to ensure internal requests aren't routed through external proxies.

检查 NO_PROXY 设置以确保内部请求不会通过外部代理路由。

---

Related Issues / 相关问题: #307, #309, #383

Feel free to adapt this solution to your specific deployment environment!

请随意根据您的具体部署环境调整此解决方案！

@KeepMovingNo1 @614030751 @Charlotte5622 我在K8S重部署，AI生成的一个解决方案，对我有效，生成大纲最后会有报错不过不影响使用。我的环境是用onehub接入claude使用sonet4.5。

[rohgeuntak76]

Hi @MZerg, I've successfully generated presentations using locally deployed LLMs ( e.g. Qwen3-30B-A3B...etc ) with your llm-proxy approach. This has been extremely helpful—thank you for sharing the original implementation. I'd like to share a few updates I made on top of your codebase which is enabling:

• Tool Calling
• enable_thinking support
• Additional request Logging for debugging

172c172
<         
---
>         logger.info(f"response_format: {response_format}")
190c190,193
<         
---
>         body['chat_template_kwargs'] = {
>             "enable_thinking": body.pop("enable_thinking", True)
>         }
199c202
<             
---
>             logger.info(f"body for Stream:\n{body}\n")
219c222
<             
---
>             logger.info(f"body for Constant:\n{body}\n")
225c228
<                     if "message" in choice and "content" in choice["message"]:
---
>                     if "message" in choice and "content" in choice["message"] and choice["message"]["content"] is not None :
229c232,233
<                         
---
>                         logger.info(f"Orignal_content:\n{original_content}\n")
>                         logger.info(f"Cleaned_content:\n{cleaned_content}\n")