Skip to content

Conversation

@IAMDAVID0920
Copy link
Contributor

@IAMDAVID0920 IAMDAVID0920 commented Nov 8, 2025

📋 PR Title Format

The PR title should follow the format:

type(scope): concise message (max 50 chars)

Where:

  • type is one of: feat, fix, docs, refactor, perf, test, chore.
  • scope is optional and describes the part of the codebase affected (e.g., auth, ui, api).
  • concise message is a short description of the change (max 50 chars).

📝 Change Type

Please select the type of change this PR introduces (choose one or more):

  • feat: New feature.
  • fix: Bug fix.
  • docs: Documentation only changes.
  • refactor: A code change that neither fixes a bug nor adds a feature.
  • perf: Performance improvement.
  • test: Adding missing tests or correcting existing tests.
  • chore: Maintenance tasks (e.g., updating dependencies).

💡 Description

The code uses ORJSONResponse from FastAPI which requires the orjson package, but it is not listed in dependencies. This causes AssertionError: orjson must be installed to use ORJSONResponse when making non-streaming requests stream: false

Reproducible from local M1 Mac when i try to follow the example, here are my reproducible steps

> Terminal 1
parallax run -m Qwen/Qwen3-0.6B -n 2
> Terminal 2
parallax join --port 3000
> Terminal 3
parallax join --port 3003
> Terminal 4
curl http://localhost:3001/v1/chat/completions \                                                                                           ─╯
-H "Content-Type: application/json" \
-d '{"messages": [{"role":"user", "content":"Hello!"}], "stream": false}'

Nov 07 22:32:43.934 [ERROR   ] http_server.py:409        Error processing non-streaming request 6875ece1-6c66-4d8e-8807-83f1b1b94b32: orjson must be installed to use ORJSONResponse
INFO:     ::1:55944 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
Nov 07 22:32:43.937 [ERROR   ] server.py:183             Error in chat completion: Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
  File "/Users/dingyunguo/work/parallax/src/parallax/p2p/server.py", line 180, in chat_completion
    ).json()
      ^^^^^^
  File "/Users/dingyunguo/work/parallax/venv/lib/python3.12/site-packages/httpx/_models.py", line 832, in json
    return jsonlib.loads(self.content, **kwargs)                                                                                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/__init__.py", line 346, in loads       return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                              File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())                                                                                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Users/dingyunguo/work/parallax/src/parallax/server/http_server.py", line 406, in v1_chat_completions
    return ORJSONResponse(status_code=200, content=response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dingyunguo/work/parallax/venv/lib/python3.12/site-packages/starlette/responses.py", line 189, in __init__
    super().__init__(content, status_code, headers, media_type, background)
  File "/Users/dingyunguo/work/parallax/venv/lib/python3.12/site-packages/starlette/responses.py", line 46, in __init__
    self.body = self.render(content)
                ^^^^^^^^^^^^^^^^^^^^
  File "/Users/dingyunguo/work/parallax/venv/lib/python3.12/site-packages/fastapi/responses.py", line 45, in render
    assert orjson is not None, "orjson must be installed to use ORJSONResponse"
           ^^^^^^^^^^^^^^^^^^
AssertionError: orjson must be installed to use ORJSONResponse

Proved that after installed orjson, the curl can return proper response

❯ curl http://localhost:3001/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"messages": [{"role":"user", "content":"Hello!"}], "stream": false}'
{"id":"06b00462-1908-4811-9a43-8ae0865544a0","object":"chat.completion","model":"default","created":1762573046.6555982,"choices":[{"index":0,"logprobs":null,"finish_reason":"stop","matched_stop":0,"messages":{"role":"assistant","content":"<think>\nOkay, the user just said \"Hello!\" so I need to respond appropriately. Let me think about the best way to greet them. Since they used \"Hello,\" maybe a simple and friendly reply would work. I should make sure to acknowledge their message and offer help. Let me check if there's any specific context I should consider, but since there isn't, a general response is good. I'll go with something like \"Hello! How can I assist you today?\" to keep it open and helpful.\n</think>\n\nHello! How can I assist you today? 😊","reasoning_content":null,"tool_calls":null}}],"usage":{"prompt_tokens":10,"total_tokens":126,"completion_tokens":116}}

Key Changes

  1. Add orjson to pyproject.toml, i didn't specify a version though, i let pip to resolve that for me, but if we want to be more conservative i can try to fix with orjson>=<minimum version>

🔗 Related Issues

List any issues this PR closes or relates to:

✅ Checklist

Please ensure the following points are addressed before merging:

  • I have performed a self-review of my own code.
  • I have added/updated tests that prove my fix or feature works (if applicable).
  • I have updated the documentation (if necessary).
  • My code follows the project's style guidelines.

@IAMDAVID0920 IAMDAVID0920 requested a review from a team November 8, 2025 04:02
@gufengc gufengc enabled auto-merge (squash) November 8, 2025 16:11
@gufengc gufengc merged commit 7ff90da into GradientHQ:main Nov 8, 2025
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: 500 Internal Server Error when stream is set to false

2 participants