Skip to content

[Bug]: Failed requests cause the node to be unavailable #158

@sl-gn

Description

@sl-gn

### Steps to Reproduce

When the host model is oss-120b, carrying think content in the request context will cause this error

Expected Behavior

After reporting an error, Node continues to process subsequent requests correctly instead of hanging up.

Actual Behavior

After the error, the node is unavailable.

Oct 27 21:03:19.784 [ERROR  ] executor.py:303      Error receiving http request: You have passed a message containing <|channel|> tags in the content field. Instead of doing this, you should pass analysis messages (the string between '<|message|>' and '<|end|>') in the 'thinking' field, and final messages (the string between '<|message|>' and '<|end|>') in the 'content' field.
Traceback (most recent call last):
 File "/Users/gradient/parallax/src/parallax/server/executor.py", line 298, in recv_requests_from_http
  req = self._handle_raw_request(raw_request)
 File "/Users/gradient/parallax/src/parallax/server/executor.py", line 686, in _handle_raw_request
  prompt = self.tokenizer.apply_chat_template(
    messages,
  ...<2 lines>...
    **chat_template_kwargs,
  )
 File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/transformers/tokenization_utils_base.py", line 1666, in apply_chat_template
  rendered_chat, generation_indices = render_jinja_template(
                    ~~~~~~~~~~~~~~~~~~~~~^
    conversations=conversations,
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  ...<6 lines>...
    **template_kwargs,
    ^^^^^^^^^^^^^^^^^^
  )
  ^
 File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/transformers/utils/chat_template_utils.py", line 539, in render_jinja_template
  rendered_chat = compiled_template.render(
    messages=chat,
  ...<3 lines>...
    **kwargs,
  )
 File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/jinja2/environment.py", line 1295, in render
  self.environment.handle_exception()
  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
 File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/jinja2/environment.py", line 942, in handle_exception
  raise rewrite_traceback_stack(source=source)
 File "<template>", line 340, in top-level template code
 File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/jinja2/sandbox.py", line 401, in call
  return __context.call(__obj, *args, **kwargs)
      ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
 File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages/transformers/utils/chat_template_utils.py", line 447, in raise_exception
  raise jinja2.exceptions.TemplateError(message)
jinja2.exceptions.TemplateError: You have passed a message containing <|channel|> tags in the content field. Instead of doing this, you should pass analysis messages (the string between '<|message|>' and '<|end|>') in the 'thinking' field, and final messages (the string between '<|message|>' and '<|end|>') in the 'content' field.

Version

v0.0.1

Environment & Context

  • I'm using the latest version.
  • I have searched existing issues.

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions