feat(llm-completion): support Anthropic-specific message roles in ChatML #9722
marliessophie
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Describe the feature or potential improvement
Problem
Experiments using dataset items with Anthropic message roles result in unexpected behaviour. Any message role not matching OpenAI's schema gets force-converted to
assistant.Root Cause
Langfuse currently only recognizes OpenAI message roles:
user→HumanMessagesystem/developer→SystemMessageAIMessage(defaults to assistant)This hardcoded OpenAI-first approach ignores that Anthropic uses different conventions:
system,user,assistant,tooluser,assistant(nosystemrole in messages array; uses separatesystemparameter)Dataset items with Anthropic's native role names are silently mangled.
Proposed Solution
Extend
ChatMessageRoleenum inpackages/shared/src/server/llm/types.ts(line 120):Update message transformation logic in
fetchLLMCompletion.tsto map these roles correctly per provider.Additional information
No response
All reactions