Skip to content

Conversation

@sxlijin
Copy link
Collaborator

@sxlijin sxlijin commented Jan 28, 2026


Note

Implements specialize_prompt operation for LLM clients, applying provider-specific transformations (message merging, system prompt consolidation, metadata filtering) based on model capabilities.

Written by Mendral for commit b092e5c.

@vercel
Copy link

vercel bot commented Jan 28, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
beps Ready Ready Preview, Comment Jan 28, 2026 8:47pm
promptfiddle Ready Ready Preview, Comment Jan 28, 2026 8:47pm

Request Review

mendral-app[bot]

This comment was marked as outdated.

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 81e1c42103

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +109 to +110
if !seen_first_system {
seen_first_system = true;
PromptAst::Message {
role,
content,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Only allow system messages in position 0

In consolidate_system_prompts, the “first” system message is kept even if it appears after a user message because the code tracks only the first system encountered, not the message index. For providers with max_one_system_prompt (e.g., Anthropic/Vertex), the runtime logic only permits a system message in slot 0 and converts any later system messages to user. With the current implementation, a prompt like [user, system, user] will preserve the system message in the middle, which violates those providers’ ordering requirements and changes behavior vs. the existing runtime semantics.

Useful? React with 👍 / 👎.

@codspeed-hq
Copy link

codspeed-hq bot commented Jan 28, 2026

CodSpeed Performance Report

Merging this PR will not alter performance

Comparing sam/rp25 (b092e5c) with canary (1191a7c)

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

Summary

✅ 15 untouched benchmarks
⏩ 49 skipped benchmarks1

Footnotes

  1. 49 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

mendral-app[bot]

This comment was marked as outdated.

Copy link
Contributor

@mendral-app mendral-app bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Review

Assessment ⚠️

Follow-up Review: Previous Issues Not Addressed

The update from 2ef6f36 to b092e5c contains only cosmetic changes (formatting, import ordering, comment backticks). Both critical issues from the previous review remain unaddressed:

  1. High severity: Metadata loss in merge_adjacent_messages - silently discards metadata from merged messages (line 56 in transformations.rs)
  2. Medium severity: Missing bounds check in execute_specialize_prompt - can panic on invalid args (line 1410 in lib.rs)

The implementation is otherwise well-structured with comprehensive tests and clear documentation. However, the metadata loss issue is a correctness bug that will lose user data without warning, and should be fixed before merging.

Recommendation: Address the metadata handling in message merging before proceeding. The bounds check is also recommended for defensive programming.

Note

Tag @mendral-app with feedback or questions.

@sxlijin sxlijin added this pull request to the merge queue Jan 28, 2026
Merged via the queue into canary with commit 8e3e605 Jan 28, 2026
39 checks passed
@sxlijin sxlijin deleted the sam/rp25 branch January 28, 2026 22:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants