Skip to content

[BUG] - user-profile-learning: confidence max = 0.5 rejects update_user_profile calls from Llama 3.3 70B on Groq #231

Description

@Pyrdon

Description
When memoryProvider is openai-chat and memoryModel is llama-3.3-70b-versatile
(Groq), user-profile-learning fails with HTTP 400. Groq's API validates the
tool call parameters server-side and rejects the request when the model's
confidence value exceeds the JSON schema maximum of 0.5. Two separate
enforcement points produce the same rejection: the Zod validation schema
(opencodeProvider code path) and the JSON tool function schema
(memoryApiUrl code path).

Steps to Reproduce

  1. Configure opencode-mem with memoryProvider: "openai-chat" and
    memoryModel: "llama-3.3-70b-versatile" pointing to Groq.
  2. Send enough prompts to trigger user-profile-learning (default threshold 10).
  3. Wait for the next idle analysis cycle.
  4. Observe the log: "OpenAI Chat Completion API error" with status 400
    and the error body shown under Actual Behavior.

Expected Behavior
The tool call succeeds and the user profile is updated with the model's
confidence estimates.

Actual Behavior
Groq returns HTTP 400:

tool call validation failed:
parameters for tool update_user_profile did not match schema:
errors: [/preferences/0/confidence: must be <= 0.5 but found 0.7,
/preferences/1/confidence: must be <= 0.5 but found 0.8]

After retries, the analysis run fails permanently. Those prompts are
marked as captured and never re-queued.

Location

  • File: src/services/user-memory-learning.ts
  • Function: analyzeUserProfile
  • Lines with the constraint:
    • confidence: z.number().min(0).max(0.5) (Zod schema, approx line 381)
    • confidence: { type: "number", minimum: 0, maximum: 0.5 }
      (JSON tool schema, approx line 463)
  • Commit: cfbe178 on main (identical in v2.22.2 at ab0a940)

Impact
User-profile-learning is broken for any model that uses a native 0–1
confidence scale instead of respecting the prompt's guidance
("Assign confidence 0.3-0.5"). Since prompts are marked captured on
failure, they are never retried — the analysis work is permanently lost.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions