Skip to content

fix: handle LLM responses wrapped in markdown code fences#3972

Merged
skwowet merged 11 commits intomainfrom
bugfix/processMemberBotAnalysisWithLLM
Mar 27, 2026
Merged

fix: handle LLM responses wrapped in markdown code fences#3972
skwowet merged 11 commits intomainfrom
bugfix/processMemberBotAnalysisWithLLM

Conversation

@skwowet
Copy link
Copy Markdown
Collaborator

@skwowet skwowet commented Mar 27, 2026

Note

Medium Risk
Introduces shared LLM-answer parsing logic and replaces direct JSON.parse calls, which changes how LLM outputs are interpreted and could surface new parse failures if the heuristic extraction mis-detects JSON boundaries.

Overview
Adds a shared parseLlmJson helper (exported from @crowd/common) to tolerate common LLM response formats (raw JSON, json fenced blocks, or JSON embedded in surrounding text) and to provide clearer errors when parsing fails.

Updates domain selection, member bot validation, and multiple LlmService result helpers to use parseLlmJson instead of JSON.parse, making these workflows more resilient to markdown-wrapped or noisy LLM answers.

Written by Cursor Bugbot for commit 45be045. This will update automatically on new commits. Configure here.

@skwowet skwowet self-assigned this Mar 27, 2026
Copilot AI review requested due to automatic review settings March 27, 2026 16:53
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

1 similar comment
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

skwowet added 6 commits March 27, 2026 22:24
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
…ysis

Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
…LLM queries

Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
@skwowet skwowet force-pushed the bugfix/processMemberBotAnalysisWithLLM branch from a4205bc to 25910d9 Compare March 27, 2026 16:55
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a shared JSON parsing helper to make LLM integrations resilient to responses wrapped in markdown/code-fences or containing extra surrounding text, and updates LLM call sites to use it.

Changes:

  • Introduces parseLlmJson in @crowd/common to parse JSON from raw LLM answers (direct, fenced, or extracted).
  • Replaces multiple JSON.parse(response.answer) usages with parseLlmJson(...) across LLM consumers.
  • Exports the new helper from the @crowd/common barrel.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
services/libs/common_services/src/services/llm.service.ts Switches LLM result parsing from JSON.parse to parseLlmJson for multiple query methods.
services/libs/common/src/llm.ts Adds parseLlmJson helper that attempts direct parse, fenced-block parse, then balanced JSON extraction.
services/libs/common/src/index.ts Re-exports the new llm helper module from @crowd/common.
services/apps/profiles_worker/src/workflows/member/processMemberBotAnalysisWithLLM.ts Uses parseLlmJson to parse the bot-analysis LLM response inside a Temporal workflow.
services/apps/organizations_enrichment_worker/src/activities/llm.ts Uses parseLlmJson when parsing domain-selection LLM responses.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

1 similar comment
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

  • feat: add user authentication (CM-123)
  • feat: add user authentication (IN-123)

Projects:

  • CM: Community Data Platform
  • IN: Insights

Please add a Jira issue key to your PR title.

@skwowet skwowet merged commit bd60722 into main Mar 27, 2026
14 checks passed
@skwowet skwowet deleted the bugfix/processMemberBotAnalysisWithLLM branch March 27, 2026 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants