Skip to content

optimize AI Context #1328

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
iceljc merged 2 commits into from
Apr 21, 2026
+8 −2
Merged

optimize AI Context #1328

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
611f0fe
optimize AI Context
yileicn Apr 21, 2026
76505f3
optimize AI Context
yileicn Apr 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions src/Infrastructure/BotSharp.Abstraction/Conversations/Enums/MessageTypeName.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,10 @@ namespace BotSharp.Abstraction.Conversations.Enums;
public static class MessageTypeName
{
public const string Plain = "plain";
/// <summary>
/// Persisted for record/audit but excluded from default LLM dialog history.
/// </summary>
public const string RecordOnly = "record_only";
Comment on lines +6 to +9
Copy link
Copy Markdown
Contributor

@qodo-code-review qodo-code-review Bot Apr 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Recordonly leaks to llm 🐞 Bug ⛨ Security

MessageTypeName.RecordOnly is documented as “excluded from default LLM dialog history”, but
ConversationService.GetConversationSummary and EvaluatingService still build LLM inputs from
_storage.GetDialogs without filtering RecordOnly. This can send record/audit-only content to LLM
providers through summarization/evaluation flows.
Agent Prompt 
### Issue description
`RecordOnly` dialogs are intended to be persisted for audit/record, but excluded from **default** LLM dialog history. Some LLM-facing flows (conversation summarization + evaluation) currently include dialogs fetched directly from storage without filtering out `MessageTypeName.RecordOnly`, so audit-only content can be sent to LLM providers.

### Issue Context
- `MessageTypeName.RecordOnly`’s XML doc sets an explicit contract.
- `ConversationService.GetConversationSummary` builds a text prompt from stored dialogs and sends it via `GetChatCompletions`.
- `EvaluatingService` converts stored dialogs into `ref_conversation` strings passed into `instructService.Instruct`.

### Fix Focus Areas
- src/Infrastructure/BotSharp.Core/Conversations/Services/ConversationService.Summary.cs[23-29]
- src/Infrastructure/BotSharp.Core/Conversations/Services/ConversationService.Summary.cs[101-119]
- src/Infrastructure/BotSharp.Core/Evaluations/Services/EvaluatingService.Evaluate.cs[19-28]
- src/Infrastructure/BotSharp.Core/Evaluations/Services/EvaluatingService.Evaluate.cs[144-159]

### Suggested fix
Apply a consistent “LLM history” filter (at least excluding `RecordOnly`, and ideally using the same comparer conventions as elsewhere) before building prompt content in these flows. Consider centralizing the filter into a shared helper to avoid future drift.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

public const string Notification = "notification";
public const string FunctionCall = "function";
public const string Audio = "audio";
Expand Down
6 changes: 4 additions & 2 deletions src/Infrastructure/BotSharp.Core/Routing/RoutingService.GetConversationContent.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
using BotSharp.Abstraction.Conversations.Enums;

namespace BotSharp.Core.Routing;

public partial class RoutingService
Expand All @@ -6,8 +8,8 @@ public async Task<string> GetConversationContent(List<RoleDialogModel> dialogs,
{
var agentService = _services.GetRequiredService<IAgentService>();
var conversation = "";

foreach (var dialog in dialogs.TakeLast(maxDialogCount))
var conversationDialogs = dialogs.Where(x => x.MessageType != MessageTypeName.RecordOnly).TakeLast(maxDialogCount).ToList();
foreach (var dialog in conversationDialogs)
{
var role = dialog.Role;
if (role != AgentRole.User)
Expand Down
Loading