Fixed system prompt extraction inside of the bedrock client #1937
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Currently, system prompt are extracted by taking the first message (if it is from
user
and second message is also fromuser
). Furthermore,system
role are being changed touser
, but only after trying to extract the system prompt creating inconsistencies.This is not clear and create bug when using
giskard.rag.generate_testset
(system followed by user is translated to two user messages which is forbidden by the bedrock client):Solution
Remove the previous logic and extract
system_prompt
by using thesystem
role. This is cleaner and way easier to understand.Related Issue
Type of Change