Skip to content

Commit

Permalink
Do not include images in ChatGPT "assistant" messages
Browse files Browse the repository at this point in the history
GPT-4o model does not allow "assistant" messages to contain image URLs.

https://chat.stackexchange.com/transcript/message/65657410#65657410
  • Loading branch information
mangstadt committed May 15, 2024
1 parent c54f2e3 commit b89bc61
Showing 1 changed file with 7 additions and 5 deletions.
12 changes: 7 additions & 5 deletions src/main/java/oakbot/listener/chatgpt/ChatGPT.java
Original file line number Diff line number Diff line change
Expand Up @@ -340,16 +340,20 @@ private List<ChatCompletionRequest.Message> buildChatCompletionMessages(String p
.role("system")
.text(prompt)
.build());

chatMessages.stream()
.filter(chatMessage -> chatMessage.getContent() != null) //skip deleted messages
.map(chatMessage -> {
var content = chatMessage.getContent();
var contentStr = content.getContent();
var fixedWidthFont = content.isFixedWidthFont();
var contentMd = ChatBuilder.toMarkdown(contentStr, fixedWidthFont);
var messagePostedByOak = (chatMessage.getUserId() == bot.getUserId());

var imageUrls = extractImageUrlsIfModelSupportsVision(contentStr);
/*
* GPT-4o model does not allow "assistant" messages to contain image URLs.
*/
var imageUrls = messagePostedByOak ? List.<String>of() : extractImageUrlsIfModelSupportsVision(contentStr);

String truncatedContentMd;
if (latestMessageCharacterLimit > 0) {
Expand All @@ -358,11 +362,9 @@ private List<ChatCompletionRequest.Message> buildChatCompletionMessages(String p
truncatedContentMd = contentMd;
}

var messagePostedByOak = (chatMessage.getUserId() == bot.getUserId());

return new ChatCompletionRequest.Message.Builder()
.role(messagePostedByOak ? "assistant" : "user")
.name(messagePostedByOak ? bot.getUsername() : chatMessage.getUsername())
.name(chatMessage.getUsername())
.text(truncatedContentMd)
.imageUrls(imageUrls, "low")
.build();
Expand Down

0 comments on commit b89bc61

Please sign in to comment.