Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

More user-friendly way to create image messages from local image files #58

Closed
bertilmuth opened this issue Aug 14, 2024 · 3 comments
Closed
Labels
enhancement New feature or request

Comments

@bertilmuth
Copy link
Contributor

Creating an user image message from a URL is straight forward using UserMessage.buildImageMessage.

Creating an user image message from a local image file is much more tedious, here's some sample code:

public void postDiagramImage(String userMessageText, Path diagramImagePath) {
  List<ChatMessage> messages = ...
  UserMessage imageMessage = createImageMessage(userMessageText, diagramImagePath);
  messages.add(imageMessage);
  ...
}

private UserMessage createImageMessage(String userMessageText, Path imagePath) {
  ImageContent userMessageContent = new ImageContent(userMessageText);
  String imagePathString = imagePath.toAbsolutePath().toString();
  String extension = imagePathString.substring(imagePathString.lastIndexOf('.') + 1);
  String imageUrl = "data:image/" + extension + ";base64," + encodeImage(imagePath);
  ImageContent imageContent = new ImageContent(new ImageUrl(imageUrl));
  UserMessage imageMessage = new UserMessage(List.of(userMessageContent, imageContent));
  return imageMessage;
}
	
private String encodeImage(Path imagePath) {
  byte[] fileContent;
      try {
            fileContent = Files.readAllBytes(imagePath);
            return Base64.getEncoder().encodeToString(fileContent);
      } catch (IOException e){
        throw new RuntimeException(e);
      }
}

It would be great if code like this could be encapsulated, and provided to the user in a friendly manner.

@Lambdua
Copy link
Owner

Lambdua commented Aug 15, 2024

ok

@Lambdua Lambdua added the enhancement New feature or request label Aug 15, 2024
@Lambdua
Copy link
Owner

Lambdua commented Aug 15, 2024

now add method in UserMessage.java

    /**
     * 构件一个图片识别请求消息,支持多个图片
     * @author liangtao
     * @date 2024/8/15
     * @param prompt query text
     * @param imagePaths 文件本地路径
     * @return com.theokanning.openai.completion.chat.UserMessage
     **/
    public  static UserMessage buildImageMessage(String prompt, Path... imagePaths) {
        List<ImageContent> imageContents = Arrays.stream(imagePaths).map(ImageContent::new).collect(Collectors.toList());
        imageContents.add(0, new ImageContent(prompt));
        return new UserMessage(imageContents);
    }
```java

@Lambdua Lambdua closed this as completed Aug 15, 2024
@bertilmuth
Copy link
Contributor Author

Works great. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants