If we define a AI Assistant with
import dev.langchain4j.service.Result;
public interface Assistant {
Result<String> answer(String query);
}
When an error occurs during the generate phase, then the Response built got a null TokenUsage.
See here for example line 268:
|
try { |
|
ChatCompletions chatCompletions = client.getChatCompletions(deploymentName, options); |
|
return Response.from( |
|
aiMessageFrom(chatCompletions.getChoices().get(0).getMessage()), |
|
tokenUsageFrom(chatCompletions.getUsage()), |
|
finishReasonFrom(chatCompletions.getChoices().get(0).getFinishReason()) |
|
); |
|
} catch (HttpResponseException httpResponseException) { |
|
logger.info("Error generating response, {}", httpResponseException.getValue()); |
|
FinishReason exceptionFinishReason = contentFilterManagement(httpResponseException, "content_filter"); |
|
return Response.from( |
|
aiMessage(httpResponseException.getMessage()), |
|
null, |
|
exceptionFinishReason |
|
); |
But, the AiService will try to map the response into a Result (following the interface)
|
if (isReturnTypeResult) { |
|
return Result.builder() |
|
.content(parsedResponse) |
|
.tokenUsage(tokenUsageAccumulator) |
|
.sources(augmentationResult == null ? null : augmentationResult.contents()) |
|
.build(); |
|
} else { |
|
return parsedResponse; |
|
} |
and there is a rule in Result constructor to avoid null for TokenUsage
|
public Result(T content, TokenUsage tokenUsage, List<Content> sources) { |
|
this.content = ensureNotNull(content, "content"); |
|
this.tokenUsage = ensureNotNull(tokenUsage, "tokenUsage"); |
|
this.sources = copyIfNotNull(sources); |
|
} |
To fix that, can we create an empty instance of TokenUsage in the catch block ?
return Response.from(
aiMessage(httpResponseException.getMessage()),
new TokenUsage(),
exceptionFinishReason
);
