[BUG] MLModelTool returns null if the response of LLM is a pure json object

[qianheng-aws]

What is the bug?
MLModelTool returns null if the response of LLM is a pure json object

How can one reproduce the bug?
This bug happens when developing the feature of "text2visualization", it's hard to build the environment since it has not been released. But we can simplify the case with below several steps:

1. register a flow agent with MLModelTool whose prompt should be like "\n\nHuman:You should generate a json object directly with format {name: ${NAME}, age: ${AGE}}. \n\nHuman:${parameters.question}\n\nAssistant:"
2. request flow agent with question to generate a json object directly, e.g. "Generate a json object for Jack who is 18"
3. it will occasionally produce error message like below

"{\"error\":{\"reason\":\"System Error\",\"details\":\"Cannot invoke \\\"org.opensearch.ml.common.output.Output.toXContent(org.opensearch.core.xcontent.XContentBuilder, org.opensearch.core.xcontent.ToXContent$Params)\\\" because \\\"this.output\\\" is null\",\"type\":\"NullPointerException\"},\"status\":500}"

What is the expected behavior?
It should return the right json object {name: Jack, age: 18}

What is your host/environment?

• OS: [e.g. macOS]
• Version [e.g. 3.0.0-SNAPSHOT]
• Plugin

Do you have any screenshots?
The error message in dev-tools:

The debug result of ModelTensors:

Do you have any additional context?
It's caused by the fragile implementation of MLModelTool which misses the case that LLM can return a json object directly. See this line:

ml-commons/ml-algorithms/src/main/java/org/opensearch/ml/engine/tools/MLModelTool.java

Line 69 in b6618b2

return mlModelOutputs.get(0).getMlModelTensors().get(0).getDataAsMap().get(responseField);