Language Server API for AI #9679
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4e6
added
the
CI: No changelog needed
4e6
force-pushed
the
wip/db/9656-language-server-api-for-ai
branch
from
69f8b64
to
27db5ed
Compare
|
The communication with OpenAi looks like this: {
"model" : "gpt-4-turbo-preview",
"response_format" : {
"type" : "json_object"
},
"messages" : [
{
"role" : "system",
"content" : "You are a data analyst. You use Python3. Installed libraries: ['pandas'].\nYour task is to output JSON object with fields:\n- 'kind': 'final'\n- 'fn': String, Python function returning what user wants. Always write as generic code as possible that will work even if the input data (e.g. file content) changes. Do not assume any input data exists if not provided with it explicitly. Use your knowledge about the world if the provided data is missing.\n- 'fnCall': String, Python code that calls the generated function.\n- 'resultPreview': Code in Python. When evaluated, prints to stdout a preview of the result, e.g. 'Visualization.AI.print(\"Number 5\")'. Make it as generic as possible. It should work even if the input data changes. The string written to stdout should be one-line, as short as possible, and as informative as possible, e.g. 'Table with 50 rows and columns \"c1\", \"c2\", and \"c3\"'. It can assume that the 'fnCall' result is in scope.\n- 'queryParts': Array of user query divided into either non-editable text, or widgets. The idea is that users can click widgets to change them to other values. Every part should be one of the JSON objects:\n * Fields:\n a. 'kind': 'text'\n b. 'text': Part of the user query that should not be widget. In particular, numbers shoul not be widgets.\n * Fields:\n a. 'kind': 'dropdown'\n b. 'values': list of possible values. For example, if the query contains name of a column in a data set, provide all other column names, like ['columnName1', 'columnName2']. If the query contains a common comparator like 'less than', provide other comparators like ['greater than', 'equal to']. The same applies to other comparators like 'most popular'.\n\nIf in order to provide the answer you need to investigate what is inside the data, you can run code and be asked again the same question with provided stdout by outputing JSON object with fields:\n- 'kind': 'eval'\n- 'code': Python code required to investigate data. The code should write to stdout as little as possible. Use 'Visualization.AI.print' function for printing to stdout. You can only use data you are already provided with, no more data can be provided and you can't ask for more data.\n- 'reason': Reason why you were not able to provide final code.\nAlways prefer outputting the final code. Use kind \"eval\" only if you can't output object with kind \"final\".\n\nIf you can't provide the answer, because the current data and your knowledge about the world is not enough, output JSON object with fields:\n- 'kind': 'fail'\n- 'reason': Reason why you were not able to provide the answer. As short as possible. Do not mention Python nor code, this is information for non-tech users.\n"
},
{
"role" : "user",
"content" : "There is 'operator70395' variable defined in the program. What is the result of the variable 'operator70395'?"
}
]
}{
"id": "chatcmpl-9JPO6mDpY9zZ7hJxQcQw2w95grj1F",
"object": "chat.completion",
"created": 1714412850,
"model": "gpt-4-0125-preview",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "{\n \"kind\": \"eval\",\n \"code\": \"Visualization.AI.print(operator70395)\",\n \"reason\": \"To provide the result of 'operator70395', I need to know its current value.\"\n}"
},
"logprobs": null,
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 655,
"completion_tokens": 44,
"total_tokens": 699
},
"system_fingerprint": "fp_d65ac1064c"
}{
"model" : "gpt-4-turbo-preview",
"response_format" : {
"type" : "json_object"
},
"messages" : [
{
"role" : "system",
"content" : "You are a data analyst. You use Python3. Installed libraries: ['pandas'].\nYour task is to output JSON object with fields:\n- 'kind': 'final'\n- 'fn': String, Python function returning what user wants. Always write as generic code as possible that will work even if the input data (e.g. file content) changes. Do not assume any input data exists if not provided with it explicitly. Use your knowledge about the world if the provided data is missing.\n- 'fnCall': String, Python code that calls the generated function.\n- 'resultPreview': Code in Python. When evaluated, prints to stdout a preview of the result, e.g. 'Visualization.AI.print(\"Number 5\")'. Make it as generic as possible. It should work even if the input data changes. The string written to stdout should be one-line, as short as possible, and as informative as possible, e.g. 'Table with 50 rows and columns \"c1\", \"c2\", and \"c3\"'. It can assume that the 'fnCall' result is in scope.\n- 'queryParts': Array of user query divided into either non-editable text, or widgets. The idea is that users can click widgets to change them to other values. Every part should be one of the JSON objects:\n * Fields:\n a. 'kind': 'text'\n b. 'text': Part of the user query that should not be widget. In particular, numbers shoul not be widgets.\n * Fields:\n a. 'kind': 'dropdown'\n b. 'values': list of possible values. For example, if the query contains name of a column in a data set, provide all other column names, like ['columnName1', 'columnName2']. If the query contains a common comparator like 'less than', provide other comparators like ['greater than', 'equal to']. The same applies to other comparators like 'most popular'.\n\nIf in order to provide the answer you need to investigate what is inside the data, you can run code and be asked again the same question with provided stdout by outputing JSON object with fields:\n- 'kind': 'eval'\n- 'code': Python code required to investigate data. The code should write to stdout as little as possible. Use 'Visualization.AI.print' function for printing to stdout. You can only use data you are already provided with, no more data can be provided and you can't ask for more data.\n- 'reason': Reason why you were not able to provide final code.\nAlways prefer outputting the final code. Use kind \"eval\" only if you can't output object with kind \"final\".\n\nIf you can't provide the answer, because the current data and your knowledge about the world is not enough, output JSON object with fields:\n- 'kind': 'fail'\n- 'reason': Reason why you were not able to provide the answer. As short as possible. Do not mention Python nor code, this is information for non-tech users.\n"
},
{
"role" : "user",
"content" : "There is 'operator70395' variable defined in the program. What is the result of the variable 'operator70395'?"
},
{
"role" : "user",
"content" : "EVALUATED:\nVisualization.AI.print(operator70395)\n\nOUTPUT:\n226"
}
]
}{
"id": "chatcmpl-9JPOALptlvpvX52NJXd22STfeGSZL",
"object": "chat.completion",
"created": 1714412854,
"model": "gpt-4-0125-preview",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "{\n \"kind\": \"final\",\n \"fn\": \"def get_operator70395_result():\\n return operator70395\",\n \"fnCall\": \"get_operator70395_result()\",\n \"resultPreview\": \"Visualization.AI.print('Result is 226')\",\n \"queryParts\": [\n {\n \"kind\": \"text\",\n \"text\": \"There is 'operator70395' variable defined in the program. What is the result of the variable 'operator70395'?\"\n }\n ]\n}"
},
"logprobs": null,
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 674,
"completion_tokens": 108,
"total_tokens": 782
},
"system_fingerprint": "fp_d65ac1064c"
} |
4e6
requested review from
jdunkerley,
radeusgd,
GregoryTravis,
AdRiley,
hubertp and
Akirathan
as code owners
hubertp
reviewed
Apr 30, 2024
| ("response", responseUtf8String.asJson) | ||
| ) | ||
| replyTo ! ResponseError(Some(id), AiApi.AiError(Some(payload))) | ||
| }(success => replyTo ! ResponseResult(AiCompletion2, id, success)) |
There was a problem hiding this comment.
It doesn't look like we ever stop the handler?
hubertp
approved these changes
May 2, 2024
Comment on lines
+308
to
+311
| private def stop(): Unit = { | ||
| self ! PoisonPill | ||
| } | ||
| } |
There was a problem hiding this comment.
I thought we normally did context.stop(self)? Is there any reason for it to process any existing messages in the queue?
There was a problem hiding this comment.
Not really. It could be a stop method of a context.
radeusgd
approved these changes
May 2, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Description
close #9656
Changelog:
ai/completion_v2methodVisualization.AI.printmethod for converting the expression to text formatVisualization.AI.printmethod for printing.Important Notes
The project New_Project_1.zip contains the following main file:
To test the functionality, I asked AI to show me the result of the
operator70395variable:{"jsonrpc":"2.0","id":0,"method":"session/initProtocolConnection","params":{"clientId":"d8e948fd-6418-43c8-9f02-54827f09e10a"}}{"jsonrpc":"2.0","id":0,"method":"session/initProtocolConnection","params":{"clientId":"d8e948fd-6418-43c8-9f02-54827f09e10a"}}{"jsonrpc":"2.0","id":0,"method":"executionContext/push","params":{"contextId":"730a66ef-4222-46f8-8a03-d766946ab2bd","stackItem":{"methodPointer":{"module":"local.New_Project_1.Main","definedOnType":"local.New_Project_1.Main","name":"main"},"positionalArgumentsExpressions":[],"type":"ExplicitCall"}}}{"jsonrpc":"2.0","id":1,"method":"ai/completion_v2","params":{"contextId":"730a66ef-4222-46f8-8a03-d766946ab2bd","expressionId":"f09a4372-3231-4f2e-99f4-84aa751f9b60","prompt":"There is 'operator70395' variable defined in the program. What is the result of the variable 'operator70395'?"}}I got the following responses:
{"jsonrpc":"2.0","method":"ai/completionProgress","params":{"code":"Visualization.AI.print(operator70395)","reason":"To provide the result of 'operator70395', I need to know its current value.","visualizationId":"edfb00a3-6ce5-41e1-bb8f-ab191809114e"}}{"jsonrpc":"2.0","id":1,"result":{"Success":{"fn":"def get_operator70395_result():\n return operator70395","fnCall":"get_operator70395_result()"}}}Checklist
Please ensure that the following checklist has been satisfied before submitting the PR:
Scala,
Java,
and
Rust
style guides. In case you are using a language not listed above, follow the Rust style guide.
./run ide build.