[Feature][SolidUI] Dev0.2.0 SolidUI Proposal

[dlimeng]

Background

The theme of this version, the project design page supports large model input conversion, text conversion graphics

Motivation

This version is to support the mature large language model in the community and support text to graphics.
Integrate with the model API through the link and memory functions provided by langchain.

• Integrate langchain with big model

• Provide selection model function (ChatGLM/GPT3.5/GPT4)

• Provide clear context memory function

• Provide text conversion graphics, graphics to generate local file functions.

• Provides drag and drop to generate graphics into the canvas

Implementation

Business structure

User path: login -> click project -> enter the design page -> select canvas -> select model input

• Model input -> model type can be selected
• Model input -> support context awareness, can clear session
• Model input -> text box input plus data to generate graphics

Technology Architecture

architecture design

1. Create a Flask route to handle model input requests.
2. Receive user input and pass it to langchain.
3. Use the chain function and memory management function of langchain to call the corresponding language model according to user input.
4. Return the output of the model to the client.

technical realization

1. The user enters a command or question in the front-end model application.
2. The front-end application sends commands to the /generate interface of the Flask server.
3. The Flask server processes the request and calls the OpenAI API to generate Python code.
4. The OpenAI API returns the generated Python code, which the server sends back to the front-end application as a response.
5. The front-end application receives the generated Python code and sends it to the server for execution. Requests to execute code are sent to a generic proxy interface /api/path:path that forwards requests to another locally running service that executes code and manages the Jupyter kernel.
6. The Flask server executes the code (in a Jupyter kernel) and returns the result to the front-end application.
7. The front-end application receives the execution result and displays the output on the chat interface according to the result type. For example, if the result is an image, display it as an image; if it is text, display it as text.

interface documentation

POST /solidui/models/generate

Function: Submit a query to the specified language model.
Request parameters:
modelId : The ID of the language model to use (eg, gpt-3.5, gpt-4, or chatglm).
prompt : prompt.

GET /solidui/models/keys

Function: Returns a list of supported models.

POST /solidui/models/api/path:path

Function: Generate content and send it to the server. For example, if the result is an image, it will be displayed in image form; if it is text, it will be displayed in text form.

{
"code": 0,
"msg": "成功",
"data": [
{
"type": "image/png",
"value": ""
}
],
"success": true,
"failed": false
}

Compatibility, deprecation, and migration planning

What impact( if any)will there be on existing users?

The user does not need to select a static legend on the design page, and converts the graphics according to the text in the canvas input box

If we are changing behavior, how will we phase out old behavior?

Does not extend old behavior, does not add new preset legends

Do you need special migration tools?

None

When do we remove existing behavior?

No, I think the current method should be preserved and not expanded. If there is a partner to expand, let him raise an issue. The core of the project is to convert text into graphics based on the large model

Test plan

• Test case verification for new features
• All test case coverage verification

https://github.com/CloudOrc/SolidUI-Doc/tree/main/en_US/Test_Documentation/TestCase

Development Process

2023.7.1 ~ 2023.7.8 The model module code is built
2023.7.9 ~ 2023.7.15 Model related interface and joint debugging
2023.7.16 ~ 2023.7.22 The model-related interface has been developed
2023.7.23 ~ 2023.7.29 The model-related interface has been developed
2023.7.30 ~ 2023.8.5 Medium priority feature development
2023.8.6 ~ 2023.8.14 Medium priority feature development
2023.8.15 ~ 2023.8.27 Documentation and Testing
2023.8.28 ~ 2023.8.31 Publishing, operation stage

Issue list

• project->Sliding window on the right side-> Support to select GPT3.5/GPT4/baize-v2 text input box [Feature][SolidUI-Model] Support to select ChatGLM/GPT3.5/GPT4 text input box #51

[{"x":"A","y":5},{"x":"B","y":8},{"x":"C","y":12},{"x":"D","y":6},{"x":"E","y":15},{"x":"F","y":10}]

• [Feature][SolidUI-Web] Support to select ChatGLM/GPT3.5/GPT4 text input box UI #60
• [Feature][SolidUI-Web] Dev0.2.0 preview optimization  #43
• [Feature][Spaces] Hugging Face Spaces #55
• [Feature][SolidUI-Web] Dev0.2.0 preview optimization  #43
• [Feature][SolidUI-Web] Dev0.2.0 Design pages, scenes and page optimization #44
• [Feature][SolidUI-Web] add ESLint  #63
• [Feature][SolidUI] Dev0.2.0 Business test List  #61
• [Feature][SolidUI] Dev0.2.0 Document writing #62

[github-actions]

😊 Welcome to the SolidUI community!!
We are glad that you are contributing by opening this issue.

Please make sure to include all the relevant context.
We will be here shortly.

If you are interested in contributing to our project, please let us know!
You can check out our contributing guide on
👉 How to Contribute.

If you have any questions or suggestions, please feel free to post them here or contact us via email at mengyoupanshan@gmail.com.

We appreciate your interest in SolidUI and look forward to working with you!