QWEN2.5-1.5B Local Deployment with Ollama and Gradio Interface

Introduction

This README provides instructions on how to deploy the latest QWEN2.5-1.5B large language model locally using Ollama and create an interactive web interface using Gradio.

Prerequisites

• Ollama Installation: Ensure that Ollama is installed on your local machine. You can download and install Ollama from the official website according to your operating system.
• Python Environment: You need to have Python installed on your system, along with the following Python packages:
  • gradio
  • requests
  • json

You can install the required Python packages using pip:

pip install gradio requests

Deployment Steps

1. Pull the QWEN2.5-1.5B Model

Use the following command to pull the QWEN2.5-1.5B model using Ollama:

ollama pull qwen2.5-1.5b

2. Start the Ollama Server

Before running the Gradio interface, make sure the Ollama server is running:

ollama serve

3. Run the Gradio Interface

Execute the Python script qwen-gradio-interface.py to start the Gradio interface:

python qwen-gradio-interface.py

4. Access the Web Interface

Once the script is running, open your web browser and go to the URL provided in the terminal output (usually http://127.0.0.1:7860 ). You should see the QWEN2.5-1.5B chat interface.

Model Information

• Parameter Scale: 1.5B
• Local Run Based on Ollama: The model runs locally using Ollama.
• Support for Chinese and English: The model supports conversations in both Chinese and English.
• Suitable for Personal Computers: The lightweight model is suitable for personal computers.

Interface Features

• Chat Interface: You can interact with the QWEN2.5-1.5B model through a chat interface.
• Advanced Options: You can set system prompts, temperatures, top-p, and maximum token lengths through the advanced options.
• Example Questions: You can click on the example questions to quickly start conversations.

License

This project is provided by Alibaba Cloud. Please refer to the relevant license agreements for more information.