Chat interface for LLM running on RK3588 NPU. Responsive design for desktop & mobile.
- Major Refactoring: Frontend completely rewritten using React + TypeScript + Vite + Tailwind CSS
- Modern component-based architecture replacing vanilla JavaScript
- Improved type safety and developer experience
- Enhanced build tooling and development workflow
- Maintained all existing API compatibility
- Added automated installer script to make installation much easier.
- New version: 0.27
- New alternate UI theme availalble. Select it in settings.ini.
- New visual UI enhancements (touched up bloated rounded styles), better user-input text formatting.
- Front-end error handling in JS now alerts if any servers are offline.
- New
contextcommand prints current context.
- New version: 0.26
- Experimental: You can now (optionally) use chat contexts. Use the commands
clear,off,onto manipulate the state. - Improved error handling for non-2xx HTTP status codes in case of error or server offline.
- Added new parameters to
settings.inipertaining to contexts and error handling. - Code cleanup: Added docstrings and improved commenting for better code readability.
test: Run the tests to ensure proper functionality:
python3 -m pytest tests
configure:
Edit settings.ini with your information.
run:
python3 npuchat.py
then navigate to the IP/port in your browser.
development:\nFor frontend development:\n\ncd frontend\nnpm install\nnpm run dev # Runs Vite dev server on port 5173\n\n\nFor production build:\n\ncd frontend\nnpm run build # Builds to ../static/dist\ncd ..\npython3 npuchat.py # Serves the built React app\n\n\nWorks in any browser besides Firefox desktop or mobile (but maybe nightly, with the correct settings).
Known issues:
Some models don't output valid markdown for everything. For example, if you ask Qwen show me a list of emoji, it will not be valid markdown.
The problem is it uses newline (\n) after each item in the emoji list, but we can't convert it to <br> without potentially destroying all code that passes through. It would be a bad idea. The problem is further complicated by markdown validators do not consider this emoji list to be invalid markdown, so we cannot apply conditional rules. If you have a solution I would love to hear it.