Fork 1.4

Merged recent changes from the mainline MNN branch (mostly bug fixes)

Fork 1.3

• Optimized UI translation
  • Significantly reduced the probability of producing invalid JSON in the UI-translating pipeline by directly injecting the necessary JSON tokens, including the object keys (similar to prompt processing or speculative decoding with guaranteed 100% acceptance rate).

Fork 1.2

Fixed a few issues and optimized Quick Chat.

Fixed WebView's inability to scroll when needed
Fixed context menu going off screen (especially the bottom)
Fixed bringing the app to the front via its default home screen icon; it was resetting to the My Models page but leaving the server running
Fixed the context menu also allowing the default behavior of selecting text at the same time, without blocking selection entirely
Enabled another kind of UI translation retry, using higher temperature if it doesn't look like a fixable JSON issue

Optimized Quick Chat by identifying messages that exactly match any built-in string, then using the UI translation (from English) for that text if it does match:

• Saves time and battery power
• The translation is always done from English, which should give better results on average than English->user's language->other user's language, especially if those are low-resource languages.
• Potential for even higher accuracy and more time/power savings if we hard-code more UI translations

Fork 1.1

New Features

• Added the ability to start the server in "use existing Wi-Fi network" mode in addition to the original hotspot mode.
• Added a quick chat menu for some common phrases and suggested topics.
• Added the ability to create a home screen shortcut with your current server settings.
• Added support allowing other users to share the connection QR code(s).
• Added the ability to change your name/avatar via the user list.
• Added chunked UI translations for hopefully quicker processing.
• Added more feature support to the in-app WebView (such as the native back button to close panels).
• Added the ability to view user avatars in a larger size.
• Added the ability to set a custom hotspot SSID and passphrase.
• Added support for message notifications (currently works out-of-the-box for the host device only).

Fixes

• Stability: Fixed a bug where the app would crash if the server port was already in use.
• LLM Translation Failsafes:
  • The app will now automatically attempt to repair broken JSON if a UI translation exhibits certain traits.
  • Added a failsafe to stop UI translation attempts early if the AI output devolves into a repetitive cycle of 1-3 tokens (a common issue with quantized LLMs).

Other Changes

• Image Handling: Avatars are now automatically shrunk/compressed when uploading.
• Cleanup: Removed some unused strings and sorted the rest to take advantage of chunked UI translation.
• HTTPS: With extra effort, notifications can work on client devices if users install the certificate in their trust stores.

Fork initial release

Initial release of this fork; see the readme:

Fork (dpmm99/MNN-Android-Interpreted-Chat-Server) Readme: MNN Android Interpreted Chat Server

A fork of alibaba/MNN modifying their MNN Chat app for Android into a multi-user chat server with on-device LLM-powered interpretation.

Specifically, this modification adds a Wi-Fi hotspot mode, wherein the app hosts a text-based chatroom for multiple nearby users, with automatic language translation using the LLM of your choice.

LLM Support

Any LLM supported by MNN Chat can be used as the interpreter. Of those, both non-thinking models and those using <think>/</think> tags are fully supported.

Language Support

• The language selection screen and QR code labels cycle through ~46 hard-coded languages.
• Any language supported by the LLM is also supported by the app, in theory.
• All web UI text (outside of the language selection screen) has 3 hard-coded languages and will auto-translate on demand when a user selects a language for which it lacks a translation.
• Automatic UI translations are saved in the app cache for later reuse.
• The UI translation pipeline includes a few regular expressions to repair malformed JSON, since LLMs (especially small ones that are also quantized) often produce slightly incorrect output.

Connecting

Users scan the top QR code to connect to the server as a hotspot/router. If the host device has no internet connection, each user has to select "connect only this time" or "stay connected" on Android when prompted.
After connecting, users scan the bottom QR code to navigate to the chat web page using the web browser of their choice--they don't have to have this app, nor do they need to have internet access.

Chat Features

• Users must enter a name and may select a profile picture before joining.
• Newcomers receive the full chat history (since the server was last started), translated into their language from newest to oldest (but lower priority than newly sent messages).
• Messages are primarily plain text, but there is support for replying to a specific message.
• Via the context menu, any message can be retranslated using one additional previous message as context for improved accuracy. Each message is initially translated with no context other than the languages.
• For any message, you can view the original text, the previous translation, or the current translation.
• You can view the current chat participants from the web UI.
• You can export the chat, including the original language and the auto-translated version of each message.
• The host may chat within the app, where they can easily scroll to the QR codes or the latest LLM input/output, or in their default browser.
• The web app listens for broadcast events and updates the UI immediately when a translation is ready.
• The server continues running in the background as long as the OS does not terminate it due to low resources.