Feature Request: support `q` URL parameter in webui

[odrling]

Prerequisites

• I am running the latest code. Mention the version if possible as well.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new and useful enhancement to share.

Feature Description

Support the q URL parameter in the webui of llama-server to provide an initial prompt. With the current version of llama-server at commit 19a5a3e it does nothing.

Motivation

The Firefox AI chatbot tool seems to rely on the q parameter to send its prompts (for page summaries, explaining the selected text, ...) when the about:config flag to use a local LLM server is set.

The integration also used to work with the previous React webui (I tested with version b6318).

Possible Implementation

Here’s a page describing the integration in open-webui which also mentions other URL parameters, but AFAIK q would already enable most of the contextual functionalities of Firefox. I’m not even sure if the other URL parameters are currently used by Firefox.
https://docs.openwebui.com/tutorials/integrations/firefox-sidebar/

Without requiring Firefox to test, http://localhost:8080/?q=<prompt> would start a conversation with the given prompt.

[chansikpark]

Very interesting feature!

I've been playing with summarizing full articles and ran into length limits in current Firefox.

The short of it is I had to:

1. Set/Increase CPPHTTPLIB_REQUEST_URI_MAX_LENGTH from the default of 8192 in utils.hpp. Perhaps preferrable to pass build system a command line parameter instead.

diff

diff --git a/tools/server/utils.hpp b/tools/server/utils.hpp
index cc48f5a9d..3f03ef764 100644
--- a/tools/server/utils.hpp
+++ b/tools/server/utils.hpp
@@ -13,6 +13,8 @@
 #define CPPHTTPLIB_FORM_URL_ENCODED_PAYLOAD_MAX_LENGTH 1048576
 // increase backlog size to avoid connection resets for >> 1 slots
 #define CPPHTTPLIB_LISTEN_BACKLOG 512
+// increase max URI length to allow for prompts in q URL parameter
+#define CPPHTTPLIB_REQUEST_URI_MAX_LENGTH 100000
 // disable Nagle's algorithm
 #define CPPHTTPLIB_TCP_NODELAY true
 #include <cpp-httplib/httplib.h>

2. Increase browser.ml.chat.maxLength from the default of 7000 in Firefox's about:config

Doing 1. appears to stop the 414 errors in llama-server, and doing 2. seems to raise the ceiling on the "Summarize Page" button.

Unclear on the security implications etc., but I'd think even a conservative mere doubling of maximum request URI length would cover a lot more use cases than it does currently. On the other hand, I'd speculate that the feature/implementation in Firefox is prone to a great deal of evolution going forward so maybe not worth the extra commit for now.. 🤷🏼

PS: Dusted off a dormant account just for this. Kudos to all the ggml contributors!

[allozaur]

@chansikpark could you please create an issue for this? Let's continue that thread over there