ref!: enable static builds #10

coolaj86 · 2023-10-21T20:46:00Z

switch from adapter-node to adapter-static
rename ENDPOINT to API_BASE_URL to follow common convention and avoid confusing with webserver vs api server
match on /api prefix to more easily allow control of Reverse Proxy routes from Web Server to API Server

Preview Documentation

https://github.com/coolaj86/ollama-webui/tree/feat-static#how-to-build-for-static-deployment

How to build

git clone git@github.com:ollama-webui/ollama-webui.git
pushd ./ollama-webui/
npm run build

Example `Caddyfile`

Reverse Proxy to localhost:11434
Serve ./build/

# replace localhost with example.com or whatever
localhost {
    handle /api/* {
        reverse_proxy localhost:11434
    }

    file_server {
        root ./build/
    }
}

caddy run --config ./Caddyfile

with Working CORS Snippet

# CORS Preflight (OPTIONS) + Request (GET, POST, PATCH, PUT, DELETE)
(cors-api) {
	@match-cors-api-preflight method OPTIONS
	handle @match-cors-api-preflight {
		header {
			Access-Control-Allow-Origin "{http.request.header.origin}"
			Access-Control-Allow-Methods "GET, POST, PUT, PATCH, DELETE, OPTIONS"
			Access-Control-Allow-Headers "Origin, Accept, Authorization, Content-Type, X-Requested-With"
			Access-Control-Allow-Credentials "true"
			Access-Control-Max-Age "3600"
			defer
		}
		respond "" 204
	}

	@match-cors-api-request {
		not {
			header Origin "{http.request.scheme}://{http.request.host}"
		}
		header Origin "{http.request.header.origin}"
	}
	handle @match-cors-api-request {
		header {
			Access-Control-Allow-Origin "{http.request.header.origin}"
			Access-Control-Allow-Methods "GET, POST, PUT, PATCH, DELETE, OPTIONS"
			Access-Control-Allow-Headers "Origin, Accept, Authorization, Content-Type, X-Requested-With"
			Access-Control-Allow-Credentials "true"
			Access-Control-Max-Age "3600"
			defer
		}
	}
}

# replace localhost with example.com or whatever
localhost {
	## HTTP Basic Auth
	## (uncomment to enable)
	# basicauth {
	# 	# see .example.env for how to generate tokens
	# 	{env.OLLAMA_API_ID} {env.OLLAMA_API_TOKEN_DIGEST}
	# }

	handle /api/* {
		# Comment to disable CORS
		import cors-api

		reverse_proxy localhost:11434
	}

	# Same-Origin Static Web Server
	file_server {
		root ./build/
	}
}

tjbck · 2023-10-21T21:05:50Z

Looking good! Great work on enabling the static build. Could you also please verify that it functions seamlessly with Docker? It would be beneficial to ensure compatibility across different environments. Thanks!

tjbck · 2023-10-21T21:11:04Z

Additionally, I also suggest providing users with the option to bypass the reverse proxy, allowing direct usage of the ollama API_ENDPOINT (https://localhost:11434/api). This will ensure flexibility for users while maintaining a straightforward setup process. Thank you for your efforts on this!

coolaj86 · 2023-10-21T21:28:20Z

I don't use Docker, but I can verify that both npm run dev and npm run build are working, which means that the docker bits should still be working since it's just a node image, which I believe is just an ubuntu image.

Help Needed

Here's where I need help:

I don't understand how the svelte environment system works and it seems to behave differently between adaptor-node and adapter-static as well as between npm run dev and npm run build.

This is what should happen:

# this should build from `src/lib/constants.ts`
npm run build

# this should use the ENV
API_SERVER=https://localhost/api npm run build

# this should have access to `${location.protocol}` and `${location.host}`
npm run dev

I'm not sure how to get that behavior. That needs someone with Svelte expertise, which I absolutely am not.

I'm very familiar with Linux and Node and JavaScript, but this is doing proprietary things, not following any normal conventions. (I believe Svelte is is completely its own language that just happens to use JS syntax)

tjbck · 2023-10-21T23:29:05Z

@coolaj86

I've just pushed a few commits and attempted to align the behaviour as per your requirements. If there are any discrepancies or if I've misunderstood any part of your request, please don't hesitate to let me know.

tjbck · 2023-10-21T23:33:38Z

Forgot to mention, to build with the environment variable, please run:

OLLAMA_API_ENDPOINT="http://[Your Ollama URL]/api" npm run build

This should run with the specified environment variable. Let me know if you have any questions.

.gitignore

coolaj86 · 2023-10-22T00:11:29Z

README.md

 - 💻 **Code Syntax Highlighting**: Enjoy enhanced code readability with our syntax highlighting feature.
- 🔗 **External Ollama Server Connection**: Link to the model when Ollama is hosted on a different server via the environment variable -e OLLAMA_ENDPOINT="http://[insert your Ollama address]".
+
+- 🔗 **External Ollama Server Connection**: You can seamlessly connect to an external Ollama server hosted on a different address by setting the environment variable during the Docker build process. Execute the following command to include the Ollama API endpoint in the Docker image: `docker build --build-arg OLLAMA_API_ENDPOINT="http://[Your Ollama URL]/api" -t ollama-webui .`.


My understanding is that there is no Ollama API, but rather Ollama uses the ChatGPT API, so calling it OLLAMA may be a bit of a misnomer.

That said, I may be wrong, and I could see a future in which OLLAMA begins to add its own features to create a separate API (maybe that's already the case with /api/tags? Or maybe that's how GPT distinguishes between 3, 3.5, 4, etc too?)

I'm uncertain about your point regarding Ollama's use of the ChatGPT API. Could you please provide further clarification? Ollama functions locally and relies on API calls to access the downloaded models.

My point is that it's like S3. Whether you're using Amazon or Digital Ocean or Minio, the ENVs are always named according to S3 because S3 is a protocol more than it is a specific product.

S3_REGION=... S3_BUCKET=... S3_PREFIX=... S3_KEY=...

It would be more confusing to do something like

DIGITAL_OCEAN_BUCKET=...

Because everyone just knows it as "S3" regardless of who is actually providing the service.

Likewise, the GPT is generic. Lots and lots of tools are being built that either provide as a service or consume as a client the GPT API.

So naming it OLLAMA_API_ENDPOINT is actually confusing - it makes it sound like it's NOT compatible with GPT, but actually it is.

So if someone had another client product that works with GPT and they wanted to try it with ollama, it would be much more consistent for it to either not carry the protocol name at all API_ENDPOINT, or to carry the generic protocol name GPT_API_ENDPOINT.

# confusing, sounds like there's a new API that's incompatible with GPT OLLAMA_API_ENDPOINT=... # clear, this is a generic, GPT-compatible Chat UI that relies on the GPT API GPT_API_ENDPOINT=... # not as clear, but at least not confusing API_ENDPOINT

I had seen is some of the ollama or mistral documentation that they use the GPT API, but in looking at https://platform.openai.com/docs/api-reference/fine-tuning, it doesn't appear to be the case from what I can see.

I see what you mean but I think you might be misunderstanding what Ollama is offering. FYI, Ollama does not make any requests to proprietary OpenAI GPT-related APIs and everything is processed on a local machine, hence needing to download LLMs locally beforehand (it can also be operated offline!). Additionally, Ollama WebUI only supports Ollama APIs at the moment, so I believe it's appropriate to use the name "OLLAMA_API_ENDPOINT" for the env variable.

If you have any other questions regarding Ollama to help you understand how it works, feel free to ask, I'll do my best to answer to the best of my ability! (discord [at]timbk) Thanks for the explanation though!

@tjbck your discord handle appears to be my GitHub account name. If you tag it with the @ I automatically get notifications for this whole discussion. Just wanted to make you aware and commenting here was the only option I saw.
Cool project btw and have a nice Sunday guys!

Oops, so sorry 😅. Thanks for your understanding. Have a good day as well!

I know it doesn't make requests to OpenAI's GPT, but I was under the impression that it used the same API signatures.

After checking the documentation and API calls, it looks like that was not correct, or I misunderstood what I had read.

Agreed, OLLAMA_* is the best way to go.

tjbck · 2023-10-22T03:41:06Z

@coolaj86 Is everything working as it should? everything lgtm! Will merge this branch once you confirm everything is working as intended for you.

coolaj86 · 2023-10-22T03:58:12Z

I'm just now taking a look at this again.

coolaj86 · 2023-10-22T08:09:08Z

Okay, I believe I've adequately tested and documented everything.

Also:

renamed .env.example to example.env so that it is immediately visible to all users
renamed *_API_ENDPOINT to *_API_BASE_URL to match the common convention
(this is also what's documented in the Svelte documentation on ENVs)
added caddy example with HTTP Basic Auth for GitHub-style API Tokens

coolaj86 · 2023-10-22T09:35:56Z

One last thing I didn't test or check:

if API_BASE_URL='' (empty) it should default to http://localhost:11434/api
if API_BASE_URL='/api' (path only) then it should be relative to the current scheme/host/port

Then the main release could be built for '' and embedded releases could be built for '/api'

It would also be nice (in another iteration) to have a setting to change it at runtime.

src/lib/constants.ts

coolaj86 · 2023-10-22T17:50:07Z

README.md

@@ -48,9 +48,10 @@ OLLAMA_HOST=0.0.0.0 OLLAMA_ORIGINS=* ollama serve

 ### Using Docker 🐳

-```bash
-docker build --build-arg OLLAMA_API_BASE_URL='http://localhost:11434/api' -t ollama-webui .


"explicit is better than implicit" - it means users have to reference less documentation, and have to hold less in their heads.

https://github.com/ewjoachim/zen-of-python/blob/master/zen.png

It looks "cool" to copy and paste fewer characters, but it's much more frustrating to have to reference documentation for things that aren't intuitive.

That said, now that we know that '' will work correctly, showing OLLAMA_API_BASE_URL='' would both be fewer characters and still be mostly self-documenting.

coolaj86 · 2023-10-22T18:28:03Z

I just rebased this on the current main and squashed both of our most recent doc commits into one.

I think this is ready for your final review.

tjbck · 2023-10-22T19:43:34Z

LGTM! Great work, Thanks a lot!

ref!: enable static builds

This was referenced Oct 21, 2023

doc: update Caddy Cors in Cheat Sheet webinstall/webi-installers#693

Closed

403 OPTIONS "/api/generate" #2

Closed

coolaj86 force-pushed the feat-static branch 2 times, most recently from ee754a6 to 613a656 Compare October 21, 2023 21:04

coolaj86 force-pushed the feat-static branch from 613a656 to a8e3964 Compare October 21, 2023 21:21

coolaj86 force-pushed the feat-static branch from a8e3964 to 283dd18 Compare October 21, 2023 21:29

coolaj86 commented Oct 22, 2023

View reviewed changes

.gitignore Outdated Show resolved Hide resolved

coolaj86 commented Oct 22, 2023

View reviewed changes

This was referenced Oct 22, 2023

Documentation for Environment Variables? How to change API Host? #8

Closed

GitHub Releases for Build version of this chat app? #7

Closed

This was referenced Oct 22, 2023

Add flag --web-root for serving UI (w/ code example) ollama/ollama#874

Closed

Embed a UI with Ollama ollama/ollama#875

Closed

coolaj86 force-pushed the feat-static branch 7 times, most recently from bc8f80a to 49443f7 Compare October 22, 2023 07:05

tjbck requested review from tjbck and removed request for tjbck October 22, 2023 07:20

coolaj86 force-pushed the feat-static branch from 49443f7 to 5d83b5f Compare October 22, 2023 08:03

coolaj86 changed the title ~~WIP: enable static builds~~ ref!: enable static builds Oct 22, 2023

coolaj86 mentioned this pull request Oct 22, 2023

Add installers for ollama & ollama-webui webinstall/webi-installer-requests#72

Open

15 tasks

coolaj86 commented Oct 22, 2023

View reviewed changes

src/lib/constants.ts Outdated Show resolved Hide resolved

coolaj86 commented Oct 22, 2023

View reviewed changes

coolaj86 mentioned this pull request Oct 22, 2023

Differents images same container #15

Closed

coolaj86 force-pushed the feat-static branch from 2527338 to 44294ab Compare October 22, 2023 18:26

coolaj86 and others added 7 commits October 22, 2023 12:26

feat: enable static builds

f4f1283

feat: enable buildtime API_ENDPOINT env var

86395a8

Update node.js.yaml

b035de1

feat: update .env.example and add Caddyfile

c307777

chore: change to API_ENDPOINT to conventional name API_BASE_URL

859adee

doc: how to build and test static site

87c8467

doc: clarify usage of OLLAMA_API_BASE_URL

6129f41

coolaj86 force-pushed the feat-static branch from 44294ab to 6129f41 Compare October 22, 2023 18:27

doc: contributor added & typo fix

f25359f

tjbck merged commit f2bdbfa into open-webui:main Oct 22, 2023
1 check passed

tjbck mentioned this pull request Oct 22, 2023

Usage in Kubernetes #17

Closed

explorigin pushed a commit to explorigin/open-webui that referenced this pull request Feb 2, 2024

Merge pull request open-webui#10 from coolaj86/feat-static

a1e2d1c

ref!: enable static builds

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ref!: enable static builds #10

ref!: enable static builds #10

coolaj86 commented Oct 21, 2023 •

edited

tjbck commented Oct 21, 2023

tjbck commented Oct 21, 2023

coolaj86 commented Oct 21, 2023

tjbck commented Oct 21, 2023

tjbck commented Oct 21, 2023

coolaj86 Oct 22, 2023

tjbck Oct 22, 2023

coolaj86 Oct 22, 2023 •

edited

coolaj86 Oct 22, 2023

tjbck Oct 22, 2023 •

edited

timbk Oct 22, 2023

tjbck Oct 22, 2023

coolaj86 Oct 22, 2023 •

edited

tjbck commented Oct 22, 2023

coolaj86 commented Oct 22, 2023

coolaj86 commented Oct 22, 2023 •

edited

coolaj86 commented Oct 22, 2023 •

edited

coolaj86 Oct 22, 2023 •

edited

tjbck Oct 22, 2023

coolaj86 commented Oct 22, 2023

tjbck commented Oct 22, 2023

ref!: enable static builds #10

ref!: enable static builds #10

Conversation

coolaj86 commented Oct 21, 2023 • edited

Preview Documentation

How to build

Example Caddyfile

with Working CORS Snippet

tjbck commented Oct 21, 2023

tjbck commented Oct 21, 2023

coolaj86 commented Oct 21, 2023

Help Needed

tjbck commented Oct 21, 2023

tjbck commented Oct 21, 2023

coolaj86 Oct 22, 2023

Choose a reason for hiding this comment

tjbck Oct 22, 2023

Choose a reason for hiding this comment

coolaj86 Oct 22, 2023 • edited

Choose a reason for hiding this comment

coolaj86 Oct 22, 2023

Choose a reason for hiding this comment

tjbck Oct 22, 2023 • edited

Choose a reason for hiding this comment

timbk Oct 22, 2023

Choose a reason for hiding this comment

tjbck Oct 22, 2023

Choose a reason for hiding this comment

coolaj86 Oct 22, 2023 • edited

Choose a reason for hiding this comment

tjbck commented Oct 22, 2023

coolaj86 commented Oct 22, 2023

coolaj86 commented Oct 22, 2023 • edited

coolaj86 commented Oct 22, 2023 • edited

coolaj86 Oct 22, 2023 • edited

Choose a reason for hiding this comment

tjbck Oct 22, 2023

Choose a reason for hiding this comment

coolaj86 commented Oct 22, 2023

tjbck commented Oct 22, 2023

coolaj86 commented Oct 21, 2023 •

edited

Example `Caddyfile`

coolaj86 Oct 22, 2023 •

edited

tjbck Oct 22, 2023 •

edited

coolaj86 Oct 22, 2023 •

edited

coolaj86 commented Oct 22, 2023 •

edited

coolaj86 commented Oct 22, 2023 •

edited

coolaj86 Oct 22, 2023 •

edited