Add -w/--webui switch for llama-cpp webui integration#15
Merged
Conversation
- Add -w/--webui CLI flag to enable webui and open browser - Add llama.cpp as git submodule (sparse checkout for webui only) - Add /props endpoint for llama.cpp webui compatibility - Add webui serving with gzip decompression and CSS injection - Make 'model' field optional in chat completion requests - Add Makefile targets: submodules, webui, build-with-webui - Update build-portable.sh to include webui resources - Update .gitignore for webui build artifacts The webui uses OpenAI-compatible endpoints (/v1/chat/completions, /v1/models, /health) which are already implemented. CSS injection hides the attachment button since AFM doesn't support file uploads. Note: Most webui settings (penalties, top-k, etc.) have no effect as Apple Foundation Model only supports temperature parameter. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Homebrew-style paths to webui discovery (/usr/local/share/afm/webui/, /opt/homebrew/share/afm/webui/) - Update create-distribution.sh to include webui in tarball - Update portable install script to install webui to share directory This ensures the webui works when installed via Homebrew tap or portable distribution package. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Reviewer's GuideAdds an optional llama.cpp-compatible web UI mode (toggled via -w/--webui) that serves a gzip-compressed SPA, exposes a /props endpoint, and relaxes chat completion model requirements, along with build/distribution plumbing for bundling the web UI and llama.cpp submodule. Sequence diagram for webui request handling with gzip and CSS injectionsequenceDiagram
actor User
participant Browser
participant Server
participant FileSystem
User->>Browser: Navigate to afm URL (afm -w)
Browser->>Server: GET /
Server->>FileSystem: Read index.html.gz
FileSystem-->>Server: Gzipped HTML bytes
Server->>Server: gunzip(data)
alt Decompression succeeds
Server->>Server: Inject customCSS before </head>
Server-->>Browser: 200 OK
note right of Server: contentType=text/html
else Decompression fails
Server-->>Browser: 200 OK (gzip payload)
note right of Server: contentEncoding=gzip
end
Browser->>Server: GET /props
Server-->>Browser: 200 OK PropsResponse JSON
Updated class diagram for server, commands, and webui-related modelsclassDiagram
class Server {
- app: Application
- port: Int
- hostname: String
- verbose: Bool
- streamingEnabled: Bool
- instructions: String
- adapter: String?
- temperature: Double?
- randomness: String?
- permissiveGuardrails: Bool
- webuiEnabled: Bool
- webuiPath: String?
+ init(port: Int, hostname: String, verbose: Bool, streamingEnabled: Bool, instructions: String, adapter: String?, temperature: Double?, randomness: String?, permissiveGuardrails: Bool, webuiEnabled: Bool) async throws
+ start() async throws
+ shutdown()
- serveWebuiWithCustomCSS(webuiFilePath: String, req: Request) async throws -> Response
- openBrowser(url: String)
- static findWebuiPath() -> String?
- static gunzip(data: Data) throws -> Data
- static customCSS: String
}
class ChatCompletionsController {
- streamingEnabled: Bool
- instructions: String
- adapter: String?
- temperature: Double?
- randomness: String?
- permissiveGuardrails: Bool
+ boot(routes: RoutesBuilder) throws
+ create(req: Request) async throws -> Response
}
class ChatCompletionRequest {
+ model: String?
+ messages: [Message]
+ temperature: Double?
+ maxTokens: Int?
}
class ModelsResponse {
+ object: String
+ data: [ModelInfo]
}
class ModelInfo {
+ id: String
+ object: String
+ created: Int
+ owned_by: String
}
class PropsResponse {
+ default_generation_settings: DefaultGenerationSettings
+ total_slots: Int
+ model_path: String
+ role: String
+ modalities: Modalities
+ chat_template: String
+ bos_token: String
+ eos_token: String
+ build_info: String
}
class DefaultGenerationSettings {
+ n_ctx: Int
+ params: GenerationParams
}
class GenerationParams {
+ n_predict: Int
+ temperature: Double
+ top_k: Int
+ top_p: Double
+ min_p: Double
+ stream: Bool
+ max_tokens: Int
}
class Modalities {
+ vision: Bool
+ audio: Bool
}
class ServeCommand {
+ port: Int
+ hostname: String
+ verbose: Bool
+ noStreaming: Bool
+ instructions: String
+ adapter: String?
+ temperature: Double?
+ randomness: String?
+ permissiveGuardrails: Bool
+ webui: Bool
+ run() throws
}
class RootCommand {
+ port: Int
+ hostname: String
+ verbose: Bool
+ noStreaming: Bool
+ instructions: String
+ adapter: String?
+ temperature: Double?
+ randomness: String?
+ permissiveGuardrails: Bool
+ webui: Bool
+ run() throws
}
Server o-- PropsResponse
PropsResponse o-- DefaultGenerationSettings
DefaultGenerationSettings o-- GenerationParams
PropsResponse o-- Modalities
ModelsResponse o-- ModelInfo
ChatCompletionsController o-- ChatCompletionRequest
ServeCommand --> Server
RootCommand --> ServeCommand
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 1 issue, and left some high level feedback:
- The custom
gunzipimplementation uses a fixed 10MB buffer andcompression_decode_bufferonce, which risks truncating larger webui bundles; consider either streaming/decompressing in chunks or dynamically growing the buffer until decompression succeeds. - Auto-opening the browser whenever
-w/--webuiis enabled may be undesirable in headless/CI or remote server scenarios; consider making the browser launch optional (e.g., a separate flag or tied toverbose) while still serving the web UI. - In
create-distribution.sh, the install script message now saysRequires macOS 26+, which looks like a typo compared to the previous15.1+; please confirm and correct the version string if needed.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The custom `gunzip` implementation uses a fixed 10MB buffer and `compression_decode_buffer` once, which risks truncating larger webui bundles; consider either streaming/decompressing in chunks or dynamically growing the buffer until decompression succeeds.
- Auto-opening the browser whenever `-w/--webui` is enabled may be undesirable in headless/CI or remote server scenarios; consider making the browser launch optional (e.g., a separate flag or tied to `verbose`) while still serving the web UI.
- In `create-distribution.sh`, the install script message now says `Requires macOS 26+`, which looks like a typo compared to the previous `15.1+`; please confirm and correct the version string if needed.
## Individual Comments
### Comment 1
<location> `create-distribution.sh:110` </location>
<code_context>
+echo "Start with webui: afm -w"
echo ""
-echo "Note: Requires macOS 15.1+ and Apple Intelligence enabled"
+echo "Note: Requires macOS 26+ and Apple Intelligence enabled"
EOF
</code_context>
<issue_to_address>
**issue (typo):** The minimum macOS version in the installer message appears to be a typo.
The script now prints `Requires macOS 26+`, which looks accidental and may mislead users or packagers about supported versions. If the minimum hasn’t actually changed, this should remain `macOS 15.1+` (or whatever the correct minimum is) to reflect the real requirement.
```suggestion
echo "Note: Requires macOS 15.1+ and Apple Intelligence enabled"
```
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| echo "Start with webui: afm -w" | ||
| echo "" | ||
| echo "Note: Requires macOS 15.1+ and Apple Intelligence enabled" | ||
| echo "Note: Requires macOS 26+ and Apple Intelligence enabled" |
There was a problem hiding this comment.
issue (typo): The minimum macOS version in the installer message appears to be a typo.
The script now prints Requires macOS 26+, which looks accidental and may mislead users or packagers about supported versions. If the minimum hasn’t actually changed, this should remain macOS 15.1+ (or whatever the correct minimum is) to reflect the real requirement.
Suggested change
| echo "Note: Requires macOS 26+ and Apple Intelligence enabled" | |
| echo "Note: Requires macOS 15.1+ and Apple Intelligence enabled" |
- Inject JavaScript to rebrand webui ("Apple Foundation Models" instead of "llama.cpp")
- Change subtitle to "Type a message to get started" (removes upload reference)
- Update Makefile to document pinned llama.cpp commit
- Add submodule-status target to show pinned versions
- Remove --recursive flag (not needed for webui-only sparse checkout)
The llama.cpp submodule is pinned to commit 0e4ebeb05 for reproducible builds.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
-w/--webuiCLI flag to enable webui and open browser automatically/propsendpoint for llama.cpp webui compatibilitymodelfield optional in chat completion requests (webui compatibility)submodules,webui,build-with-webuiUsage
Test plan
-wand--webuiflags work/healthendpoint with webui enabled/v1/modelsendpoint with webui enabled/v1/chat/completions(non-streaming) with webui enabled/v1/chat/completions(streaming) with webui enabledmodelfield (webui style)/propsendpoint returns expected formatNotes
make webui), not at runtime🤖 Generated with Claude Code
Summary by Sourcery
Add optional llama.cpp-compatible web UI integration and metadata endpoints, and update build and distribution tooling to bundle and serve the web UI when requested.
New Features:
Enhancements:
Build: