Meeting Notes

Lemonade Community Maintainers Meeting Notes

June 5, 2026

Attendees: (super)Mario, anditherobot, Anthony Mikinka (AMD), Bone, Daniel Holanda (AMD), fl0rianr, gavinbarnard, geramyl (mod), Jeremy Fowers (AMD), Kalin Ovtcharov (AMD), kenvandine, Krishna (AMD), mikkoph, Occam, Phoenix, primaL- (mod), sawan, TG95, Victoria (AMD)

Milestone for v10.7

• LMX omni models exposed to external OpenAI-compatible apps (Open WebUI, etc.), not just the Lemonade app
• Benchmarking tool ships with 10.7; early testers already finding non-optimal params on their machines
• Action: Finalize external app exposure for LMX models before tag

CI Testing / Health

• Hundreds of tests run on every PR; head of main is generally release-ready
• AMD-sponsored hardware covers Strix Halo on Windows and Linux
• Gap: no equivalent CI for Arm, Intel, CUDA
• Robust CI is the gating factor for faster release cadence
• Action: Ken to explore vendor donations of CI hardware via industry contacts

Release Cadence

• Strong consensus to move faster with more predictability
• Direction: weekly or bi-weekly stable tags on a fixed weekday (Wednesday preferred, avoid Friday)
• Cherry-picked point releases for urgent fixes
• Add a continuously refreshed "tip" channel (Ghostty-style) for developers wanting main builds
• Snap already does this via edge channel; Debian PPAs similar
• Windows signing capped daily, so signed Windows builds stay on stable cadence
• Unsigned Windows, Docker, Fedora could update daily
• Action: Continue discussion in Discord thread; Jeremy to share Ghostty link

Lemonade Working Groups

• New structure: six working groups, each led by a maintainer with autonomy to merge PRs and drive roadmap within scope
• Designed to unblock the ~80 open PR backlog and concentrate reviewer enthusiasm
• Some groups have a discrete end state (cross vendor), others are open-ended (omni)
• Separate maintainer table handles areas outside working groups (security, quality, distros, ROCm, etc.)
• Open question: possible new working group for backend / plugin system; Geramyl volunteered to lead
• Action: Leads to draft charters with scope and, where possible, completion criteria

Lemonade Omni Models

• Led by Jeremy
• Virtual omni via LMX router behind an OpenAI-style interface
• Already shipped: LMX representation, image-gen perf improvements, GUI import/export
• Next: expose to any OpenAI-compatible client (in 10.7)
• Halo Tales reference RPG app is coded, pending release
• Occam noted naming overlap with true omni research models; current design is intentional and gives per-model customization that single omni models lack
• Action: Jeremy to release Halo Tales after 10.7 ships

Lemonade Auto Tune

• Led by Michele
• Splits machine-specific args (global to server) from model-specific args
• Benchmarking tool layered on top to find optimized params
• Goal: community-shared profiles for common archetypes (Strix Halo, dGPU configs)
• Ships in 10.7, CLI only for now
• Not using llama-bench (needs to work with FLM, vLLM and other backends; uses normal inference endpoint instead)
• Custom scenarios supported: short, multiturn, long-context (32K, 64K), coding vs non-coding (relevant for MTP)
• Action: Community help wanted on more benchmark scenarios; UI button for benchmarking to come later

Cross Vendor Support

• Led by Ken Van Dine
• Goal: Lemonade runs on all mass-market hardware so app developers can target Lemonade regardless of backend (AMD, NVIDIA RTX/Spark, Intel, Qualcomm, Arm64)
• Discrete checkbox-style scope; dissolves when complete
• Ken to leverage vendor relationships through day job, including potentially sourcing CI hardware
• IoT and embedded (Canonical core devices) in scope; mobile may need Geramy
• Action: Ken to draft the hardware matrix and pursue vendor partnerships

Cloud Local Hybrid

• Led by Krishna
• Open PR adds cloud backend: bring your own OpenAI-compatible API key, provider models appear in Lemonade, chat works seamlessly
• Not about upselling to cloud
• Intermediate step toward intelligent routing between cloud and local based on user-defined policy
• Lemonade owns the platform and interface, not the policy
• Policy format TBD (markdown file, separate HTTP service, etc.)
• AMD research teams looking at this, plus alignment with vLLM Semantic Router
• Geramyl and Sawan asked for an API to retrieve, manage, and secure policies; Krishna agreed
• Action: Land scaffolding PR, then add basic router, then policy API

Lemonade UI

• Led by Kyle
• Stack stays React plus Tauri to avoid framework-change debate
• New capabilities:
  • Self-configuration (LLM loads model and builds recipe)
  • Pulls recommended settings from HuggingFace
  • Multiple-choice question tool for agent interactions
  • Markdown rendering
• Roughly a month from release-ready
• Florian helping; more contributors needed
• Time to market is the differentiator now, not the features themselves
• Action: Kyle to bring a roadmap to next Friday's meeting; testers and UI contributors wanted

Networking

• Led by Geramy, with Nathan (kg95) on attestation
• Mesh network vision: install Lemonade and reach your nodes from anywhere; add servers and mobile devices to a private mesh
• Tier 1 nodes require TPM 2.0 hardware attestation
• VMs require TEE; otherwise capped at Tier 2 or lower
• Security model includes ban plus master key rotation on detected abuse (self-healing)
• Router demoed: token-based API, delegates by occupancy and TPS across two Strix Halos plus two Thunderbolt 5 R9700s (over 110 TPS)
• Pulls existing Lemonade collections; supports new ones
• Open proposal: subscription or credits model so mobile users without high-end hardware can use community-donated inference, with a percentage flowing back to Lemonade
• Raises broader question about commercial ventures built on Lemonade and how close they sit to the project itself
• Action: Dedicated discussion next week or later; Geramy to organize separate call for interested parties

July 1, 2026

---

Release Highlights

• Non-Apple device support landed (Ken, Mario, Florian); big platform expansion
• New website with embedded SDK docs published; improves discoverability and adoption

Merge Process (Rebase Merges)

• Squash-only workflow breaks release-branch tagging; enabling rebase merges alongside squash as a fix
• Default stays squash; rebase reserved for clean, logically-split commits
• Action: Enable rebase merges as a trial next cycle (Mario has admin); maintainers check squash-vs-rebase before merging, don't blindly click

Versioning System

• Proposal to move from semver to date-based versioning (e.g. 2026.07.01) to avoid confusion and breaking-change flak
• Trade-off: loses marketing pop and semver signal, but less mental load given weekly releases
• Action: Mario to post a Discord poll (date-based vs semver), asking voters to justify their choice

PR Backlog

• Open PRs up to 90–100; limited reviewer bandwidth, plus duplicate issues/PRs, missing issue-links, and stale PRs (oldest ~4 months)
• Direction: automate cross-linking, auto-close on merge, flag large/new-contributor PRs, auto-tag maintainers, "stand down" tag for intentionally-open PRs; direct complex feature PRs to Discord
• Action: Krishna to expand the issue-labeling framework to handle these cases; issue-linking becomes part of review criteria

Backend Refactor & Policy

• Merged ~7000-line refactor cutting backend touch points from ~30 to ~5; enables adding ~10 more backends quickly
• Inclusion criteria now needed: value, usability, reachability, maintainability
• Action: Apply value/reach/maintainability basis for backend inclusion; formalize later

Beta Backends / Channels

• Consensus on one source tree with a beta toggle (disabled by default, own folder) instead of forks, which tend to die
• New backends land as beta, get bumped to stable later
• Action: Jeremy to open a PR integrating MLX behind a beta toggle; coordinate with Florian's existing MLX PR

Cross-Vendor / Edge Expansion

• Kyle wiring stable CDNA on Hot Aisle so Lemonade scales Strix Halo → CDNA and runs large models (GLM 5.2)
• Exploring Lemonade as a LiteRT stand-in and an Android backend for edge/telecom developers
• Action: Kyle to hand CDNA/Hot Aisle wiring to agents (targeting Thursday); chat offline with Krishna on VLLM + CDNA

CLI / TUI (Launch, Run, Pull)

• Building terminal UI for launch/run/pull; wants design feedback; heavy config should go through a dedicated onboard flow, not launch
• Mesh integration into the TUI requested as first-class
• Action: Open draft PRs for run and pull; gather feedback; coordinate with networking on mesh onboarding

GUI 3 (Merge Readiness)

• Strong baseline but scope creep risk (database, user accounts, auto-optimization)
• Strategy: define core launch feature set, disable/hide immature features rather than delay, land MVP sooner; defer database until mesh lands (local JSON for now)
• User accounts debated as arguably Open Web UI's lane; not freezing existing UI yet
• Action: Race GUI 3 to merge readiness; Florian to hide auto-optimization work until it functions; circle back on landing the MVP

July 17, 2026

---

1. Cloud/Local Router (Omni Router)

• ~75% complete on the current milestone; remaining work is polish, docs, and tests.
• UI design still under discussion: keep the default experience simple for new users, while allowing advanced setups (classifier models, embedding models, multi-source routing) for power users.
• Ken demoed a working prototype, "Nimbus" — a personal cloud appliance (LXD containers) that runs agents (e.g., pico claw) and routes their queries through Lemonade based on policy (image content, message length, or keyword → cloud; otherwise local). Shown live: local-vs-cloud query stats, automatic image-query routing to a GPU-equipped node, zero-config agent onboarding.
• Router policies will ship as downloadable JSON checkpoints on Hugging Face (e.g., cost-saving, privacy-first) that users can use as-is or fine-tune.
• Action: Close out remaining docs/tests/polish for the milestone. Ken to continue iterating on Nimbus and explore a "privacy-sensitive" routing recipe.

2. GUI 3 Transition

• Decision: freeze GUI 2 feature scope — bug-fixing and polish only from here.
• Plan: ship a GUI 2 beta (target Monday) via a new branch + beta tag, then run a 1–2 week feedback/bug-bash window before merging.
• Nexus's UI won't be ready for the beta; the Nexus "bundle" will instead be integrated directly into Lemonade around the same time so it can be tested as its own workstream.
• Noted for awareness only: LM Studio shipped a cloud-integration update this week ("LM Studio Bionic").
• Action: Florian to stand up the beta branch/tag. Team to block time for a bug-bash once the beta ships (Jeremy to review when back from vacation next week).

3. Lemonade Nexus (Mesh Networking)

• One remaining blocker: support for manually created certificates.
• Adding a file-transfer helper library so chat history/files can move between nodes.
• Auth model uses hierarchical, tree-based group permissions (no fine-grained ACLs planned).
• Status: close to functionally done ("barely a day away," per Nate) once the cert blocker clears.
• Practical blocker: needs a TEE-capable machine to host another tier-one node; Jeremy's home server is offline while he's traveling.
• Action: Resolve the manual-certificate gap. Find/allocate a TEE-capable machine for the next tier-one node.

4. CDNA / Studio Q Article

• Article in progress on the new CDNA integration story — prototype locally at low power, then move the same package/OS to CDNA in the cloud.
• Jon from Hot Aisle is reviewing/editing content and related PRs.
• Publish timing likely tied to when the CDNA bundle lands in a Lemonade release (release date still pending).
• Action: Ian/Jon to finish the article; align publish timing with the CDNA bundle's release.

5. Backend Version Bump & New Backend Work

• llama.cpp ROCm backend bump (713 → 714) targeted for the next release (~v11.1);
• Requires the updated llama.cpp revision plus a matching stable-diffusion.cpp update; ABI expected compatible across 713/714, so SD may be tested against both.
• VLLM and Moonshot backend updates are separate items;
• 714 adds hardware support for Gorgon Point devices; other optimizations TBD pending changelog review.

6. PR Review Process & Contribution Limits

• Problem: PR volume is outpacing maintainer review bandwidth.
• Ideas discussed:
  • Stale-bot to auto-close inactive PRs/issues (threshold TBD).
  • Cap open PRs per contributor tier — likely 1 (maybe 2) for contributors without write access; no cap for maintainers. GitHub's native PR-limit feature enforces a flat cap + bypass list but can't natively distinguish first-time vs. repeat contributors.
  • A bot to auto-close PRs beyond the limit was considered, with concern about how that lands with contributors.
  • Contributors could graduate off the limit after a track record of merged, good-quality PRs (e.g., 5–10).
  • Ken to check how the Ghostty project handles this — they use a "vouch" system to gate contributions.
• No final policy decided.
• Action: Ken to research the "vouch" model. Team to finalize PR-limit + stale thresholds next week (Mario won't attend — will weigh in async).

7. New Backend Inclusion Criteria (incl. VTE Backend PR)

• An open PR adding a VTE backend raised the broader question of acceptance criteria for new backends.
• Direction agreed: new backends land as "experimental" (preferred over "beta" — implies less permanence), graduating to first-class/production status only with demonstrated usage and active maintenance.
• UI: add a filter/toggle in the Backend Manager to show/hide experimental backends — hidden by default for new users; prompt users on backend updates about whether to keep something flagged experimental.
• Criteria to weigh: project maturity/maintainer activity, whether it fills a real gap, and security exposure (backends run as subprocesses with access to the Lemonade user's directory/service).
• Action: Florian to file an issue for the experimental-backend UI filter (ties into GUI 3). Team to draft a formal backend-inclusion policy; VTE PR decision pending that policy. Geramy to work on policy doc for experimental category.

8. Security & Project Growth

• Project is ~30 stars from 5,000 on GitHub — more visibility, more attack surface.
• Recent incident: a GitHub issue drew a convincing comment with a zip attachment from an account created about an hour earlier; Krishna deleted the comment before anyone opened the file.
• Discussed restricting issue/PR comment attachments to safe file types (e.g., images only) — unclear if GitHub allows this to be enforced.
• Discussed a security audit of the Lemonade codebase and the backends it pulls in, plus automation to flag risky changes (noting automation itself can be an abuse vector).
• Noted: an earlier internal scan of the codebase with an early-access Claude model had already surfaced several security items still being worked through.
• Action: Assign an owner for a security audit + scanning policy, ideally before the 5,000-star mark. Revisit alongside the PR-limit/stale-bot policy next week.

9. Versioning

• Confirmed: sticking with semantic versioning (per an earlier team poll).
• Current release is v11; next release with GUI3 will be v12 rather than v11.5, since GUI 3 counts as a breaking/major UI change.

10. ModelScope Integration

• Florian integrated ModelScope as a second model source alongside Hugging Face in GUI 3; users can toggle which source(s) to search.
• Rationale: serves AMD-hardware users in China (no Hugging Face access) and aligns with other silicon partners (Qualcomm, MediaTek, Broadcom) who also use ModelScope.
• ModelScope team is pleased with the integration; a joint promotion is being discussed, possibly including a homepage feature. ModelScope has roughly 200K daily active users, similar to Hugging Face, but is the only accessible option for users in China.
• Raised, not yet started: whether GUI 3 should support localization (e.g., Simplified Chinese) given the growing China-based audience. Flagged as a future discussion item.

Other (no action)

• Interest in wiring in LiteRT as a backend, potentially extending Lemonade to more devices (mobile, home routers, edge/telco) — feasible, not yet scoped.