Skip to content

Meeting Notes

Jump to bottom
Ramakrishnan Sivakumar edited this page Aug 15, 2026 · 7 revisions

Lemonade Community Maintainers Meeting Notes

June 5, 2026

Attendees: (super)Mario, anditherobot, Anthony Mikinka (AMD), Bone, Daniel Holanda (AMD), fl0rianr, gavinbarnard, geramyl (mod), Jeremy Fowers (AMD), Kalin Ovtcharov (AMD), kenvandine, Krishna (AMD), mikkoph, Occam, Phoenix, primaL- (mod), sawan, TG95, Victoria (AMD)

Milestone for v10.7

  • LMX omni models exposed to external OpenAI-compatible apps (Open WebUI, etc.), not just the Lemonade app
  • Benchmarking tool ships with 10.7; early testers already finding non-optimal params on their machines
  • Action: Finalize external app exposure for LMX models before tag

CI Testing / Health

  • Hundreds of tests run on every PR; head of main is generally release-ready
  • AMD-sponsored hardware covers Strix Halo on Windows and Linux
  • Gap: no equivalent CI for Arm, Intel, CUDA
  • Robust CI is the gating factor for faster release cadence
  • Action: Ken to explore vendor donations of CI hardware via industry contacts

Release Cadence

  • Strong consensus to move faster with more predictability
  • Direction: weekly or bi-weekly stable tags on a fixed weekday (Wednesday preferred, avoid Friday)
  • Cherry-picked point releases for urgent fixes
  • Add a continuously refreshed "tip" channel (Ghostty-style) for developers wanting main builds
  • Snap already does this via edge channel; Debian PPAs similar
  • Windows signing capped daily, so signed Windows builds stay on stable cadence
  • Unsigned Windows, Docker, Fedora could update daily
  • Action: Continue discussion in Discord thread; Jeremy to share Ghostty link

Lemonade Working Groups

  • New structure: six working groups, each led by a maintainer with autonomy to merge PRs and drive roadmap within scope
  • Designed to unblock the ~80 open PR backlog and concentrate reviewer enthusiasm
  • Some groups have a discrete end state (cross vendor), others are open-ended (omni)
  • Separate maintainer table handles areas outside working groups (security, quality, distros, ROCm, etc.)
  • Open question: possible new working group for backend / plugin system; Geramyl volunteered to lead
  • Action: Leads to draft charters with scope and, where possible, completion criteria

Lemonade Omni Models

  • Led by Jeremy
  • Virtual omni via LMX router behind an OpenAI-style interface
  • Already shipped: LMX representation, image-gen perf improvements, GUI import/export
  • Next: expose to any OpenAI-compatible client (in 10.7)
  • Halo Tales reference RPG app is coded, pending release
  • Occam noted naming overlap with true omni research models; current design is intentional and gives per-model customization that single omni models lack
  • Action: Jeremy to release Halo Tales after 10.7 ships

Lemonade Auto Tune

  • Led by Michele
  • Splits machine-specific args (global to server) from model-specific args
  • Benchmarking tool layered on top to find optimized params
  • Goal: community-shared profiles for common archetypes (Strix Halo, dGPU configs)
  • Ships in 10.7, CLI only for now
  • Not using llama-bench (needs to work with FLM, vLLM and other backends; uses normal inference endpoint instead)
  • Custom scenarios supported: short, multiturn, long-context (32K, 64K), coding vs non-coding (relevant for MTP)
  • Action: Community help wanted on more benchmark scenarios; UI button for benchmarking to come later

Cross Vendor Support

  • Led by Ken Van Dine
  • Goal: Lemonade runs on all mass-market hardware so app developers can target Lemonade regardless of backend (AMD, NVIDIA RTX/Spark, Intel, Qualcomm, Arm64)
  • Discrete checkbox-style scope; dissolves when complete
  • Ken to leverage vendor relationships through day job, including potentially sourcing CI hardware
  • IoT and embedded (Canonical core devices) in scope; mobile may need Geramy
  • Action: Ken to draft the hardware matrix and pursue vendor partnerships

Cloud Local Hybrid

  • Led by Krishna
  • Open PR adds cloud backend: bring your own OpenAI-compatible API key, provider models appear in Lemonade, chat works seamlessly
  • Not about upselling to cloud
  • Intermediate step toward intelligent routing between cloud and local based on user-defined policy
  • Lemonade owns the platform and interface, not the policy
  • Policy format TBD (markdown file, separate HTTP service, etc.)
  • AMD research teams looking at this, plus alignment with vLLM Semantic Router
  • Geramyl and Sawan asked for an API to retrieve, manage, and secure policies; Krishna agreed
  • Action: Land scaffolding PR, then add basic router, then policy API

Lemonade UI

  • Led by Kyle
  • Stack stays React plus Tauri to avoid framework-change debate
  • New capabilities:
    • Self-configuration (LLM loads model and builds recipe)
    • Pulls recommended settings from HuggingFace
    • Multiple-choice question tool for agent interactions
    • Markdown rendering
  • Roughly a month from release-ready
  • Florian helping; more contributors needed
  • Time to market is the differentiator now, not the features themselves
  • Action: Kyle to bring a roadmap to next Friday's meeting; testers and UI contributors wanted

Networking

  • Led by Geramy, with Nathan (kg95) on attestation
  • Mesh network vision: install Lemonade and reach your nodes from anywhere; add servers and mobile devices to a private mesh
  • Tier 1 nodes require TPM 2.0 hardware attestation
  • VMs require TEE; otherwise capped at Tier 2 or lower
  • Security model includes ban plus master key rotation on detected abuse (self-healing)
  • Router demoed: token-based API, delegates by occupancy and TPS across two Strix Halos plus two Thunderbolt 5 R9700s (over 110 TPS)
  • Pulls existing Lemonade collections; supports new ones
  • Open proposal: subscription or credits model so mobile users without high-end hardware can use community-donated inference, with a percentage flowing back to Lemonade
  • Raises broader question about commercial ventures built on Lemonade and how close they sit to the project itself
  • Action: Dedicated discussion next week or later; Geramy to organize separate call for interested parties

July 1, 2026

Release Highlights

  • Non-Apple device support landed (Ken, Mario, Florian); big platform expansion
  • New website with embedded SDK docs published; improves discoverability and adoption

Merge Process (Rebase Merges)

  • Squash-only workflow breaks release-branch tagging; enabling rebase merges alongside squash as a fix
  • Default stays squash; rebase reserved for clean, logically-split commits
  • Action: Enable rebase merges as a trial next cycle (Mario has admin); maintainers check squash-vs-rebase before merging, don't blindly click

Versioning System

  • Proposal to move from semver to date-based versioning (e.g. 2026.07.01) to avoid confusion and breaking-change flak
  • Trade-off: loses marketing pop and semver signal, but less mental load given weekly releases
  • Action: Mario to post a Discord poll (date-based vs semver), asking voters to justify their choice

PR Backlog

  • Open PRs up to 90–100; limited reviewer bandwidth, plus duplicate issues/PRs, missing issue-links, and stale PRs (oldest ~4 months)
  • Direction: automate cross-linking, auto-close on merge, flag large/new-contributor PRs, auto-tag maintainers, "stand down" tag for intentionally-open PRs; direct complex feature PRs to Discord
  • Action: Krishna to expand the issue-labeling framework to handle these cases; issue-linking becomes part of review criteria

Backend Refactor & Policy

  • Merged ~7000-line refactor cutting backend touch points from ~30 to ~5; enables adding ~10 more backends quickly
  • Inclusion criteria now needed: value, usability, reachability, maintainability
  • Action: Apply value/reach/maintainability basis for backend inclusion; formalize later

Beta Backends / Channels

  • Consensus on one source tree with a beta toggle (disabled by default, own folder) instead of forks, which tend to die
  • New backends land as beta, get bumped to stable later
  • Action: Jeremy to open a PR integrating MLX behind a beta toggle; coordinate with Florian's existing MLX PR

Cross-Vendor / Edge Expansion

  • Kyle wiring stable CDNA on Hot Aisle so Lemonade scales Strix Halo → CDNA and runs large models (GLM 5.2)
  • Exploring Lemonade as a LiteRT stand-in and an Android backend for edge/telecom developers
  • Action: Kyle to hand CDNA/Hot Aisle wiring to agents (targeting Thursday); chat offline with Krishna on VLLM + CDNA

CLI / TUI (Launch, Run, Pull)

  • Building terminal UI for launch/run/pull; wants design feedback; heavy config should go through a dedicated onboard flow, not launch
  • Mesh integration into the TUI requested as first-class
  • Action: Open draft PRs for run and pull; gather feedback; coordinate with networking on mesh onboarding

GUI 3 (Merge Readiness)

  • Strong baseline but scope creep risk (database, user accounts, auto-optimization)
  • Strategy: define core launch feature set, disable/hide immature features rather than delay, land MVP sooner; defer database until mesh lands (local JSON for now)
  • User accounts debated as arguably Open Web UI's lane; not freezing existing UI yet
  • Action: Race GUI 3 to merge readiness; Florian to hide auto-optimization work until it functions; circle back on landing the MVP

July 17, 2026

1. Cloud/Local Router (Omni Router)

  • ~75% complete on the current milestone; remaining work is polish, docs, and tests.
  • UI design still under discussion: keep the default experience simple for new users, while allowing advanced setups (classifier models, embedding models, multi-source routing) for power users.
  • Ken demoed a working prototype, "Nimbus" — a personal cloud appliance (LXD containers) that runs agents (e.g., pico claw) and routes their queries through Lemonade based on policy (image content, message length, or keyword → cloud; otherwise local). Shown live: local-vs-cloud query stats, automatic image-query routing to a GPU-equipped node, zero-config agent onboarding.
  • Router policies will ship as downloadable JSON checkpoints on Hugging Face (e.g., cost-saving, privacy-first) that users can use as-is or fine-tune.
  • Action: Close out remaining docs/tests/polish for the milestone. Ken to continue iterating on Nimbus and explore a "privacy-sensitive" routing recipe.

2. GUI 3 Transition

  • Decision: freeze GUI 2 feature scope — bug-fixing and polish only from here.
  • Plan: ship a GUI 2 beta (target Monday) via a new branch + beta tag, then run a 1–2 week feedback/bug-bash window before merging.
  • Nexus's UI won't be ready for the beta; the Nexus "bundle" will instead be integrated directly into Lemonade around the same time so it can be tested as its own workstream.
  • Noted for awareness only: LM Studio shipped a cloud-integration update this week ("LM Studio Bionic").
  • Action: Florian to stand up the beta branch/tag. Team to block time for a bug-bash once the beta ships (Jeremy to review when back from vacation next week).

3. Lemonade Nexus (Mesh Networking)

  • One remaining blocker: support for manually created certificates.
  • Adding a file-transfer helper library so chat history/files can move between nodes.
  • Auth model uses hierarchical, tree-based group permissions (no fine-grained ACLs planned).
  • Status: close to functionally done ("barely a day away," per Nate) once the cert blocker clears.
  • Practical blocker: needs a TEE-capable machine to host another tier-one node; Jeremy's home server is offline while he's traveling.
  • Action: Resolve the manual-certificate gap. Find/allocate a TEE-capable machine for the next tier-one node.

4. CDNA / Studio Q Article

  • Article in progress on the new CDNA integration story — prototype locally at low power, then move the same package/OS to CDNA in the cloud.
  • Jon from Hot Aisle is reviewing/editing content and related PRs.
  • Publish timing likely tied to when the CDNA bundle lands in a Lemonade release (release date still pending).
  • Action: Ian/Jon to finish the article; align publish timing with the CDNA bundle's release.

5. Backend Version Bump & New Backend Work

  • llama.cpp ROCm backend bump (713 → 714) targeted for the next release (~v11.1);
  • Requires the updated llama.cpp revision plus a matching stable-diffusion.cpp update; ABI expected compatible across 713/714, so SD may be tested against both.
  • VLLM and Moonshot backend updates are separate items;
  • 714 adds hardware support for Gorgon Point devices; other optimizations TBD pending changelog review.

6. PR Review Process & Contribution Limits

  • Problem: PR volume is outpacing maintainer review bandwidth.
  • Ideas discussed:
    • Stale-bot to auto-close inactive PRs/issues (threshold TBD).
    • Cap open PRs per contributor tier — likely 1 (maybe 2) for contributors without write access; no cap for maintainers. GitHub's native PR-limit feature enforces a flat cap + bypass list but can't natively distinguish first-time vs. repeat contributors.
    • A bot to auto-close PRs beyond the limit was considered, with concern about how that lands with contributors.
    • Contributors could graduate off the limit after a track record of merged, good-quality PRs (e.g., 5–10).
    • Ken to check how the Ghostty project handles this — they use a "vouch" system to gate contributions.
  • No final policy decided.
  • Action: Ken to research the "vouch" model. Team to finalize PR-limit + stale thresholds next week (Mario won't attend — will weigh in async).

7. New Backend Inclusion Criteria (incl. VTE Backend PR)

  • An open PR adding a VTE backend raised the broader question of acceptance criteria for new backends.
  • Direction agreed: new backends land as "experimental" (preferred over "beta" — implies less permanence), graduating to first-class/production status only with demonstrated usage and active maintenance.
  • UI: add a filter/toggle in the Backend Manager to show/hide experimental backends — hidden by default for new users; prompt users on backend updates about whether to keep something flagged experimental.
  • Criteria to weigh: project maturity/maintainer activity, whether it fills a real gap, and security exposure (backends run as subprocesses with access to the Lemonade user's directory/service).
  • Action: Florian to file an issue for the experimental-backend UI filter (ties into GUI 3). Team to draft a formal backend-inclusion policy; VTE PR decision pending that policy. Geramy to work on policy doc for experimental category.

8. Security & Project Growth

  • Project is ~30 stars from 5,000 on GitHub — more visibility, more attack surface.
  • Recent incident: a GitHub issue drew a convincing comment with a zip attachment from an account created about an hour earlier; Krishna deleted the comment before anyone opened the file.
  • Discussed restricting issue/PR comment attachments to safe file types (e.g., images only) — unclear if GitHub allows this to be enforced.
  • Discussed a security audit of the Lemonade codebase and the backends it pulls in, plus automation to flag risky changes (noting automation itself can be an abuse vector).
  • Noted: an earlier internal scan of the codebase with an early-access Claude model had already surfaced several security items still being worked through.
  • Action: Assign an owner for a security audit + scanning policy, ideally before the 5,000-star mark. Revisit alongside the PR-limit/stale-bot policy next week.

9. Versioning

  • Confirmed: sticking with semantic versioning (per an earlier team poll).
  • Current release is v11; next release with GUI3 will be v12 rather than v11.5, since GUI 3 counts as a breaking/major UI change.

10. ModelScope Integration

  • Florian integrated ModelScope as a second model source alongside Hugging Face in GUI 3; users can toggle which source(s) to search.
  • Rationale: serves AMD-hardware users in China (no Hugging Face access) and aligns with other silicon partners (Qualcomm, MediaTek, Broadcom) who also use ModelScope.
  • ModelScope team is pleased with the integration; a joint promotion is being discussed, possibly including a homepage feature. ModelScope has roughly 200K daily active users, similar to Hugging Face, but is the only accessible option for users in China.
  • Raised, not yet started: whether GUI 3 should support localization (e.g., Simplified Chinese) given the growing China-based audience. Flagged as a future discussion item.

Other (no action)

  • Interest in wiring in LiteRT as a backend, potentially extending Lemonade to more devices (mobile, home routers, edge/telco) — feasible, not yet scoped.

Lemonade Meetup: July 22, 2026

We were fortunate to have the opportunity for a Lemonade meet up at AMD Advancing AI Day! We discussed a range of topics from the user personas we serve, to the way we maintain the project, to the future roadmap and working groups.

Attendees: Jeremy, @mikkoph @geramyl @sofiageo @fl0rianr_89165 @clemperorpenguin @.supermario1 @danielhn4744 @ramkrishna2910 @kovtcharov_amd_74552 @avatarh @kenvandine @.sawan. @sreeram.20 Adrian Macias, Kevin Cazabon, Logan Wemyss, Ali Haidous, Rakesh Anigundi

Who we serve

Lemonade today serves (at least!) 3 personas, with 3 more emergent personas on the horizon.

Current personas:

  • Exploring local AI: people getting their first great experience with local LLMs, image gen, etc. Hopefully getting inspired and becoming a builder!
  • Personal AI server: connected Lemonade to great apps for productivity, coding, and creativity as part of daily workflows.
  • Software vendors: building Lemonade support into their own apps for first-class local AI support.

Emergent personas:

  • Personal agent appliance: plug-and-play device that works without a keyboard or monitor, and serves agents over wifi right away.
  • Enterprise token server: use local AI to replace cloud AI for the employees of a company.
  • Embedded: robotics, automotive, etc. use cases that rely on vision, VLAs, and Lemonade's current capability set.

Top Topics

Project Management


We have a consensus that the fast pace of the project has allowed bugs, issues, and PRs to pile up. We also want to take a look at future feature scope increases. Our goal is to ensure Lemonade and the core lemond SDK are a consistent experience for contributors, users, and downstream projects.

New GUI

The GUI3 project is nearly ready to merge! This is a complete overhaul of the Lemonade GUI. It adds many features for users and developers alike, and provides a new foundation for future work.

GUI3 also needs to do an even better job of referring users to state of the art apps like AnythingLLM, Hermes, and OpenCode that use Lemonade as an inference runtime.

We’re going to need everyone’s help with the beta!

AMD Skills

AMD has launched agent skills into the official skills marketplaces for Claude, OpenAI, and Cursor. This includes Lemonade-based skills! We have an opportunity to expand this catalogue, which can reach the massive user bases of those companies.

Next Initiatives

  • Improve the marketplace with metadata, mcp, install buttons, and links to more AMD playbooks.
  • Extend the mcp to let people completely explore lemonade in natural language.
  • Pursue first class support in third party apps (Hermes, OpenCode, etc.).
  • Daniel and Sawan to co-found a new Skills working group.
  • Daniel to investigate additional dGPU runners for CI.
  • Jeremy to found a QA working group.
    • Upgraded bug auditing: track amount of live bugs, and trace them back to the PR that introduced them, see if repo-manager caught it or not.
    • Code deduplication bot to combat the way agentic coding repeating functionality.
    • Geramy: Stress testing long running agents and finding graceful failure scenarios (telling the agent it ran out of memory, ran out of context, etc).
    • Add Radeon emulators to CI.
  • Declaring victory on the Omni Models working group! The foundation has been built, and we will continue maintaining and extending it over time.
  • Daniel and Florian to analyze the apparent overlap between the skills and MCP.
  • Krishna to recruit Ken, Sawan, and Geramy to explore the router functionality.
  • Florian to launch the GUI3 beta after we return from the conference!

Lemonade Maintainer Meeting, Minutes - Aug 14, 2026

1. GUI 3 MVP and Release Push

  • A functioning mobile GUI will land in this release, which has been a long-standing community ask.
  • Accessibility work included: Kyle added a11y tests covering screen readers and similar assistive tooling. Plus a large batch of new features.
  • Status: feature frozen and effectively architecture frozen. The remaining job is to land the MVP, merge to main, and release.
  • Acknowledged that this back-pressures a lot of contributor ideas waiting to merge. The pace is deliberately aggressive, possibly slightly too aggressive. Remaining work is finite but will not close out today.

2. Internationalization and Translation

  • Very strong demand for Chinese across Lemonade, consistent with the recent model config changes.
  • Framing: the goal right now is not to ship translation, it is to avoid architectural decisions that make translation painful later.
  • Target surface is the web GUI, not the Tauri shell. Tauri is just desktop window chrome around the web version and introduces little written language of its own.
  • Stack constraints discussed: React front end with text in JSX, no server-side rendering. Likely approach is a lookup or template function that swaps text chunks, plus locale files (the common pattern being XML-style translation files).
  • Concern raised that the diff from an automated refactor would be hard to review for silent breakage, so scope and diff shape matter.
  • AMD already has translation capability in house: Ben has translated playbooks into roughly 29 languages. Consensus that if this is an AMD business initiative, AMD should lead and fund the translation work.
  • Preference for keeping translated strings in separate files owned by separate people, so they are not constantly rewritten by feature work.
  • Action: Florian to run an AI scan of the codebase for translation readiness and post findings in Discord, tagging Jeremy and Mario.

3. Plugin System (Lemonade Backend Gateway)

  • Proposal is under discussion in a dedicated Discord channel. Link shared in chat. The effort lead was not on the call, so no decisions were taken.
  • Objective 1: a backend or engine should not need to be in-tree to be tried with Lemonade.
  • Objective 2: sandbox backends. Today Lemonade pulls a backend artifact from GitHub and runs it. As more backends from lesser known authors are added, both deliberate compromise and upstream supply chain attacks become more likely.
  • Preference expressed for OCI containers with a container store, minimal pinned images built from something like Alpine/musl, and lifecycle management around them, rather than pip-level isolation.
  • Security-first policy suggestion: standardized policies per engine. Text in, text out, defined acceptable file paths, no arbitrary network egress or DNS lookups.
  • Counterpoint raised: sandboxing that pins dependencies away from upstream security maintainers creates its own vulnerability surface. Published research puts the CVE exploitation window at roughly 8 hours, far tighter than the release cycle, so pinned libraries must be upgradable independently of a Lemonade release.
  • Action: Feedback goes to the proposal author in the Discord channel. Converge quickly on an MVP (llama.cpp working easily, plus sandboxing) and put everything else on a follow-on roadmap.

4. Benchmarking

  • Iswarya is building an ongoing llama.cpp performance dataset. Dimensions are model x backend x performance, where performance splits into TTFT/prefill tokens per second and decode tokens per second.
  • Coverage: hot models from Lemonade's model registry on a rolling basis, plus a fixed preset list so some models are tracked over long durations.
  • Backends today include Vulkan and several internal AMD llama.cpp forks, all public forks, so they plug in easily. More backends with better performance are known to be coming.
  • Pipeline: a nightly regression generates artifacts that are committed to a benchmark-data branch inside the Lemonade repo. An HTML dashboard reads the latest commit from that branch. Preference to move artifacts to GitHub releases later.
  • Current limits: short context only, capped at 4K by the llama.cpp bench harness. Long context is the next priority.
  • Hardware today is GFX1151 (Strix Halo), with a DGX box also available for some runs. Expanding to more hardware configurations is planned.
  • Intended payoff: before each release, use regression data to pick the best backend per model, so pulling a model also pulls the right backend for it.
  • Raised on the call: backend choice depends on hardware and concurrency as well as model (serving one user vs 100 vs 1000 points to different backends), so there are at least three decision layers. Lemonade Router already supports rule-based selection here; applicability to be checked.
  • George is building an outward-facing benchmarking website. Plan is Aishwarya's nightly data as the common baseline, plus community-submitted results from their own hardware and scenarios via the CLI bench tool and eventually the GUI.
  • The GUI 3 branch had local-benchmark code that was cut for MVP scope. It could return to support community result upload and sharing.
  • Action: Aishwarya and George to work out how nightly artifacts feed the public website.
  • Action: Aishwarya to add long context, expand backend coverage, and integrate with the backend plugin work so plugin authors can benchmark their backends.

5. Multi-Node Clustering (Strix Halo over Thunderbolt)

  • Clustering here means physically stacked boxes joined by a Thunderbolt cable, splitting one model at high speed across two machines. Not pooled cloud machines accessed one at a time.
  • The project cannot drive this without hardware, so the ask is for community members who own multiple Strix Halo boxes to work on it together. Several people on the call have such setups.
  • vLLM supports this today. A llama.cpp PR is in flight.
  • Works with the Thunderbolt 5 driver using RDMA. Reported working end to end after changes made against the repo. Throughput figure cited as roughly 50 to 60 percent, but the transcript is garbled on whether that is a gain or a caveat (verify).
  • Two Strix Halo machines with RDMA over Thunderbolt have been attached as GitHub runners so the RDMA driver can be tested.
  • Technical notes: the Thunderbolt/USB stream driver is not RDMA. The current implementation is RDMA verbs on top of existing infrastructure and still uses CPU to move data, so most of the gain is reduced overhead. Polling mode and newer kernel features are further upside.
  • An internal AMD team is working on this but is not ready to publish.
  • Existing published results live in Donato's Strix Halo home lab Discord channel and an associated GitHub discussion.

6. Issue and PR Triage

  • A local-AI PR triage project is up and doing a good job. Plan is to extend it to issues and use the same framework to burn down the backlog faster.
  • Manual gardening also done: comments left on the oldest page of PRs asking contributors to either make a merge plan or close.
  • Work stalled because CI broke and the GUI 3 release became the bottleneck. Now back to top-two priority.
  • Action: Resume and complete triage automation once GUI 3 ships.

7. CDNA Packaging and ROCm 7.14

  • Open question: ship CDNA tarballs the old way, via the experimental path, before Mario's new packaging work in PR #2768 lands (verify PR number).
  • CDNA has been working for over a month. A couple of PRs in the queue unblock the CI problems, including PR #3333 for vLLM ROCm (verify), which enables the old path with the existing comments addressed.
  • Argument for shipping now: all the RDNA work moves to the old method today, so bringing CDNA and RDNA into parallel before migrating everything to the new method in a week or two means fewer surprises, and gets Hot Aisle qualification sorted before Mario's work lands.
  • CI and hardware: Hot Aisle can be pulled per cycle to run the qualification leg, and that logic does not change when packaging changes. MI300 access on Fireworks was not obvious to set up, though there are Fireworks credits available to use.
  • Release behavior unchanged: stable artifacts either stay put when no qualification leg ran, or move forward when wheels are in place and qualified on GPU.
  • ROCm 7.14 packaging change worth broadcasting: the default acquisition path is now a PyPI wheel that gets downloaded and unzipped as your copy of ROCm. Expect to see pip installs during C++ builds. A config option to use tarballs remains, but tarballs are roughly twice the size. On Windows, wheels are tried first with a fallback if Python pieces are missing.
  • Action: Jeremy to re-review Mario's ROCm 7.14 PR immediately after the call, targeting merge today so it lands in the next release.
  • Action: Ian to land the queued CI-unblocking PRs and ship CDNA via the existing path.

8. HRX Backend for llama.cpp

  • The HRX team is targeting a public alpha shortly. Early Qwen 3 8B numbers were described as very good.
  • Plan once the alpha lands: wire it into the benchmarking suite and into Lemonade as an experimental backend. Expectations set at alpha quality.
  • Expected side benefit: smaller llama.cpp artifacts, since a complete copy of the ROCm kernel library is not needed to run it. Unreleased, so unverified.

9. Smart Router

  • Milestone 1 / MVP shipped with v11.5. Foundations are holding, with no need so far to revisit early design decisions.
  • Customer traction: conversations underway with customers other AMD teams are engaged with, good interest from several, and meetings already scheduled.
  • Phase 2 milestone is open, driven by issues found in real use plus ideas from the recent wave of router announcements, looking specifically for approaches not already tried.
  • Research issues opened on KV cache transfer and session transfer across different backends and heterogeneous environments.
  • Position taken: the unit of transfer is the session, not just KV cache. A session includes KV, tool calls, file handles and more. No portable session definition or spec exists today, which is what the vLLM collaboration is aimed at, including the security implications of moving a session package across a local/cloud boundary. Cross-backend transfer comes after that.
  • Reference point: an NVIDIA publication from roughly a week and a half prior on cross-model KV cache transfer, showing good results within a model family.
  • Use case framing from the discussion: work locally, ship the session to the cloud when the routing policy calls for it, run it there, then bring it back. Applies across local, on-prem and multiple cloud tiers, and potentially across models of different capability operating on the same session.
  • Near-term deliverables: ship routing policies through Hugging Face so users can try them without authoring their own, and get to working with Claude Code in one command.
  • Also engaging vLLM and folks at Google. Discord channel: cloud-local-hybrid.

10. Lemonade Nexus

  • Naming note: Fireworks announced a product called Nexus. Ours is Lemonade Nexus, small L, different thing.
  • Current work is remote vTPM attestation, the final piece before it is ready.
  • Mechanism: a VM's TPM partners with the TPM on a pre-approved machine running the server software, presents information for verification, and repeats across servers until all approve the join. Servers keep a local database of approved binary checksums, cross-attest as new releases appear in the GitHub repo, and allow both forward upgrades and backward downgrades without being denied network access.
  • User model: install Lemonade and that machine becomes an inference device on the network. Install Lemonade mobile and you reach all your devices, in network and out. Companies can create employee groups (IT, engineering, graphics) and grant device access by group. No driver installs or special setup required.
  • The network is democratically distributed. The initial binary and server is currently master, which is a risk until quorum is reached. Four servers are now running, which is judged barely enough to start a quorum and distribute the master key, after which no single server holds it.
  • Called out as critical for the new GUI, since users will expect accounts and saved data. Argues for moving fast here.
  • Phase 2 is the community network, an open-router-style but community-based model with a token exchange. Usage and borrowing tracking already exists, and much of the exchange code is written, but the design needs care so that Lemonade is not the only possible exchange. Gateway participation, where others run gateways and get paid for it, is also being considered.

Clone this wiki locally