Skip to content

Conversation

@danielholanda
Copy link
Contributor

Description

This PR adds models labeled ryzenai-hybrid and ryzenai-npu to the Lemonade app instructions. This makes Lemonade the first Hugging Face app to offer instructions on how to run AMD NPU-accelerated models.

Once this PR is merged, we intend add the proposed tags to all AMD compatible models.

Currently, two models are labeled to test this integration: amd/Phi-3-mini-4k-instruct-awq-g128-int4-asym-fp16-onnx-hybrid and amd/Phi-3-mini-4k-instruct-awq-g128-int4-asym-bf16-onnx-ryzen-strix.

Note on implementation

@krampstudio during our last PR you made a change called "multi line snippet and format" as shown here. Breaking the lines with \ as you proposed caused the instructions to not work for Windows users (they had to manually remove those slashes). I reverted this change here. Please let me know if that is ok or if those are absolutely needed for the content to be properly displayed.

How it looks

Hybrid model example

1. Pull the model
-----------------
Setup:
  # Download Lemonade from https://lemonade-server.ai/

Content:
  lemonade-server pull user.Phi-3-mini-4k-instruct-Hybrid --checkpoint amd/Phi-3-mini-4k-instruct-awq-g128-int4-asym-fp16-onnx-hybrid --recipe oga-hybrid
  # Note: If you installed from source, use the lemonade-server-dev command instead.

2. Run and chat with the model (requires RyzenAI 300 series)
------------------------------------------------------------
Content:
  lemonade-server run user.Phi-3-mini-4k-instruct-Hybrid

3. List all available models
----------------------------
Content:
  lemonade-server list

NPU model example

1. Pull the model
-----------------
Setup:
  # Download Lemonade from https://lemonade-server.ai/

Content:
  lemonade-server pull user.Phi-3-mini-4k-instruct-NPU --checkpoint amd/Phi-3-mini-4k-instruct-awq-g128-int4-asym-bf16-onnx-ryzen-strix --recipe oga-npu
  # Note: If you installed from source, use the lemonade-server-dev command instead.

2. Run and chat with the model (requires RyzenAI 300 series)
------------------------------------------------------------
Content:
  lemonade-server run user.Phi-3-mini-4k-instruct-NPU

3. List all available models
----------------------------
Content:
  lemonade-server list

GGUF model example

1. Pull the model
-----------------
Setup:
  # Download Lemonade from https://lemonade-server.ai/

Content:
  lemonade-server pull user.gpt-oss-20b-GGUF --checkpoint unsloth/gpt-oss-20b-GGUF:{{QUANT_TAG}} --recipe llamacpp
  # Note: If you installed from source, use the lemonade-server-dev command instead.

2. Run and chat with the model
------------------------------
Content:
  lemonade-server run user.gpt-oss-20b-GGUF

3. List all available models
----------------------------
Content:
  lemonade-server list

@Vaibhavs10 Please review :)

@danielholanda
Copy link
Contributor Author

@krampstudio I would also appreciate if you could verify whether the nice Quant dropdown menu is still showing up on GGUF models after the changes proposed above. I think this should be fine, but I could not verify on my side.
image

Copy link
Member

@Vaibhavs10 Vaibhavs10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay with me but let's wait for @krampstudio to give a review too before merge!

@krampstudio
Copy link
Collaborator

krampstudio commented Sep 8, 2025

@krampstudio during our last PR you made a change called "multi line snippet and format" as shown here. Breaking the lines with \ as you proposed caused the instructions to not work for Windows users (they had to manually remove those slashes). I reverted this change here. Please let me know if that is ok or if those are absolutely needed for the content to be properly displayed.

Yes, the single line command doesn't really look good: the quant selector doesn't appear without scrolling and a part of the command remains hidden:

Screenshot 2025-09-08 at 09 52 32

We usually prefer for the command to look good in the ui, and we use this pattern in many places (assuming it'll work for most of the users including windows users on WSL).

But if lemonde's users are mostly on windows and you think it's a barrier to adoption we can keep it single line.

@danielholanda
Copy link
Contributor Author

Yeah, most Lemonade users seem to be on Windows now, so keeping it as a single line is preferable. Let's keep it that way for now if that is ok.

@krampstudio Please review and let me know if you have any other suggestions. Otherwise, we should be good to merge.

@danielholanda
Copy link
Contributor Author

@krampstudio Thanks for approving! Looks like we are now only waiting for workflows to be approved to merge!

@krampstudio krampstudio merged commit 30d4b20 into huggingface:main Sep 10, 2025
2 of 4 checks passed
@krampstudio
Copy link
Collaborator

@danielholanda I'll ping you once deployed on hf.co

@danielholanda
Copy link
Contributor Author

danielholanda commented Sep 12, 2025

@krampstudio I noticed it has been deployed, but I do see a bug that appears to be outside the scope of this PR.

Let me know if this is something you can easily take care of here or if you would like me to open an issue elsewhere.

The bug

The "Use this model" dropdown does not show up at all if Lemonade is the only provider for that model. It only shows up if the user already configured Lemonade as a preferred local app.

Here is a model that shows this:
https://huggingface.co/amd/Phi-3-mini-4k-instruct-awq-g128-int4-asym-fp16-onnx-hybrid

When logged in and app pre-selected

image

When not logged in with app pre-selected (e.g. incognito mode)

image

Expected behavior

(Assuming no user configurations) If Lemonade is the only provider available for that model, the "Use this model" button dropdown should show up and include Lemonade there.

Suggested Solution

My understanding is that this issue happens because Lemonade is not marked as a local app that is suggested by default. Making it one of the apps suggested by default (similar to what it was when it was considered a "new" app) should fix this. Lemonade is developed by AMD, which should help justify making this part of the default list.

You could also change the logic of how/when the "Use this model" dropdown shows up in different ways, which would also solve the issue. That said, the solution above would be definitely preferred :)

@danielholanda
Copy link
Contributor Author

I created an issue here to raise visibility
#1751

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants