[Do Not Merge] model : LFM2.5-Audio-1.5B #18641
Draft
+4,237
−278
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
model
4 tasks
tdakhran
force-pushed
the
tarek/feat/os-lfm2.5-audio-1.5b-upstream
branch
from
c275436 to
e1a8fd1
Compare
Contributor
Author
Collaborator
Member
|
If the string |
Collaborator
Or that. We just have to remember to remove them all from the merge message. :) |
Change is decoupled from ggml-org#18641. [LFM2.5-Audio-1.5B](https://huggingface.co/LiquidAI/LFM2.5-Audio-1.5B) needs streaming istft for generating output audio. * add streaming ISTFT class (`mtmd_audio_streaming_istft`) with overlap-add for audio reconstruction * replace global audio cache with per-instance cache, the model requires two independent caches, for preprocessing (audio input) and for istft (audio output). * unified templated FFT/IFFT implementation supporting both forward and inverse transforms
… tarek/feat/os-lfm2.5-audio-1.5b-upstream [no ci]
ngxson
pushed a commit
that referenced
this pull request
Jan 6, 2026
Change is decoupled from #18641. [LFM2.5-Audio-1.5B](https://huggingface.co/LiquidAI/LFM2.5-Audio-1.5B) needs streaming istft for generating output audio. * add streaming ISTFT class (`mtmd_audio_streaming_istft`) with overlap-add for audio reconstruction * replace global audio cache with per-instance cache, the model requires two independent caches, for preprocessing (audio input) and for istft (audio output). * unified templated FFT/IFFT implementation supporting both forward and inverse transforms
|
Hello Tarek, I am trying to build your WIP PR. With the last commit: 'Read n_layer from gguf', using LTO, building fails at the very end of building here: llama-server and llama-liquid-audio-server are succefully built, cli fails. If there is anything I can do to help testing let me know. Thank you so much. |
Contributor
Author
|
@elfarolab , mentioned commit didn't change anything related to compilation or LTO, could it be that there are stale object files somewhere? Tested that the clean build in UPD: it's related to miniaudio cli defines implementation here https://github.com/ggml-org/llama.cpp/pull/18641/changes#diff-73f13371b37801825dc2cdbfacadf9af40aef9dca4770d9dacbbe3534c7a7dacR13 , another implementation is defined in mtmd audio. try commenting this line |
|
Before building I delete the building destination directory every time. I always build llama.cpp the same way with the options above, never get failures. |
Contributor
Author
|
@elfarolab , it should work now, there were two implementations of miniaudio |
rebuilding |
|
Built successfully. that CUDA error: out of memory is weird, I am not running anything.. I reboot just in case |
|
with fresh system, anything GPU or CPU intensive running: free -h loading tensors.. then: free -h same error: |
|
audio file is < 30 sec duration I know, 24KHz looks strange but it is because later is used with libopus |
Contributor
Author
|
@elfarolab can you try with a smaller audio file (e.g., 4 seconds)? Add |
TEST Q4_0with fresh rebooted system, anything GPU or CPU intensive is running: tegrastats same error: looks like the model try to expand into the whole available memory and beyond Full Log |
Contributor
Author
|
@elfarolab , I was able to reproduce the issue on Orin Nano, will look into it |
Contributor
Author
|
@elfarolab temporary solution will be to build without VMM |
ok rebuilding now with -DGGML_CUDA_NO_VMM=ON |
Contributor
Author
|
The alternative is to build with VMM, but reduce the VMM size to 16 GB with --- a/ggml/src/ggml-cuda/ggml-cuda.cu
+++ b/ggml/src/ggml-cuda/ggml-cuda.cu
@@ -418,7 +418,7 @@ struct ggml_cuda_pool_leg : public ggml_cuda_pool {
// pool with virtual memory
#if defined(GGML_USE_VMM)
struct ggml_cuda_pool_vmm : public ggml_cuda_pool {
- static const size_t CUDA_POOL_VMM_MAX_SIZE = 1ull << 35; // 32 GB
+ static const size_t CUDA_POOL_VMM_MAX_SIZE = 1ull << 34; // 16 GB
int device;
CUdeviceptr pool_addr = 0; |
|
I found this discussion about VMM on Orin:
On Orin and on all Jetson series and DGX Spark memory is unified. Thank you! |
Contributor
Author
|
Orin Nano reports lfm2.5-audio creates 4 cuda context and 4 cuda pools, with CUDA_POOL_VMM_MAX_SIZE =16GB, they fit, with CUDA_POOL_VMM_MAX_SIZE = 32GB they don't. For 4070 For Orin |
|
it works with -DGGML_CUDA_NO_VMM=ON |
Do you suggest to use -DGGML_CUDA_NO_VMM=ON or rather to reduce the VMM memory size with: CUDA_POOL_VMM_MAX_SIZE =16GB |
Contributor
Author
I suggest trying both and picking the option that works faster |
|
I can confirm that also ASR with llama-liquid-audio-server works as expected in streaming mode only. Do you plan to integrate your changes to llama-server when everything is working? Thank you so much. |
Contributor
Author
|
@elfarolab , thanks for testing!
Note that ASR works with the existing llama-cli from the main branch.
We are working on it. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Liquid AI released LFM2.5-Audio-1.5B.
This PR is intended to provide a functional implementation in
llama.cppuntil necessary infrastructure is implemented.The plan is to split and merge it into upstream in smaller chunks, while keeping and tracking functional implementation here. It will be rebased from time to time.
GGUFs, precompiled runners, and instructions, live in https://huggingface.co/LiquidAI/LFM2.5-Audio-1.5B-GGUF.
Merge plan:
n_embd_outmodel : add LFM2-ColBert-350M #18607Demo of capabilities (watch with audio on)
demo.mp4
Thank you, @ngxson for the help!