Skip to content

Support launching with --no-mmproj-offload or without mmproj on llama.cpp #2358

Description

@hun

Feature Description

When a model contains an mmproj, it will be put in VRAM, even if i don't intend to use it. Using --no-mmproj-offload would work, but it is forbidden by lemonade. Just allowing the option would be fine for me.

Use Case / Motivation

Have more VRAM for useful things

Platform Relevance

All platforms

Additional Context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    engine::llamacppllama.cpp backend (LlamaCppServer); GPU/CPU LLM inference (Vulkan, ROCm, Metal)enhancementNew feature or requestpriority::😎warm

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions