Filtering Context size , filtering Quants, filtering Tps

[tomsun66-star]

Just found the tool, very interesting.
i miss the possibility to filter the context size as for many usecases contextsize is crucial.
i miss the possibility to filter for quants as for several Tasks it is crucial usually e.g Q3 not good enough for other Q4 is good enough and q8 complete overkill
i would like to see what second GPU is recognized not just + 1 GPU
in case of multiple installed GPUs the possibility to select GPU1 or GPU 2 or both combined for tripple or more even more permutations (probably Tickbox which GPUs used)
i would like to see the possibility to selct multiple Fits at once e.g. Perfect AND Good

So in conclusion the possibility to select CTX From to combined with Quant From to using GPU 2 AND 3 with Parameters From To (what allready works)

[AlexsJones]

Thanks for spelling out the workflow. Context size, quant, fit grade, and selected GPUs are all useful filters, but the multi-GPU selector changes the fit calculation rather than just narrowing a list.

Could you open an issue with two or three concrete filter combinations you would expect, including what should happen when only GPU 2 is selected and when GPUs 2 and 3 are combined? That gives us a testable first slice. I would start with display filters for context, quant, and fit grade, then treat explicit GPU selection and multi-GPU capacity as the follow-up because they need the calculation semantics pinned down.

[tomsun66-star]

Sorry Alex for the delay, just recognizing how deep the rabbithole is, So concrete combination 128 or 256K Context, Q4, Dual 16 GB GPU Setup combined so 32 GB in total, dense model (what i currently use e.g. for author role [large prompt and long product output] and for architectural project brainstorming in the Frontend), Small context in the discussion is not leading to success because almost 100K are eaten up to bring fresh window to current status, so no room for new work.

[AlexsJones]

No worries on the delay, and that combination is exactly the kind of concrete case that pins this down: 128K or 256K context, Q4, a dense model, two 16 GB cards pooled to 32 GB.

You can check most of it today. llmfit plan <model> --context 131072 --quant Q4_K_M prints the weights, KV cache and total VRAM for one specific dense model, so you can compare that against 32 GB directly. For the list view, llmfit --max-context 131072 caps the context used for memory estimation at 128K. At that length the KV cache for a 30B-class dense model can rival the Q4 weights, which is why so few survive.

One caveat: treating two 16 GB cards as one 32 GB pool is optimistic, since a split across GPUs carries per-card overhead. That's the part that needs real semantics before we add a GPU selector.

Could you turn this into an issue with that exact setup and one model you expected to fit? Then quant and context filters can land as the first slice, and dual-GPU pooling can follow.