Support/Outlines in LORAX Inference Server

[AdithyanI]

Hi,

Thanks for the work here.
It would be great if we can add outlines support to the Lorax repo : https://github.com/predibase/lorax
The main advantage with Lorax is that it supports S-LoRA. And it's also becoming quite popular.

Its based out of TGI and they offer nice APIs, so I don't know how difficult it is to add the support here.

I am user of that repo (and not a maintainer/developer).
Would be great if the support for outlines is available via API in Lorax in the same way that you do for other repos.

Thanks you.

Best,
Adi

[rlouf]

This would require Lorax allowing users to specify a logit processing function. I opened an issue on their repo to ask if they would consider it: predibase/lorax#176