Cookbook

[razonjasky]

Hi, just a noob question. I've already downloaded the best fit model from the cookbook, I don't know to use it as my model. Thanks!

[BoRedo-64]

Im also a noob but i think u should go to serve and create it with the one u installed then u can click on the three dots in the running tab and click on register endpoint

[razonjasky]

Thanks, I'll try it in a minute. Would you say that the ones in the cookbook are better choice than local llama 3? sorry noob question again. Thanks

[CoolJohn-lab]

The honest answer is we don't really know right now. The Cookbook is tuned to find models that will fit on your hardware, not necessarily to find the best ones for chat, agent, image roles etc. I think the best thing we can do right now as early adopter users is to use the A/B testing functionality to try and settle on a good "recommended" set of models to use going forward, for each level of hardware.

So there would be a "best set" of models to use on a single 8GB card, another if you had a 16GB card, another if you had 32GB, and so on, and different recommended models depending on how much RAM you had and how slow you were prepared for responses to be I guess. Nobody knows what those "best set" of models looks like yet really, other than Pewds, and I think he's done his best to try to get the Cookbook containing every model he recommends, as far as I can tell.

[razonjasky]

That actually makes sense now that you've said it, I'll just try the compare mode to see if the recommended one or llama 3 one is better for my preference. Thanks for explaining it, he for sure did his best on this one. I'ts my first time trying it so I'm just really glad I get to try it because of him.