LLM stacks \ 1.2 LLMs, UIs

25.0625 Lab notes (Gdrive), Git

For details

see: #499_BBB_AI_API_deployments_devplan_

A1-1 LLMs build

Code from scratch. Very low level demos to learn the core stuff about weights, models, etc.. its missing the glue logic and size, both important for real world models.

A1-1b wrap the token generator

(full LLM) (OUT OF SCOPE)

external apps cant working with a bare token generator. so have to add lots of “glue logic”, that

• controls the token generator, inputs
• reformats outputs
• enables contact with outside world (API that openAI APIs can converse with, etc)
• provides the simulated human conversational partner functionality

But this is really complex. So not something even attempted in the AI sandbox.

A1-1nv LOCAL MODEL

(Ollama, auto-download, Nvidia drivers setup)

A1-1c UI (to model local or remote)

Instead usually use either

• remote model (chatGPT, etc) or
• imported local model (from HF, running in Ollama (??) )

A1-2 LLM fork/train/fine-tune (was LLMs)

IF the model requires customization, then train and/or fine tune own model. for example: 1 fork a model on HF 2 train / fine-tum

A1-6 Model deploy

How to control? Safe if HF is not CC account? Deploy model. Model can now be hijacked? 3 push to HF 4 use the model remotely