homework.md

Q1. Running Ollama with Docker

docker run -it \
    --rm \
    -v ollama:/root/.ollama \
    -p 11434:11434 \
    --name ollama \
    ollama/ollama

docker exec -it ollama bash

ollama -v

The ollama version is 0.1.48.

Q2. Downloading an LLM

docker exec -it ollama bash

ollama pull gemma:2b

cat /root/.ollama/models/manifests/registry.ollama.ai/library/gemma/2b

The content of the file is: {"schemaVersion":2,"mediaType":"application/vnd.docker.distribution.manifest.v2+json","config":{"mediaType":"application/vnd.docker.container.image.v1+json","digest":"sha256:887433b89a901c156f7e6944442f3c9e57f3c55d6ed52042cbb7303aea994290","size":483},"layers":[{"mediaType":"application/vnd.ollama.image.model","digest":"sha256:c1864a5eb19305c40519da12cc543519e48a0697ecd30e15d5ac228644957d12","size":1678447520},{"mediaType":"application/vnd.ollama.image.license","digest":"sha256:097a36493f718248845233af1d3fefe7a303f864fae13bc31a3a9704229378ca","size":8433},{"mediaType":"application/vnd.ollama.image.template","digest":"sha256:109037bec39c0becc8221222ae23557559bc594290945a2c4221ab4f303b8871","size":136},{"mediaType":"application/vnd.ollama.image.params","digest":"sha256:22a838ceb7fb22755a3b0ae9b4eadde629d19be1f651f73efb8c6b4e2cd0eea0","size":84}]}

Q3. Running the LLM

ollama run gemma:2b

prompt: "10 * 10"

yields:

Sure, here's the answer:

10 * 10<sup>end_of_turn</sup>

This is a mathematical expression that evaluates to 100 when executed.

Q4. Downloading the weights

docker run -it \
    --rm \
    -v /home/tim/play/llm-zoomcamp/ollama_files:/root/.ollama \
    -p 11434:11434 \
    --name ollama \
    ollama/ollama

docker exec -it ollama ollama pull gemma:2b

du -h

The size of the folder is app. 1.7G.

Q5. Adding the weights

COPY ./ollama_files/models/ /root/.ollama/models

Q6. Serving it

The code can be found in run_client.py.

There are 304 completion tokens.