Installation

I'm using Ubuntu (server) 24.04.4 LTS on a ~10 year old machine with 32GB 800MHz DDR3, Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz, SSD for the OS/models. The GPU is a NVIDIA GeForce GTX 1060 with 6GB VRAM, so I appear to be limited to LLMs of 8 billion parameters (or less).

The LLMs are running on Ollama, which can be installed by running the following command:

curl -fsSL https://ollama.com/install.sh | sh

Ollama is a wrapper for llama.cpp by Georgi Gerganov. It abstracts away the configuration required to run llama.cpp via scripts which detect likely good choices and exposes OpenAI model compliant LLMs through an API endpoint or chat interface directly on the terminal. Ollama also hosts LLMs and allows you to effortlessly pick and choose a (tagged) model that suits your needs. In my case 6GB VRAM is a bottleneck. Replicating the flow presented by mmajurski/NIST requires a model that implements "thinking" (reasoning tokens).

For me, that means nemotron-3-nano:4b.

More information can be found here.

Note: Before installing Ollama, install the NVIDIA drivers recommended for your graphics card. The rule of thumb seems to be that "if nvidia-smi returns results (rather than an error), you should be good to use your VRAM with Ollama".

In my case, I installed Ollama before installing the NVIDIA drivers, assuming that they were detected by the Ubuntu installation process, since I selected "use third-party drivers" (enabling the restricted, multiverse apt package repositories), but this proved to be incorrect and Ollama threw an error explicitly identifying which drivers to install for my graphics card (580x), which is curious because ubuntu-drivers devices recommends the 535x "non-free" (NVIDIA). I have yet to benchmark performance on the 535x vs. the 580x (if they even work), because at this point I'm only looking for a working proof of concept.

uv is a fast Python package manager, written in Rust. Installation instructions in detail are available here. It enables you to fetch dependencies (libraries) from their official distribution sources faster than when using pip alone.

1. Install uv

curl -LsSf https://astral.sh/uv/install.sh | sh

2. Create a directory for airflow at your desired location

mkdir airflow

3. Create virtual environment for airflow

uv venv --python=3.12

4. Activate the newly created venv

source .venv/bin/activate

5. Install airflow via uv

uv pip install "apache-airflow[celery]==3.2.2" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-3.2.2/constraints-3.12.txt"

6. Install openai provider for airflow

uv pip install apache-airflow-providers-openai

7. Install common ai module

uv pip install apache-airflow-providers-common-ai

8. Run airflow

airflow standalone

Note: On first run, the admin password will be echoed to terminal. After the first run, you can find it in the file simple_auth_manager_passwords.json.generated in the directory

9. Create Pydantic AI connection

Create a connection to your OpenAI model compliant LLM via the Apache Airflow UI at http://localhost:8080 (do not use "standalone" for production), using the following:

• Connection Type: Pydantic AI
• Connection ID: openai_default
• Description: [optional]
• Host: http://127.0.0.1:11434/v1
• API Key: [not required but apparently cannot be left blank]
• Extra JSON: { "extra": { "model": "openai:nemotron-3-nano:4b" } }

Note: "model" in extra JSON must be prefixed with "openai:", but you can use whatever model is desired, as long as it supports "thinking"

10. Create your directed acyclic graph (DAG)

Note: The default path for directed acyclic graphs (DAG) is "./dags/".