Skip to content

Getting started

Jump to bottom
Osma Suominen edited this page Aug 17, 2026 · 1 revision

Introduction

This is a very brief overview of how to get started using BIBRA. To be able to use it locally, you will need some hardware to run one of the LLM backends - currently either GreyLitLM or NuExtract3. In practice this means a GPU with a minimum of around 8GB VRAM. It is possible to run these models without a GPU, but CPU-only processing is very slow.

Install BIBRA itself

BIBRA is a Python application and installing it can be as simple as pip install bibra. See README.md for some details.

Alternatively, you can use the Docker image of BIBRA (natlibfi/bibra) that we provide through Quay.io.

Set up LLM inference

For running the LLM-based backends, BIBRA requires an LLM inference server that provides an OpenAI-style API. In practice, we use llama-server from llama.cpp to run the LLMs, as it is cross-platform, performs well on limited hardware (including CPU-only) and is relatively easy to set up and use. See llama.app for details on its installation and use.

Installation options include:

Running the server

Once you have installed llama.cpp, you can run the llama-server command with arguments that specify which model (LLM) to run and settings such as context length, host and port to listen on. There are a lot of llama-server parameters that can be used to tweak performance, memory usage and other aspects that we will not cover here. To run the OpenAI style API needed by BIBRA, you may need to set the host and port options, e.g. --host 0.0.0.0 --port 8123. By default, the API will be available on http://localhost:8080/v1/

The below commands will start a single llama-server process in the foreground.

GreyLitLM

GreyLitLM is a version of Gemma 3 fine-tuned on publications from the FinGreyLit data set. It is text-only, intended to be given text extracted from PDF documents. To run the model, you can use a command like this:

llama-server -hf NatLibFi/gemma-3-4b-it-GreyLitLM-GGUF --ctx-size 8192

NuExtract

NuExtract3 is a vision-language model for document understanding developed by NuMind; it is a fine-tuned version of Qwen3.5-4B. It is used in BIBRA to extract information from PDF pages that are first converted to bitmap images. To run the model, you can use a command like this:

llama-server -hf numind/NuExtract3-GGUF:Q8_0 --ctx-size 32768

Configure your projects and environment

Once you have the LLM service up and running, you can configure BIBRA to use it. Configuration of BIBRA happens via two mechanisms: environment variables and the project configuration file (if you are familiar with Annif configuration, this is very similar).

To specify environment variables, create a file called .env in the BIBRA installation directory. There is an example file .env.example that you can use as a starting point. If you are using Docker, set the environment variables in the docker run command with either -e (single value) or --env-file (all variables in an .env style file). The most important settings are the LLM_ENDPOINT (point to your running llama-server) and the GREYLITLM_MODEL and NUEXTRACT_MODEL settings, which should be set to e.g. greylitlm and nuextract3 (a llama-server process with a single model does not check the model ID or the API key).

To configure projects, copy the example file .projects.toml.example into projects.toml as a starting point. The example file uses ${ENV_VAR_NAME} syntax to interpolate environment variable values, so in practice, you can get started without modifying it. The Docker image uses the example file as projects.toml as well.

Try it

After setting up BIBRA, you can test the configuration with the command:

bibra list-projects

To extract metadata from a PDF, try a command like

bibra extract greylitlm path/to/mydoc.pdf

To run the BIBRA user interface, use the command

uvicorn bibra.main:app

This will open up the BIBRA web UI on http://localhost:8000 . You can use e.g. --port 12345 to set another port.

Clone this wiki locally