Repository navigation
Home
A local, high-throughput LLM-based sentiment analysis tool for structured text data (e.g. newspaper articles).
PressScan is a command-line Python tool for high-volume sentiment classification of .csv or .xlsx text datasets using a local large language model (LLM). It was created to support academic and exploratory research, especially in domains where traditional sentiment analysis methods fail to capture nuance, such as journalism, political discourse, or reporting on migration.
Before you begin, ensure you have the following installed:
- Ollama - For running LLM models locally
- Python 3.9 or above - Required to run the sentiment analyser scripts
- Ollama manages local LLM serving (e.g., Mistral, LLaMA 3, Deepseek, etc.)
- Python 3.9+ ensures compatibility with libraries like
pandas
ollama serveollama pull mistralTo use a different model:
ollama pull llama3Then pass it later using --model llama3
git clone https://github.com/003miles/PressScan.git
cd PressScanpip install -r requirements.txtRecommended to use a virtual environment (e.g.,
python -m venv .venv)
python cli_sentiment.py --file <CSV_FILE_PATH> [OPTIONS]Example:
python cli_sentiment.py --file dataset.csv --model mistral --columns title,quotes --choices positive,negative,neutral,unrelated --sample 1000 --workers 8 --debug| Option | Description |
|---|---|
--file |
(Required) Path to the input CSV file |
--model |
Model name to use (default: mistral) |
--columns |
Comma-separated list of columns to combine for text input |
--prompt |
Custom prompt text file (default: prompt.txt) |
--choices |
Comma-separated list of classification labels |
--sample |
Sample size (default: 1000 rows) |
--workers |
Max workers for CPU threading (default: 4) |
--debug |
Output full LLM reasoning text along with answers |
--full |
Ignore sampling and run on the entire dataset |
--dryrun |
Process without saving the output CSV |
| You can see help any time: |
python cli_sentiment.py --helpThe final annotated CSV file will be saved in the temp_results/ folder.
Example:
temp_results/annotated_dataset.csv_899f4b62-0383-4f87-9d9d-e86b2249186c.csv| title | quotes | Sentiment |
|---|---|---|
| Muslim girl denied entry at McDonald's | Security guard told her to remove hijab | Neutral |
| Migrants accused of abusing welfare | Minister claims abuse is "out of control" | Negative |
| Migrants praised for supporting NHS | Article celebrates healthcare workers | Positive |
| Labour donor criticises healthcare | No mention of migrants | Unrelated |
You provide only the classification logic in prompt.txt.
The script dynamically constructs the full prompt like this:
Text: {row_text}
Instructions: {user_prompt}
Classification choices: {choices}
YOU MUST RESPOND FOLLOWING THE FORMAT BELOW
Reasoning:
Answer:
Instructing the LLM to first explain its reasoning greatly improves reliability. Instead of immediately guessing a label, the model "thinks aloud" before answering. This technique is based on chain-of-thought prompting (Wei et al., 2022), which boosts model accuracy on complex tasks.
Enabling a structured "Reasoning:" field forces more deliberate, contextual classifications. It also allows manual review, aiding prompt refinement (detailed in the next section)
Prompt wording strongly influences results. To refine your setup:
- Start with small samples (
--sample 100) and inspect outputs - Adjust your
prompt.txtfor clarity and label definitions - Prefer explicit instructions over vague sentiment terms Debugging with raw model output (see Debug Mode) can highlight where confusion arises
Enable debug mode with --debug:
- Outputs the full text of each LLM response (reasoning + answer)
- Useful to diagnose misclassifications or improve prompts
- Included in addition to extracted sentiment column Example CLI:
python cli_sentiment.py --file articles.csv --debugTypical debug output excerpt:
Reasoning: The article discusses hate crimes faced by Muslim students after a terror attack. It portrays Muslims as victims of discrimination.
Answer: Neutral
The tool runs LLMs via Ollama, which uses llama.cpp and GGUF model files.
- Reasoning performance is strong for its size (7B parameters)
- Available in 4-bit quantised versions for efficient local inference
| Component | Recommended |
|---|---|
| CPU | 8-16 cores |
| GPU | 12-24 GB VRAM (optional) |
| RAM | 16-32 GB |
Ollama automatically manages GPU or CPU usage.
If your local machine lacks the GPU power or memory needed for large-scale sentiment analysis, RunPod.io provides an affordable and flexible alternative. This section walks through a full example setup based on a tested configuration.
ssh-keygen -t ed25519 -C "your_email@example.com"
cat ~/.ssh/id_ed25519.pub- Copy the public key and add it to your RunPod account under Settings > SSH Keys
- Go to RunPod.io
- Select Deploy > Secure Cloud GPU Pod
- Choose a GPU (e.g., RTX 5090, 32 GB VRAM)
- Select the "RunPod PyTorch 2.1" template (includes Python preinstalled)
- Under Overrides:
- Choose 200 GB Container Disk (skip volume storage)
- Enable SSH Terminal Access
- Confirm and Deploy
ssh root@<your_runpod_ip> -p <runpod_port> -i ~/.ssh/id_ed25519The Connect button, under your deployed pod, should give you an automatically generated SSH connection command
I recommend the FileZilla client to access the container disk where generated output will be stored. A connection can be made using the same IP, Port and SSH key used to connect. A guide on setting this up can be found here
You should first install lshw so that Ollama can access the GPU, as per RunPod's official documentation. I also recommend tmux to efficiently manage your workflow with panes or windows, all within a single terminal instance.
Run the following commands on your pod:
apt update
apt install lshw tmux
tmux newOnce tmux is running, use tmux new-window to create your windows. tmux set mouse enables you to select windows using the mouse.
curl -fsSL https://ollama.com/install.sh | sh
ollama serveAnd in a new window:
ollama pull mistralgit clone https://github.com/003miles/PressScan.git
cd PressScanpython3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txtUsing either an SFTP client (like FileZilla) or scp, copy your datasets to the PressScan directory. You can use ls in the remote terminal to verify the files were transferred.
python cli_sentiment.py --file dataset.csv --columns title,quotes \
--choices positive,negative,neutral,unrelated \
--model mistral --sample 1000 --workers 16 --debug- Output quality depends heavily on clear prompts
- LLMs occasionally misinterpret sarcasm or irony
PressScan uses large language models (LLMs) to infer sentiment in structured text (e.g. news articles) without requiring prior domain-specific training. This approach is informed by recent research in instruction-based prompting, zero-shot classification and chain-of-thought reasoning.
Traditional sentiment analysis methods, such as lexicon-based systems, logistic regression, support vector machines or pre-trained classifiers, rely heavily on domain-specific labeled data, which is often incredibly difficult to obtain. While effective in constrained contexts (e.g. product reviews), these methods struggle in domains where sentiment is nuanced, implicit, or highly contextual, such as:
- Political reporting and ideological framing
- Public discourse on polarising social issues
Large language models, by contrast:
- Are pre-trained on diverse web-scale corpora, capturing a wide range of human language patterns and discursive strategies
- Can generalise to novel sentiment tasks via carefully constructed prompts (Brown et al., 2020)
- Show emerging capabilities for reasoned judgement when structured to "think aloud" (Wei et al., 2022)
This makes LLMs particularly suitable for analysing sentiment as it is framed by the author or source text, rather than based on superficial lexical cues.
Importantly, the LLM is not asked for its own opinion, but rather to infer the framing or sentiment implied by the author
To encourage more consistent and interpretable outputs, PressScan uses a fixed template that includes:
- A user-defined instruction (
prompt.txt) - A list of allowable classification labels
- A required format for LLMs to follow"
Reasoning: Answer:
This structure is based on chain-of-thought prompting (Wei et al., 2022), which has been shown to:
- Improve model performance on reasoning and classification tasks
- Reduce ambiguity by forcing the model to justify its output
- Produce outputs that are easier for humans to inspect, critique, or override
This reasoning-first format is enforced automatically. Users only need to supply their instruction and label choices.
Several features are designed with research reproducibility in mind:
- The
--sampleflag uses a fixed random seed for consistent subsets -
--debugexposes full model responses (including rationale), allowing users to audit, refine prompts, or conduct secondary analysis - The structured output format enables downstream review or post-hoc evaluation This makes PressScan especially useful for qualitative content analysis, exploratory annotation, or as a precursor to manual coding.
LLMs are trained on human data and may reproduce societal biases, especially when analysing sensitive text. Classifications should not be interpreted as objective truths; they reflect inferred sentiment from the text's framing, not from the LLM itself or the tool author. Researchers using PressScan should remain critical of:
- Biases introduced by pre-training data (e.g. overrepresentation of Western media sources)
- Situations where subjective framing, sarcasm, or ambiguity leads to unstable outputs
- The tendency of LLMs to over-explain or over-generalise
Where appropriate, I encourage triangulating results with other qualitative or statistical methods.
- Framing analysis in media studies (e.g. portrayal of migrants, ethnic minorities)
- Rapid prototyping of annotation schemes before manual coding
- Comparative studies: LLM vs human judgement
- Generating candidate labels for weak supervision
- Krugmann, J. & Hartmann, J. (2024). Sentiment Analysis in the Age of Generative AI. Customer Needs and Solutions. https://doi.org/10.1007/s40547-024-00143-4
- Zhang, W. et al. (2023). Sentiment Analysis in the Era of Large Language Models: A Reality Check. arXiv:2305.15005
- Nadi, F. et al. (2024). Sentiment Analysis Using Large Language Models: A Case Study of GPT-3.5. In DaSET 2023. https://doi.org/10.1007/978-981-97-0293-0_12
- Wang, Z. et al. (2024). Is ChatGPT a Good Sentiment Analyzer? A Preliminary Study. arXiv:2304.04339
- Zou, H. & Wang, Y. (2025). LLM-Augmented Domain Adaptation for Sentiment Analysis. Neurocomputing, 625.
- Bojić, L. et al. (2025). Comparing LLMs and Human Annotators for Complex Sentiment Tasks. Scientific Reports, 15(1).
- Wei, J. et al. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv:2201.11903
- https://ollama.ai — Local LLM model runner
- https://github.com/ggerganov/llama.cpp — llama.cpp backend