Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
cff-version: 1.2.0
message: "If you use this software, please cite it as below."
type: software
title: "AVISE: Framework for Evaluating the Security of AI Systems"
authors:
- family-names: Lempinen
given-names: Mikko
- family-names: Kemppainen
given-names: Joni
- family-names: Raesalmi
given-names: Niklas
year: 2026
identifiers:
- type: other
value: "arXiv:2604.20833"
description: arXiv preprint
url: "https://arxiv.org/abs/2604.20833"
preferred-citation:
type: article
title: "AVISE: Framework for Evaluating the Security of AI Systems"
authors:
- family-names: Lempinen
given-names: Mikko
- family-names: Kemppainen
given-names: Joni
- family-names: Raesalmi
given-names: Niklas
year: 2026
url: "https://arxiv.org/abs/2604.20833"
journal: "arXiv preprint arXiv:2604.20833"
50 changes: 43 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,29 @@

A framework for identifying vulnerabilities in and evaluating the security of AI systems.

[![arXiv](https://img.shields.io/badge/arXiv-2604.20833-b31b1b.svg)](https://arxiv.org/abs/2604.20833)


#### Full Documentations: https://avise.readthedocs.io

<br>
<br>
---

## Table of Contents
- [Quickstart for Evaluating Language Models](#quickstart-for-evaluating-language-models)
- [Prerequisites](#prerequisites)
- [1. Install AVISE](#1-install-avise)
- [2. Run a Model](#2-run-a-model)
- [3. Evaluate the model with a Security Evaluation Test (SET)](#3-evaluate-the-model-with-a-security-evaluation-test-set)
- [Advanced Usage](#advanced-usage)
- [Configuring Connectors](#configuring-connectors)
- [Configuring Security Evaluation Tests (SETs)](#configuring-security-evaluation-tests-sets)
- [Using custom configuration files](#using-custom-configuration-files)
- [Required Arguments](#required-arguments)
- [Optional Arguments](#optional-arguments)
- [Citation](#citation)

## Quickstart for evaluating Language Models

## Quickstart for Evaluating Language Models

### Prerequisites

Expand All @@ -35,7 +52,7 @@ Install with
uv tool install avise
```

### 2. Run a model
### 2. Run a Model

You can use AVISE to evaluate any model accessible via an API by configuring a Connector. In this Quickstart, we will
assume using the Ollama Docker container for running a language model. If you wish to evaluate models deployed in other ways, see
Expand Down Expand Up @@ -84,7 +101,7 @@ avise --SET-list
```


## Advanced usage
## Advanced Usage

### Configuring Connectors

Expand Down Expand Up @@ -142,15 +159,15 @@ If you want to use custom configuration files for SETs and/or Connectors, you ca
avise --SET prompt_injection --SETconf AVISE/avise/configs/SET/languagemodel/single_turn/prompt_injection_mini.json --connectorconf AVISE/avise/configs/connector/languagemodel/ollama.json
```

### Required Arguments
## Required Arguments

| Argument | Description |
|----------|-------------|
| `--SET`, `-s` | Security Evaluation Test to run (e.g., `prompt_injection`, `context_test`) |
| `--connectorconf`, `-c` | Path to Connector configuration JSON (Accepts predefined connector configuration paths: `ollama_lm`, `openai_lm`, `genericrest_lm`)|


### Optional Arguments
## Optional Arguments

| Argument | Description |
|----------|-------------|
Expand All @@ -166,3 +183,22 @@ avise --SET prompt_injection --SETconf AVISE/avise/configs/SET/languagemodel/sin
| `--version`, `-V` | Print version |


## Citation

If you find AVISE useful, please cite it as below:

```bibtex
@misc{lempinen2026,
title={AVISE: Framework for Evaluating the Security of AI Systems},
author={Mikko Lempinen and Joni Kemppainen and Niklas Raesalmi},
year={2026},
eprint={2604.20833},
archivePrefix={arXiv},
primaryClass={cs.CR},
url={https://arxiv.org/abs/2604.20833},
}
```

Lempinen, M., Kemppainen, J., & Raesalmi, N. (2026). *AVISE: Framework for
Evaluating the Security of AI Systems*. arXiv preprint arXiv:2604.20833.

Loading