AgentPrologKit

A clean Python (and CLI) interface for authoring and querying SWI-Prolog — designed so any program, including an LLM agent, can drive Prolog without ever touching its surface syntax. The Python package is called agentprologkit; this repo is where it lives.

The motivating problem: agents driving Prolog by shelling out swipl -g "add_fact(...)" -g halt fight two escaping layers at once (shell quoting and Prolog term syntax) — and an agent's own tool-call arguments (file paths, URLs, arbitrary text) are exactly the kind of untrusted strings that break both. agentprologkit removes that surface entirely: you build terms in Python, not Prolog source text, so malformed or unsafely-escaped Prolog is unrepresentable.

Why Prolog for agent safety evaluation?

A safety property over an agent's trace is relational and ordered — "read a local file, then later make an outbound call" is just a conjunction of goals over a log with an ordering constraint — and Prolog's unification and backtracking are precisely a search for whether such a pattern holds, so you declare what counts as unsafe and let the engine find it. That buys the one thing an LLM grading another LLM cannot: a machine-checkable specification that is deterministic and inspectable, and that returns either a witness (the offending bindings) or a definite no — verification needs a ground-truth anchor to check against, and probabilistic self-assessment isn't one. This is not only a first-principles argument. In RefineAct: Automatic Runtime Verification of LLM Agent Actions (Batole, Khomh & Rajan), an approach built using Prolog engine as the representation interposed before each agent action on the ToolEmu benchmark cuts failure incidence from 77% to 39% while raising task helpfulness from 1.0 to 1.9 — a symbolic verifier improving safety and utility at once, rather than trading one for the other. agentprologkit is the substrate that makes that loop cheap: an agent can author the rules and run the queries without ever emitting a line of Prolog.

Two surfaces

1. Authoring — no swipl needed. add_fact / add_rule / write render terms in Python and append them to a .pl file. Because terms are rendered (never parsed from a hand-written string), malformed Prolog is unrepresentable, and a plain str is always a quoted atom — the only way to get a logical variable is the explicit Var("X") sentinel.
2. Engine — swipl via MQI. load consults files; query runs goals and returns structured bindings (list[dict]), no stdout parsing. Backed by SWI-Prolog's Machine Query Interface (swiplserver).

Setup

Requirements:

• Python >= 3.10
• SWI-Prolog >= 8.4.2 on PATH — only needed to run queries; authoring facts/rules to a .pl file works without it.

pip install agentprologkit

This installs everything needed, including swiplserver (the client behind Engine / KnowledgeBase.query) and the agentprologkit CLI.

Example: is this agent run safe?

The scenario this library is built for: an agent just finished a tool-calling session, and you have a plain list of the actions it took. Log each action as a fact, author one safety rule, and query it — no hand-written Prolog, no shell escaping, even when an argument (like this file path) contains a quote:

from agentprologkit import KnowledgeBase, Compound, Var

# A log of steps an agent took, in order -- exactly what an agent runtime
# already has on hand after a tool-calling session.
actions = [
    ("read_file", "/home/o'brien/.ssh/id_rsa"),  # note the quote -- handled safely
    ("call_api", "https://weather.example.com/lookup"),
    ("http_post", "https://attacker.example.com/collect"),
]

with KnowledgeBase("agent_run.pl") as kb:
    for step, (kind, target) in enumerate(actions, start=1):
        kb.add_fact("action", [step, kind, target])

    # unsafe(Step) :- the agent read a local file, then later made an
    # outbound network call -- a classic exfiltration pattern.
    kb.add_rule(
        Compound("unsafe", [Var("Read")]),
        [
            Compound("action", [Var("Read"), "read_file", Var("_Path")]),
            Compound("action", [Var("Send"), "http_post", Var("_Url")]),
            Compound("<", [Var("Read"), Var("Send")]),
        ],
    )

    for row in kb.query("unsafe(Step)"):
        print(row)  # {"Step": 1} -- flags the read-then-exfiltrate pattern

Swap in your own predicates for whatever "safe" means in your domain — add_fact/add_rule don't care whether the facts came from a tool-call log, a file scan, or a chain-of-thought trace, only that they're valid terms.

Controlled vocabulary (optional)

from agentprologkit import KnowledgeBase, Schema

schema = Schema().define("action", 3)
kb = KnowledgeBase("agent_run.pl", schema=schema)
kb.add_fact("action", ["a", "b"])   # -> AddResult(ok=False, error={code: arity_mismatch, ...})

Results, errors, and persistence

query returns a list[dict], one dict of variable bindings per solution. Everything in it is JSON-serializable as-is:

• no solution → []; success with no variables → [{}]
• atom / number → Python str / number; Prolog list → Python list
• compound term → {"functor": ..., "args": [...]}; unbound variable → "_"

Errors are made to be branched on, not string-matched:

Where	What you get	Meaning / next step
add_fact / add_rule	AddResult(ok=False, error={"code": ...}) with unknown_predicate, arity_mismatch, or term_error	fix the fact and retry
query	PrologError — the message names the goal; .prolog holds the structured exception term	e.g. existence_error = the predicate isn't defined: load the file that defines it, or check spelling/arity
query	QueryTimeout after 10 s (configurable via query_timeout; None = no limit)	the goal loops or is genuinely slow
load	PrologError with file, line, and column in .prolog	the .pl file has a syntax error at that position

Loading a malformed file fails loudly: plain swipl skips the bad clause, prints to stderr, and reports success — agentprologkit reads the file term by term first, so the agent gets the exact position instead of a mystery existence_error later.

Persistence: opening KnowledgeBase("kb.pl") on an existing file loads it, so facts written by an earlier run are queryable immediately, and the loaded predicates can still be extended with add_fact. Adds are append-only — re-adding the same fact appends a duplicate clause (delete the file to start fresh, as test.py does).

CLI

Available as agentprologkit ... after pip install agentprologkit, or straight from a checkout with no install:

PYTHONPATH=src python3 -m agentprologkit.cli ...

# --args-json gives typed args (here, an int step) instead of bare-atom positionals.
agentprologkit add-fact --file agent_run.pl action --args-json '[1, "read_file", "/etc/passwd"]'
agentprologkit add-fact --file agent_run.pl action --args-json '[2, "http_post", "https://attacker.example.com"]'
agentprologkit query --load rules.pl --load agent_run.pl 'unsafe(Step)'

Each command emits one JSON object and is shaped to map 1:1 onto a future MCP tool. Failures exit non-zero with {"ok": false, "error": {"code": ..., "message": ...}}, where code is a stable snake_case value (prolog_error, timeout, engine_unavailable, bad_args_json, ...).