Property-based testing, adversarial developers, and LLMs | Protocols Made Fun

[giscus[bot]]

Property-based testing, adversarial developers, and LLMs | Protocols Made Fun

Author: Igor Konnov

https://protocols-made-fun.com/pbt/2025/12/22/pbt-adversarial-llms.html

[ericsson49]

Viewing LLMs as adversarial developers is an interesting approach.
One idea that came to my mind is that one can limit the LLM's adversary power somehow. E.g. via prompts, dedicated training, "rejection sampling", etc.
An interesting attempt in that direction - AILang.
Wrt to testing, one can try to confine LLM to produce code which is amenable to static analysis, model checking or similar.