Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Regression proving with Coq: past, present, and future #9262
In ongoing work, we are investigating regression proving techniques for Coq. Regression proving is concerned with re-checking affected proofs after a change has been made to a verification project.
Our main goal is to increase the productivity of proof engineers in large scale projects, where a short "edit-check cycle" is critical. Inspiration comes primarily from regression testing research in software engineering: we consider Coq proofs analogous to program tests. In particular, we assume engineers make generous use of CI and other automation as in software projects.
Here is an attempt at a summary of the current state of our work and ideas for future improvements. Many of these improvements require changes in Coq itself.
Our starting point is the document-oriented model and underlying proof processing architecture introduced in Coq 8.5. Specifically, the
These two features mirror the possibilities in software batch test execution, where individual tests can be run in isolation, and where test execution can be parallelized both at the test method and test class level.
iCoq: sequential regression proof selection
Regression test selection techniques for Java-like languages track dependencies among classes and methods and use this information to perform change impact analysis when a project is modified.
iCoq is our analogue of Java regression test selection tools for Coq and builds on asynchronous checking of opaque proofs. It analyzes dependencies between definitions and lemmas, and finds and checks (only) impacted proofs. It can be run either locally or in CI.
iCoq implementation highlights:
See an example of how iCoq works.
Highlights from empirical study on version histories of large projects:
Main limitations of iCoq:
piCoq: parallel regression proving
iCoq performed only sequential checking. piCoq extends iCoq to parallel checking and includes classical modes for incremental, file-level parallel checking (similar to
Highlights of empirical study on version histories of large projects (including Verdi, Flocq, and Coquelicot):
Suggestions towards future regression proving support
@gares @charguer let me know what you think and if you have questions or additions. Thus far, I feel our work has mostly been about exploring feasibility. But with help from Coq developers, we believe regression proving tools could become robust enough to be adopted by large projects, especially in their CI.
We will also try to prepare specific API suggestions that can be discussed at CoqPL.