VeryDiff Polynomial Experiments

Experimental Evaluation for Equivalence Verification of Polynomial Neural Networks.

Repository Structure

• datasets: Contains datasets used for training the verified neural networks (if directory is missing, it will be auto-extracted from datasets.tar.gz once calling using VeryDiffPolyExperiments in Julia)
• deps: Contains code for other tools we evaluate against
• networks: Neural networks used for verification and polynomial approximation in the experiments
• results: Results of the experiments
• src: Code to run the experiments
  • generate_nns: Code to run experiments for synthesis of polynomial neural networks
  • verify_nns: Code to run experiments for verifying equivalence of standard and polynomial neural networks

Setup

Before running experiments, the Julia environment has to be set up.

First, clone the git repository and start Julia

> git clone https://github.com/phK3/VeryDiffPolyExperiments
> cd VeryDiffPolyExperiments
> julia

Within Julia, we can set up the environment via the built-in package manager.

julia> ]  # to enter the package manager
pkg> activate .  # to activate the environment in the current directory
(VeryDiffPolyExperiments) pkg> instantiate  # to download the necessary dependencies
julia>  # first type backspace to leave the package manager

Once the environment is set up, we can run the experiments via (set test_run=true if you don't want to run all experiments but just a quick test)

julia> using VeryDiffPolyExperiments
julia> generate_mnist_nets_verified_bounds(test_run=true)
julia> generate_heloc_nets_verified_bounds(test_run=true)
julia> warmup_eps_equivalence_mnist()  # to trigger precompilation
julia> verify_eps_equivalence_mnist("2025-04-08", test_run=true)
julia> warmup_eps_equivalence_heloc()
julia> verify_eps_equivalence_heloc("2025-04-11T09:43")
julia> verify_eps_equivalence_sample_mnist("2025-04-08", test_run=true)
julia> verify_eps_equivalence_sample_mnist("2025-04-08", n_sample=25)
julia> generate_har_nets_verified_bounds(test_run=true)  # for precompilation
julia> generate_har_nets_verified_bounds()
julia> verify_eps_equivalence_har("2025-04-16", test_run=true)  # for precompilation
julia> verify_eps_equivalence_har("2025-04-16")
julia> generate_mnist_nets_verified_bounds(max_iter=1, test_run=true)  # ablation for influence of remez vs. cheby
julia> generate_mnist_nets_verified_bounds(max_iter=1)
julia> verify_eps_equivalence_sample_mnist("2025-04-08", only_degrees=[5], test_run=true)
julia> verify_eps_equivalence_sample_mnist("2025-04-08", only_degrees=[200], n_sample=10)

Ensure Single-Threaded Execution

Although our Julia implementation should only use a single thread by default, calls to external libraries might use multiple threads. To ensure single-threaded execution, use

taskset -c 0 julia

to start Julia and pin it to processor $0$.

Running LIGAR

To run the LIGAR experiments, create and activate the ligar conda environment and execute the shell script via

$ conda env create -f deps/ligar_reloaded/ligar_environment.yaml --name ligar 
$ activate ligar
$ ./run_ligar.sh