Repository navigation
Quick Start
This page solves one benchmark instance from the command line and explains each line of the result. It assumes Installation is complete and benchmarks/ is unpacked.
export PYTHONPATH=$PWD/solver:$PWD/solver/complete_verifier PYTHONNOUSERSITE=1; cd solver/complete_verifier && ../../env/bin/python nlp_example.py --problem_name hs030 --benchmark gams_world --benchmark_root ../../benchmarks --config exp_configs/NLP_solver/f3.yaml --device cuda --use_beta False --override_timeout 180 --batch_size 1 --result_json ../../hs030.json 2>&1 | tail -n 25; ../../env/bin/python -c "import json; r = json.load(open('../../hs030.json')); print({k: r[k] for k in ('status', 'termination', 'primal_bound', 'dual_bound', 'gap_rel', 'x', 'replay')})"--problem_name and --benchmark select the instance (gams_world or minlplib_linear), --benchmark_root is the directory that holds both families, --config is the default configuration f3 (the same as the API preset gear-f0), --override_timeout is the time limit in seconds, and --result_json writes the result as JSON. Use --device cpu without a GPU; the same command with --device cpu was tested on a CPU-only machine.
The first run of an instance writes a .vnnlib.compiled cache next to its .vnnlib file in benchmarks/.
The log ends with:
[UnifiedRoot] closed after incumbent at 2.35s: P=1 D=1
Primal Solution tensor([1.]), Dual Solution 1.0
Solution gap: 0.0
Found optimal object: tensor([1.])
[best-primal] objective=1.0 x=[1.0, 0.0, 0.0]
problem_name=hs030 status=verified success=True
and the result file prints as:
{'status': 'OPTIMAL', 'termination': 'GAP_CLOSED', 'primal_bound': 1.0, 'dual_bound': 1.0, 'gap_rel': 0.0, 'x': {'X_0': 1.0, 'X_1': 0.0, 'X_2': 0.0}, 'replay': {'feas_tol': 1e-06, 'max_bound_violation': 0.0, 'max_row_violation': -1e-12, 'ok': True, 'worst_row': 'row 0'}}
| Item | Meaning |
|---|---|
P (Primal Solution, primal_bound) |
Objective value of the best feasible point found, in the sense of the model (also for maximization). For a minimization problem it is an upper bound on the optimum. |
D (Dual Solution, dual_bound) |
Certified bound on the optimum: a lower bound for minimization, an upper bound for maximization. |
[best-primal], x
|
The returned point and its objective value. Variables are named X_0, X_1, ... |
status in the last log line |
verified: the search closed the gap or proved infeasibility. unknown: no certificate within the time limit. |
success |
False exactly when that status is unknown. |
status and termination in the result file |
OPTIMAL with GAP_CLOSED, INFEASIBLE with PROVED_INFEASIBLE, or another status with the reason the search stopped (Python API). |
The last log line is the line that the batch runners and the scoring scripts read.
An instance is counted as solved when
(P - D) / max(1, |P|, |D|) <= 1e-4
gap_rel in the result file is this quantity.
Before a point is reported, GEAR re-evaluates it on the original model; the result is the replay entry. An instance is counted as solved only if the point satisfies every constraint and every variable bound of the original model to an absolute tolerance of 1e-6 (replay.feas_tol; replay.ok is True). max_row_violation and max_bound_violation are the largest violations found, and worst_row names the row. The certified bound D does not depend on this check.
The same instance family can be solved through the gear command or the Python API:
export PYTHONPATH=$PWD/solver:$PWD/solver/complete_verifier PYTHONNOUSERSITE=1; cd solver/complete_verifier && ../../env/bin/python -m gear.cli solve --legacy hs071 --family gams_world --root ../../benchmarks --device cuda --time_limit 180 --result_json ../../hs071.json 2>&1 | tail -n 6; ../../env/bin/python -c "import json; r=json.load(open('../../hs071.json')); print({k: r[k] for k in ('status','termination','primal_bound','dual_bound','gap_rel','x')}); print(r['replay'])"See Python API. logs/gear_results.tsv has one row per instance of the paper run; see Output and Logs.