Error when testing the robustness of networks with small non-zero weights

[vtjeng]

When testing the robustness of networks with small non-zero weights, MIPVerify can return an error with NaN values (e.g., #32). How should I handle this?

[vtjeng]

It is a known issue that having input weights which are small (relative to the largest weights) but non-zero leads to issues in the numerical methods used by the underlying solver (Gurobi, CPLEX or similar).

(In my experience, having sparse networks with weights exactly zero will actually help with solve times --- see the Appendix H to the paper linked in the README for some numerical details).

• If you are able to retrain the network with a regularization term in the loss to encourage sparsity (a small L1 penalty on weights forces weights to be zero in my experience) that will help with verification.
• Otherwise, try rounding those small values to zero