Skip to content

Mosek interface ignores basic solution when IPM ends in dual_infeas_cer but crossover finds optimal basis #665

@mgrabovsky

Description

@mgrabovsky

Version Checks (indicate both or one)

  • I have confirmed this bug exists on the lastest release of Linopy.

  • I have confirmed this bug exists on the current master branch of Linopy.

Issue Description

When solving an LP with Mosek through linopy, the solver can produce two distinct solutions with separate statuses: an interior-point solution (soltype.itr) and a basic solution (soltype.bas) obtained by basis identification (crossover). linopy's Mosek interface appears to read only the interior-point solution. If the IPM terminates with solsta.dual_infeas_cer but crossover then recovers a solsta.optimal basic solution, linopy reports infeasible_or_unbounded with Objective: nan and the actual optimal solution is silently discarded, even though Mosek itself has it.

I've hit this on a PyPSA model with two investment periods, solved via n.optimize(solver_name="mosek"). Mosek's solution summary at termination:

Interior-point solution summary
  Problem status  : DUAL_INFEASIBLE
  Solution status : DUAL_INFEASIBLE_CER
  Primal.  obj: -1.0000000000e+00   nrm: 2e-06    Viol.  con: 2e-06    var: 8e-09
Basic solution summary
  Problem status  : PRIMAL_AND_DUAL_FEASIBLE
  Solution status : OPTIMAL
  Primal.  obj: 6.4683156642e+10    nrm: 6e+07    Viol.  con: 1e-07    var: 2e-13
  Dual.    obj: 6.4683156642e+10    nrm: 3e+06    Viol.  con: 4e-12    var: 4e-10

linopy output:

WARNING:linopy.constants:Optimization potentially failed:
Status: warning
Termination condition: infeasible_or_unbounded
Solution: 0 primals, 0 duals
Objective: nan
Solver model: not available
Solver message: solsta.dual_infeas_cer

The basic solution is feasible and optimal according to Mosek, but linopy never queries it and returns NaN. There is no way for the user to recover the solution through linopy – n.objective, primals, and duals are all NaN/empty.

I've tried debugging this with Claude for a bit, but didn't want to get too deep into linopy internals. Here's what it has to say:

Mosek may return up to three solutions per task (interior, basic, integer), each with its own prosta/solsta. The IPM and the basis identification phase that follows it are independent and can disagree. In particular, when the IPM ends in a numerically marginal state near the optimum (large constraint matrix range, multi-period structure, etc.), it can mis-classify its own iterates as a dual-infeasibility ray (dual_infeas_cer) even though the actual problem is bounded — the certificate violation here is 2e-6, i.e. a near-certificate, not an actual one. Crossover then starts from those iterates and the dual simplex pivots to a true optimal vertex, which Mosek records as the basic solution with solsta.optimal.

A correct interface must query solutiondef/getsolsta for both soltype.bas and soltype.itr, pick the better status, and read the corresponding solution.

Apparently, the same bug was found in cvxpy in 2017 (see cvxpy/cvxpy#335, fixed in cvxpy/cvxpy#347).

Reproducible Example

I do not have a small reproducerthe bug is triggered by IPM termination in a large multi-period PyPSA network.

Expected Behavior

Linopy should recognise that Mosek has found an optimal solution in the crossover and load that instead of jumping to conclusions based on the IPM result only.

Installed Versions

Details linopy==0.6.7 Mosek==11.1.11 pypsa==1.2.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions