Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CPSAT 9.1: C++ wrong optimal solution and wrong infeasibility #2896

Closed
Phibedy opened this issue Nov 13, 2021 · 14 comments
Closed

CPSAT 9.1: C++ wrong optimal solution and wrong infeasibility #2896

Phibedy opened this issue Nov 13, 2021 · 14 comments
Assignees
Labels
Bug Solver: CP-SAT Solver Relates to the CP-SAT solver
Milestone

Comments

@Phibedy
Copy link

Phibedy commented Nov 13, 2021

What version of OR-Tools and what language are you using?
Version: v9.1.
Language: C++

With 9.0 it works.

Which solver are you using (e.g. CP-SAT, Routing Solver, GLOP, BOP, Gurobi)
CP-SAT

What operating system (Linux, Windows, ...) and version?
Ubuntu 20.04

What did you do?
I encountered different problems:

  • If I disable the pre-solver I can solve the problem sometimes
  • If I enable the pre-solver my problem is most of the time infeasible if I set the cost-function manually to the min-Value.

What the code does:
Solve two Objectives. After the first objective is minimized a constraint is added which forces the objective to be <= the objective value. Then it optimizes the next objective.

The fun part is "most of the time":
My console log:

$ ./bin/main --input ./test_data.json --output ./test_result.json
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20211113 17:17:42.874562 99290 solve.cpp:107] Start parsing
I20211113 17:17:42.874632 99290 solve.cpp:110] End parsing
I20211113 17:17:42.874637 99290 solve.cpp:112] Start building model
I20211113 17:17:42.875142 99290 solve.cpp:116] End building model
I20211113 17:17:42.875150 99290 solve.cpp:118] Start creating optimization steps
I20211113 17:17:42.875167 99290 solve.cpp:142] End creating optimization steps
I20211113 17:17:42.875172 99290 solve.cpp:144] Start solving
I20211113 17:17:42.875176 99290 solve.cpp:57] solving Step -100
I20211113 17:17:42.875222 99290 solve.cpp:40] Start solve threads:8 timeout: 3600
I20211113 17:17:42.883072 99298 solve.cpp:36] Solution 0 objective: 9
I20211113 17:17:42.883745 99294 solve.cpp:36] Solution 1 objective: 5
I20211113 17:17:42.883940 99294 solve.cpp:36] Solution 2 objective: 4
I20211113 17:17:42.887919 99290 solve.cpp:42] CpSolverResponse summary:
status: OPTIMAL
objective: 4
best_bound: 4
booleans: 361
conflicts: 9
branches: 484
propagations: 4536
integer_propagations: 2522
restarts: 424
lp_iterations: 0
walltime: 0.0125342
usertime: 0.0125343
deterministic_time: 0.000137172
primal_integral: 0.00533015
I20211113 17:17:42.888316 99290 solve.cpp:43] END solve
I20211113 17:17:42.888320 99290 solve.cpp:45] Restricting cost to 4
I20211113 17:17:42.888331 99290 solve.cpp:57] solving Step 10
I20211113 17:17:42.888339 99290 solve.cpp:40] Start solve threads:8 timeout: 3600
I20211113 17:17:42.893689 99306 solve.cpp:36] Solution 0 objective: -39
I20211113 17:17:42.893958 99306 solve.cpp:36] Solution 1 objective: -40
I20211113 17:17:42.900856 99290 solve.cpp:42] CpSolverResponse summary:
status: OPTIMAL
objective: -40
best_bound: -40
booleans: 361
conflicts: 8
branches: 422
propagations: 4772
integer_propagations: 4786
restarts: 422
lp_iterations: 116
walltime: 0.0123833
usertime: 0.0123834
deterministic_time: 0.003253
primal_integral: 0.00141012
I20211113 17:17:42.900872 99290 solve.cpp:43] END solve
I20211113 17:17:42.900876 99290 solve.cpp:45] Restricting cost to -40
I20211113 17:17:42.900943 99290 solve.cpp:146] End solving

second run:

$ ./bin/main --input ./test_data.json --output ./test_result.json
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20211113 17:17:43.627068 99313 solve.cpp:107] Start parsing
I20211113 17:17:43.627115 99313 solve.cpp:110] End parsing
I20211113 17:17:43.627116 99313 solve.cpp:112] Start building model
I20211113 17:17:43.627368 99313 solve.cpp:116] End building model
I20211113 17:17:43.627372 99313 solve.cpp:118] Start creating optimization steps
I20211113 17:17:43.627378 99313 solve.cpp:142] End creating optimization steps
I20211113 17:17:43.627380 99313 solve.cpp:144] Start solving
I20211113 17:17:43.627382 99313 solve.cpp:57] solving Step -100
I20211113 17:17:43.627410 99313 solve.cpp:40] Start solve threads:8 timeout: 3600
I20211113 17:17:43.635049 99321 solve.cpp:36] Solution 0 objective: 9
I20211113 17:17:43.635799 99320 solve.cpp:36] Solution 1 objective: 0
I20211113 17:17:43.640210 99313 solve.cpp:42] CpSolverResponse summary:
status: OPTIMAL
objective: 0
best_bound: 0
booleans: 361
conflicts: 4
branches: 459
propagations: 4410
integer_propagations: 2389
restarts: 431
lp_iterations: 0
walltime: 0.0126358
usertime: 0.012636
deterministic_time: 0.000107226
primal_integral: 0.00403877
I20211113 17:17:43.640709 99313 solve.cpp:43] END solve
I20211113 17:17:43.640714 99313 solve.cpp:45] Restricting cost to 0
I20211113 17:17:43.640803 99313 solve.cpp:57] solving Step 10
I20211113 17:17:43.640815 99313 solve.cpp:40] Start solve threads:8 timeout: 3600
I20211113 17:17:43.649726 99313 solve.cpp:42] CpSolverResponse summary:
status: INFEASIBLE
objective: NA
best_bound: NA
booleans: 0
conflicts: 0
branches: 0
propagations: 0
integer_propagations: 0
restarts: 0
lp_iterations: 0
walltime: 0.00878603
usertime: 0.00878613
deterministic_time: 2.4821e-05
primal_integral: 0
I20211113 17:17:43.649740 99313 solve.cpp:43] END solve

So there are two problems here:

  1. In the first run it said, that the optimal value is 4, which is wrong, as it is 0.
  2. In the second run it said that the optimal value is 0, which is true, but now the problem is infeasible.

I am 100% sure that I did not change the binary in between the runs or anything else. I just run the command twice.
Just to convince myself the timestamps:

  • first run: I20211113 17:17:42.900943 99290 solve.cpp:146]
  • second run: I20211113 17:17:43.627068 99313 solve.cpp:107]

So yes, I am sure that I did nothing stupid in the 0.7 seconds in between the two logging messages.

If I just force the first objective to be 0 the output is:

$ ./bin/main --input ./test_data.json --output ./test_result.json
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20211113 17:40:47.870162 111338 solve.cpp:109] Start parsing
I20211113 17:40:47.870227 111338 solve.cpp:112] End parsing
I20211113 17:40:47.870232 111338 solve.cpp:114] Start building model
I20211113 17:40:47.870723 111338 solve.cpp:118] End building model
I20211113 17:40:47.870730 111338 solve.cpp:120] Start creating optimization steps
I20211113 17:40:47.870746 111338 solve.cpp:144] End creating optimization steps
I20211113 17:40:47.870751 111338 solve.cpp:146] Start solving
I20211113 17:40:47.870759 111338 solve.cpp:59] solving Step -100
I20211113 17:40:47.870806 111338 solve.cpp:42] Start solve threads:8 timeout: 3600
I20211113 17:40:47.882195 111338 solve.cpp:44] CpSolverResponse summary:
status: INFEASIBLE
objective: NA
best_bound: NA
booleans: 0
conflicts: 0
branches: 0
propagations: 0
integer_propagations: 0
restarts: 0
lp_iterations: 0
walltime: 0.0111955
usertime: 0.0111957
deterministic_time: 1.7653e-05
primal_integral: 0
I20211113 17:40:47.882803 111338 solve.cpp:45] END solve

and sometimes it works:

$ ./bin/main --input ./test_data.json --output ./test_result.json
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20211113 17:42:29.693964 111431 solve.cpp:109] Start parsing
I20211113 17:42:29.694022 111431 solve.cpp:112] End parsing
I20211113 17:42:29.694026 111431 solve.cpp:114] Start building model
I20211113 17:42:29.694404 111431 solve.cpp:118] End building model
I20211113 17:42:29.694411 111431 solve.cpp:120] Start creating optimization steps
I20211113 17:42:29.694420 111431 solve.cpp:144] End creating optimization steps
I20211113 17:42:29.694424 111431 solve.cpp:146] Start solving
I20211113 17:42:29.694429 111431 solve.cpp:59] solving Step -100
I20211113 17:42:29.694473 111431 solve.cpp:42] Start solve threads:8 timeout: 3600
I20211113 17:42:29.702590 111433 solve.cpp:38] Solution 0 objective: 0
I20211113 17:42:29.706969 111431 solve.cpp:44] CpSolverResponse summary:
status: OPTIMAL
objective: 0
best_bound: 0
booleans: 361
conflicts: 9
branches: 338
propagations: 8801
integer_propagations: 5320
restarts: 309
lp_iterations: 0
walltime: 0.0123036
usertime: 0.0123038
deterministic_time: 4.0493e-05
primal_integral: 0.00179632

I also tried the master branch, but there it does nothing. It just idles using one thread. No console message after SolveCpModel

CMAKE:

  set (BUILD_SAMPLES OFF)
  set (BUILD_EXAMPLES OFF)
  FetchContent_Declare(
    ortools
    GIT_REPOSITORY https://github.com/google/or-tools.git
    GIT_TAG        v9.1
  )

  set(BUILD_DEPS ON)
  FetchContent_MakeAvailable(ortools)
  set(ORTOOLS_LIBS "ortools::ortools")
@lperron
Copy link
Collaborator

lperron commented Nov 14, 2021

I am sorry but I cannot do anything from this report.

@Phibedy
Copy link
Author

Phibedy commented Nov 14, 2021

@lperron I could not come up with a small code example, so I would just send you the proto if you don't mind.

@lperron
Copy link
Collaborator

lperron commented Nov 14, 2021 via email

@Phibedy
Copy link
Author

Phibedy commented Nov 14, 2021

Yes please

I've sent you the models + optimization logs. If you need something else just let me know.
Thank you for your support :)

@lperron
Copy link
Collaborator

lperron commented Nov 14, 2021

I ran all protos multiple times on master (1 or multiple threads, always with presolve).
It always returns 0.

@Phibedy
Copy link
Author

Phibedy commented Nov 14, 2021

It currently fails on my computer using FetchContent_Declare in CMAKE and in the CI it fails using https://github.com/google/or-tools/releases/download/v9.1/or-tools_amd64_ubuntu-20.04_v9.1.9490.tar.gz with the ubuntu:20.04 as base image.

But I could set up a Dockerfile to solve it.
Can you share the code how you executed the model proto directly?

@lperron
Copy link
Collaborator

lperron commented Nov 14, 2021 via email

@Phibedy
Copy link
Author

Phibedy commented Nov 14, 2021

I just ported the code back up 9.0 (used some functions which were added in 9.1).
If I run it in 9.0 it works. If I run it in 9.1 it fails. The diff between the two git branches is only the changed ortools-version.

$ git diff
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 4a25195..01af829 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -25,7 +25,7 @@ else()
   FetchContent_Declare(
     ortools
     GIT_REPOSITORY https://github.com/google/or-tools.git
-    GIT_TAG        v9.0
+    GIT_TAG        v9.1
   )

Tomorrow I will build the examples and try the sat_runner.

@lperron
Copy link
Collaborator

lperron commented Nov 15, 2021

you shoult try the master branch.

@Mizux Mizux added Bug Solver: CP-SAT Solver Relates to the CP-SAT solver labels Nov 15, 2021
@Mizux Mizux added this to the v9.2 milestone Nov 15, 2021
@Phibedy
Copy link
Author

Phibedy commented Nov 15, 2021

Yay today I learned how to store/load the proto-stuff.

Code:

#include "ortools/sat/cp_model.h"
#include "ortools/base/file.h"
#include "ortools/util/file_util.h"

struct OptimizationSettings
{
  int timeoutInSeconds = 3600;
  int numberOfThreads = 8;
  bool enablePreSolve = false;
};

namespace cpsat = operations_research::sat;

cpsat::CpSolverResponse solveProtoFile(std::string fileName, const OptimizationSettings &optimizationSettings)
{
  using namespace operations_research;
  using namespace operations_research::sat;
  //google::InitGoogleLogging(argv[0]);
  // Read the problem.
  CpModelProto cpModel = ReadFileToProtoOrDie<CpModelProto>(fileName);
  Model solverModel;
  SatParameters parameters;
  //parameters.set_cp_model_presolve(false);
  parameters.set_cp_model_presolve(optimizationSettings.enablePreSolve);
  parameters.set_max_time_in_seconds(optimizationSettings.timeoutInSeconds);
  parameters.set_num_search_workers(optimizationSettings.numberOfThreads);
  solverModel.Add(NewSatParameters(parameters));
  int numSolutions = 0;
  solverModel.Add(NewFeasibleSolutionObserver([&](const CpSolverResponse &r)
                                              {
                                                LOG(ERROR) << "Solution " << numSolutions << " objective: " << r.objective_value();
                                                numSolutions++;
                                              }));

  LOG(ERROR)
      << "Start solve threads:";
  const CpSolverResponse response = SolveCpModel(cpModel, &solverModel);
  LOG(ERROR) << CpSolverResponseStats(response);
  LOG(INFO) << "END solve";
  return response;
}

int main(int argc, char **argv)
{
  //google::InitGoogleLogging(argv[0]);
  OptimizationSettings withPreSolve = {.timeoutInSeconds = 3600, .numberOfThreads = 8, .enablePreSolve = true};
  OptimizationSettings withoutPreSolve = {.timeoutInSeconds = 3600, .numberOfThreads = 8, .enablePreSolve = false};
  solveProtoFile("./model_fail.proto", withoutPreSolve);
  solveProtoFile("./infeasible_model.proto", withoutPreSolve);
  LOG(ERROR) << "--------------------------------------";
  solveProtoFile("./model_fail.proto", withPreSolve);
  solveProtoFile("./infeasible_model.proto", withPreSolve);
  return EXIT_SUCCESS;
}

So in the master branch,it only works with pre-solve enabled. I will send you a small repo with the test protos + CMAKE stuff etc.

@Phibedy
Copy link
Author

Phibedy commented Nov 15, 2021

I just sent you the email.

For completeness the
LOG:

$ ./main 
WARNING: Logging before InitGoogleLogging() is written to STDERR
E1115 16:07:22.713017 185101 main.cpp:35] Start solve threads:
E1115 16:07:22.725023 185101 main.cpp:38] CpSolverResponse summary:
status: INFEASIBLE
objective: NA
best_bound: NA
booleans: 0
conflicts: 0
branches: 0
propagations: 0
integer_propagations: 0
restarts: 0
lp_iterations: 0
walltime: 0.0117276
usertime: 0.0117278
deterministic_time: 1.83996e-05
gap_integral: 0
I1115 16:07:22.725047 185101 main.cpp:39] END solve
E1115 16:07:22.728029 185101 main.cpp:35] Start solve threads:
E1115 16:07:22.738307 185101 main.cpp:38] CpSolverResponse summary:
status: INFEASIBLE
objective: NA
best_bound: NA
booleans: 0
conflicts: 0
branches: 0
propagations: 0
integer_propagations: 0
restarts: 0
lp_iterations: 0
walltime: 0.0100965
usertime: 0.0100966
deterministic_time: 0
gap_integral: 0
I1115 16:07:22.738319 185101 main.cpp:39] END solve
E1115 16:07:22.738402 185101 main.cpp:50] --------------------------------------
E1115 16:07:22.740282 185101 main.cpp:35] Start solve threads:
E1115 16:07:22.744579 185125 main.cpp:31] Solution 0 objective: 0
E1115 16:07:22.745726 185101 main.cpp:38] CpSolverResponse summary:
status: OPTIMAL
objective: 0
best_bound: 0
booleans: 0
conflicts: 0
branches: 0
propagations: 0
integer_propagations: 1
restarts: 0
lp_iterations: 0
walltime: 0.00535886
usertime: 0.0053589
deterministic_time: 0.00089118
gap_integral: 0
I1115 16:07:22.745740 185101 main.cpp:39] END solve
E1115 16:07:22.747727 185101 main.cpp:35] Start solve threads:
E1115 16:07:22.751732 185133 main.cpp:31] Solution 0 objective: 0
E1115 16:07:22.753138 185101 main.cpp:38] CpSolverResponse summary:
status: OPTIMAL
objective: 0
best_bound: 0
booleans: 0
conflicts: 0
branches: 0
propagations: 0
integer_propagations: 1
restarts: 0
lp_iterations: 0
walltime: 0.00532127
usertime: 0.00532133
deterministic_time: 0.00089118
gap_integral: 0
I1115 16:07:22.753150 185101 main.cpp:39] END solve

CMAKE:

cmake_minimum_required(VERSION 3.16)
project(ortools_runner)

if (USE_PRECOMPILED_ORTOOLS)
  link_directories(/usr/local/lib)
  set(ORTOOLS_LIBS "ortools")
else()
  include(FetchContent)

  set (BUILD_SAMPLES OFF)
  set (BUILD_EXAMPLES OFF)
  FetchContent_Declare(
    ortools
    GIT_REPOSITORY https://github.com/google/or-tools.git
    GIT_TAG        master
  )

  set(BUILD_DEPS ON)
  FetchContent_MakeAvailable(ortools)
  set(ORTOOLS_LIBS "ortools::ortools")
endif()

set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

set(CMAKE_CXX_STANDARD 17)

#is include_directories needed?
include_directories(src)

add_executable(main src/main.cpp)
target_link_libraries(main ${ORTOOLS_LIBS})

add_custom_command(
        TARGET main POST_BUILD
        COMMAND ${CMAKE_COMMAND} -E copy
                ${CMAKE_SOURCE_DIR}/model_fail.proto
                ${CMAKE_CURRENT_BINARY_DIR}/model_fail.proto)
add_custom_command(
        TARGET main POST_BUILD
        COMMAND ${CMAKE_COMMAND} -E copy
                ${CMAKE_SOURCE_DIR}/infeasible_model.proto
                ${CMAKE_CURRENT_BINARY_DIR}/infeasible_model.proto)

@fdid-git
Copy link
Collaborator

I am looking into this. It seems to be a problem with our symmetry detection code.
So while waiting for the fix and for it to be pushed, if you run with parameters.set_symmetry_level(0), it seems to work fine.

@Phibedy
Copy link
Author

Phibedy commented Nov 16, 2021

@fdid-git Thank you :)

@lperron
Copy link
Collaborator

lperron commented Nov 19, 2021

Should be all fixed on master.

@lperron lperron closed this as completed Nov 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Solver: CP-SAT Solver Relates to the CP-SAT solver
Projects
None yet
Development

No branches or pull requests

4 participants