fix: provide a more accurate problem scale log when entity dependent value ranges are used for list variables

[Christopher-Chianelli]

Previously, the logic just assumed the values from entity dependent value ranges are unique, which massively inflated the problem scale log for list variables.

Now, we track the number of entities a value is allowed to be in, and use that to scale the problem scale. For instance, if a value is only allowed to be in 25% of entities, it should reduce the problem scale by 25%.

[triceo]

You may not be aware, but since not so long ago, the solver actually has a new component - ValueRangeManager. It already does a lot of heavy lifting around value ranges, and I think it could easily be extended to providing the information for the scale computation.

The core idea here being that these loops over all entities and values aren't cheap, and it'd be best if they were performed just once. Can you please look into integrating this with the ValueRangeManager, so that the amount of iteration over large collections can be minimized?

[Christopher-Chianelli]

Are you suggesting combining computeInitializationStatistics and getProblemScale in ValueRangeStatistics? The ValueRangeManager already manages problem scale via ProblemSizeStatistics.

[triceo]

Looking at the code, I see ValueRangeManager.computeInitializationStatistics already does a loop over all entities, so it could IMO make sense to use that loop to prepare the data for the problem scale as well.

More importantly, though - ValueRangeState already holds all the values of all value ranges enumerated. Which means there already was a loop which created all of this data. And we should hook into that loop. Maybe we can update ValueRangeState to hold the value counts. See this:

private @Nullable Map<Value_, Integer> fromSolutionValueIndexMap;
private @Nullable Map<Entity_, ValueRangeItem<Solution_, Entity_, ValueRange<Value_>, Value_>> fromEntityMap;

The value in these maps can be extended to carry the information about value count, or perhaps another map added. And with that minimal update, we have all the information we need to create the problem scale when asked for it.

Here's the thing: problem scale is exclusively used to write information to log output. It has no business being expensive. It would be ridiculous if the solver had to be held back by this. If we now have to enumerate all value ranges to make the computation correct, then let's hook into the place which already does it, as opposed to duplicating those expensive loops.

[triceo]

Already much better, but I think we can still improve.

[triceo]

LGTM when green.

[sonarqubecloud]

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
92.1% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud