Cached results creates confusing simulation IDs

[yngve-sk]

When running an experiment, cache hits currently alter the number of simulations in a batch. Many users are used to this order being deterministic. We should somehow make it more understandable to them that the results are not gone, they are just not re-computed and exist within another realization.

• If we print warnings, this may crash/interfere with the CLI simulation status printout, so it should be indicated in the status
• In the GUI rundialog it would not be possible to show cache hits, as the GUI mirrors actual running forward models

[oyvindeide]

Would it work to start by sending events whenever this happens? Could be fairly straight forward, just say that we are not recomputing due to cache hit.

[yngve-sk]

Doing something whenever it happens should be fairly easy, so we could send some events.

    evaluation_infos = self._create_evaluation_infos(
            control_values=control_values,
            model_realizations=model_realizations,
            perturbations=evaluator_context.perturbations.tolist()
            if evaluator_context.perturbations is not None
            else [-1] * len(model_realizations),
            active_controls=active_control_vectors,
            control_names=self._everest_config.formatted_control_names,
            objective_names=objective_names,
            constraint_names=self._everest_config.constraint_names,
            all_results=all_results,
        )
        
    cache_hits = [e for e in evaluation_infos if e.status == _EvaluationStatus.CACHED]

would have the info needed to do something, the event could be sent, and caught and passed on to the monitor I guess. a CACHE_HIT event or something like that