[Bug]: Duplicate trials are not being flagged as such and can be requested repeatedly. #3550
Description
What happened?
The short circuit to Sobol is a nice to have feature but in our constrained problem we frequently see duplicates that do not trigger the short circuit despite having the should_deduplicate flag set in the generation node. Anecdotally this is most pronounced for cases where the point being duplicated sits on a constraint boundary.
Whether or not an arm is duplicate is checked based on a hash self.md5hash(self.parameters) of the arm parameters. This seems as though it may be sensitive to floating point differences which may be a potential explanation.
Please provide a minimal, reproducible example of the unexpected behavior.
GenerationStep(
model=Generators.BOTORCH_MODULAR,
num_trials=-1,
model_kwargs={
"surrogate": surrogate,
"acquisition_class": SEBOAcquisition,
"botorch_acqf_class": qLogNoisyExpectedHypervolumeImprovement,
"acquisition_options": {
"penalty": "L0_norm",
"target_point": target_point,
"sparsity_threshold": len(parameters),
},
"fit_out_of_design": True,
},
should_deduplicate=True,
)
[INFO 03-20 21:25:27] ax.service.ax_client: Generated new trial 44 with parameters {'a': 0.0, 'b': 0.0, 'c': 0.349082, 'd': 0.0, 'e': 0.0} using model BoTorch.
[INFO 03-20 21:26:59] ax.service.ax_client: Generated new trial 45 with parameters {'a': 0.0, 'b': 0.7, 'c': 0.0, 'd': 0.0, 'e': 0.0} using model BoTorch.
[INFO 03-20 21:27:22] ax.service.ax_client: Generated new trial 46 with parameters {'a': 0.0, 'b': 0.584608, 'c': 0.415392, 'd': 0.0, 'e': 0.0} using model BoTorch.
[INFO 03-20 21:29:09] ax.service.ax_client: Generated new trial 47 with parameters {'a': 0.0, 'b': 0.7, 'c': 0.0, 'd': 0.0, 'e': 0.0} using model BoTorch.
[INFO 03-20 21:30:11] ax.service.ax_client: Generated new trial 48 with parameters {'a': 0.0, 'b': 0.7, 'c': 0.0, 'd': 0.0, 'e': 0.0} using model BoTorch.
[INFO 03-20 21:30:52] ax.service.ax_client: Generated new trial 49 with parameters {'a': 0.0, 'b': 0.7, 'c': 0.0, 'd': 0.0, 'e': 0.0} using model BoTorch.
[INFO 03-20 21:31:47] ax.service.ax_client: Generated new trial 50 with parameters {'a': 0.0, 'b': 0.7, 'c': 0.0, 'd': 0.0, 'e': 0.0} using model BoTorch.
[INFO 03-20 21:32:39] ax.generation_strategy.generation_node: The generator run produced duplicate arms. Re-running the generation step in an attempt to deduplicate. Candidates produced in the last generator run: [Arm(name='45_0', parameters={'a': 0.0, 'b': 0.7, 'c': 0.0, 'd': 0.0, 'e': 0.0})].
[INFO 03-20 21:33:28] ax.service.ax_client: Generated new trial 51 with parameters {'a': 0.0, 'b': 0.7, 'c': 0.0, 'd': 0.0, 'e': 0.0} using model BoTorch.
Please paste any relevant traceback/logs produced by the example provided.
Ax Version
0.5.0
Python Version
3.12.8
Operating System
Linux
Code of Conduct
- I agree to follow Ax's Code of Conduct