-
Notifications
You must be signed in to change notification settings - Fork 41
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Botorch with cardinality constraint via sampling #301
base: main
Are you sure you want to change the base?
Botorch with cardinality constraint via sampling #301
Conversation
0be3285
to
1f7783d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @Waschenbacher, thanks for the work. This is not yet a review but only a first batch of very high-level comments that I would like you to address before we can go into the actual review process. The reason being that the main functionality brought by this PR is currently rather convoluted and hard to parse, so I'd prefer to to work with a more readable version, tbh
1f7783d
to
9d28b49
Compare
f68ce4f
to
fc52875
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @Waschenbacher, I've finally managed to spend some time on this important PR, thanks again for your preparation work. I've already refactored some parts that were clear to me. However, I'm not yet certain about the design at the remaining places. I have the feeling that it can potentially be simplified a lot, depending on whether it is possible to reuse the idea of reduced subspaces. Have marked the corresponding places with comments. I think we need to discuss this part first before we can continue.
baybe/searchspace/continuous.py
Outdated
# TODO: Revise function name/docstring and arguments. In particular: why | ||
# does the function expect the inactive parameters instead of the active ones? | ||
|
||
# TODO: Shouldn't the x != 0 constraints be applied on the level of the | ||
# individual constrains, also taking into account whether min_cardinality > 0? | ||
|
||
# TODO: Instead of adding additional constraints, why not alter the parameter | ||
# bounds? In case we keep the constraints: is the sign of the threshold | ||
# correct? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here some points for discussion
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I made the following changes:
- Ensure active parameters != 0 by altering the parameter bounds
- By introducing
_FixedNumericalContinuousParameter
class, we ensure inactive parameters = 0 and active parameters != 0 in the copied subspace object. This is one way of communicating the subspace with inactive parameters. As discussed, we could also go with the option of the reduced subspace, then we need to manipulated the_botorch_acqf
.
acqf_values = torch.max(torch.cat(acqf_values_all)) | ||
return points, acqf_values | ||
|
||
def _recommend_continuous_without_cardinality_constraints( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I find the design here a bit weird. The method is called ..._without_cardinality_constraint
but somehow still takes care of inactive parameters internally. My general question here is: instead of fixing inactive values to 0.0, can't we apply the same logic of creating a reduced space instead? If that was possible, I think it would also drastically reduce the amount necessary code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe design in the revised version is not that misleading. Regarding the inactive parameters, please see my reply above #301 (comment)
… parameter setting
e9d0048
to
120d717
Compare
2767dd5
to
e6248b5
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Might be a stupid question, but why did you not leverage the nonlinear_inequality_constraints
argument of optimize_acqf
for this functionality (see here)? Is there a special reason?
This PR attempts to support cardinality constraints in botorch recommender. The core idea is
There are two mechanisms for determining the configuration of inactive parameters.
In addition, a validation is added to ensure that zero is included in the bounds of all cardinality constraint parameters.