Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cost over time broken. Combined Budget does not show all configurations #154

Closed
LukasFehring opened this issue Jun 18, 2024 · 1 comment
Closed

Comments

@LukasFehring
Copy link
Collaborator

When plotting with the cost over time plugin, some configurations get neglected if looking at the combined budget.

@dengdifan please provide the zip file to replicate this.

@sarah-segel
Copy link
Collaborator

I think the problem here is that in the Combined budget case, the mouseover-text only shows the cost/time at the highest budget of the config, no matter if the point shown is on another budget. With that changed in #158, in the example run smac_run.zip
one can see that the best accuracy is first reached by the (non-incumbent) config 26 on budget 50, while the incumbent config is actually config 230 (which achieves the same accuracy on budget 100, but later than the other config on budget 50).

As I feel like Combined budget is not that intuitive for Cost-over-Time, I switched the default budget to the highest budget here.

@dengdifan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants