Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add params to control solver, default Clarabel, pin cvxpy version #1975

Merged
merged 15 commits into from
Jun 25, 2024

Conversation

minhkhul
Copy link
Contributor

@minhkhul minhkhul commented Jun 17, 2024

Rollback this change and replace with version pinning

Description

ECOS solver is used by hospital admissions (claims_hosp) and doctor_visits weekday-adjusted signals. Recently this solver causes severe numerical problems in some of doctor_visits outlier datapoints. This is not a problem in hospital admissions. Therefore, we change the solver used in doctor_visits to Clarabel and keep ECOS use in claims_hosp for consistency.

Knowing that the default solver in cvxpy 1.5 onward will be Clarabel while previous versions keep ECOS, we pin the versions accordingly for these two indicators.

Changelog

  • remove clarabel spec in weekday
  • pin claims_hosp cvxpy setup to <=1.4
  • pin doctor_visits cvxpy setup to >=1.5
  • Specify use of clarabel/ecos as params.

Fixes

@minhkhul minhkhul requested a review from melange396 June 17, 2024 18:11
@minhkhul minhkhul changed the title Rollback Clarabel, pin cvxpy version instead Add params to control solver, default Clarabel, pin cvxpy version Jun 18, 2024
_delphi_utils_python/delphi_utils/weekday.py Outdated Show resolved Hide resolved
_delphi_utils_python/delphi_utils/weekday.py Outdated Show resolved Hide resolved
_delphi_utils_python/delphi_utils/weekday.py Outdated Show resolved Hide resolved
claims_hosp/delphi_claims_hosp/update_indicator.py Outdated Show resolved Hide resolved
claims_hosp/setup.py Outdated Show resolved Hide resolved
logger,
) if weekday else None
if weekday and np.any(np.all(params == 0,axis=1)):
params = (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you want to get a smaller diff, you can probably revert the changes to this file, which will get it ignored by the linter.

minhkhul and others added 2 commits June 21, 2024 12:17
Co-authored-by: george <george.haff@gmail.com>
Co-authored-by: george <george.haff@gmail.com>
@minhkhul minhkhul requested a review from melange396 June 21, 2024 16:21
Copy link
Contributor

@melange396 melange396 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

plz change the test data formatting, and then this is good to go!

Comment on lines +33 to +50
expected_result = [
-0.05993665,
-0.0727396,
-0.05618517,
0.0343405,
0.12534997,
0.04561813,
-2.27669028,
-1.89564374,
-1.5695407,
-1.29838116,
-1.08216513,
-0.92089259,
-0.81456355,
-0.76317802,
-0.76673598,
-0.82523745,
]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you make the format of this match the non-legacy test above so its easier to eyeball the differences in values?

Comment on lines +155 to +166
wd_params = (
Weekday.get_params_legacy(
data_frame,
"den",
["num"],
Config.DATE_COL,
[1, 1e5],
logger,
)
if self.weekday
else None
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the linter is a cruel mistress! id argue that my suggestion has a better style, even if the previous format passed, but the existing code is a fossilized remnant so do what thou wilt...

Suggested change
wd_params = (
Weekday.get_params_legacy(
data_frame,
"den",
["num"],
Config.DATE_COL,
[1, 1e5],
logger,
)
if self.weekday
else None
)
if weekday:
wd_params = Weekday.get_params_legacy(
data_frame,
"den",
["num"],
Config.DATE_COL,
[1, 1e5],
logger,
)
else:
wd_params = None

@minhkhul minhkhul merged commit e2033c3 into main Jun 25, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants