Integrate new version of the model that jointly models the competing risk incidences #53

ogrisel · 2024-07-01T15:04:55Z

This involves renaming the model class to SurvivalBoost as a short and easy to remember name.

Fix TODO in survival_boost_test.py

…timation

Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>

DOC documents parameters in SurvivalBoost docstring

…survival_boost

…ck_predict_survival_boost

…oost API change the shape of the incidence curves

DOC Use data generator in marginal cumulative incidence example

…ng estimator

ogrisel · 2024-07-04T16:06:07Z

@juAlberge @Vincent-Maladiere @judithabk6 I think this branch is in a good enough shape to be reviewed for merging.

it exposes the new multi-incidence estimator with the new SurvivalBoost as decided during the sprint,
it updated the tests and example to match the new API;
it removed things that are not covered by the tests;
it hides the IPCW estimator API to keep it private for now. I plan to open a dedicated PR to implement the generic wrapper we discussed with @judithabk6 at the sprint but this is better done in a dedicated PR.

Note: I also updated the marginal competing risk estimation example to use KM based IPCW as this example is a case of a marginal censoring mechanism. Using KM-IPCW really improves the quality of the plots on censored data. Even when tweaking the other hyper-parameters I could not get the alternating IPCW strategy to work well on this example. This will also be investigated later.

* survival analysis example * change the `predict_cumutive_incidence` function to `predict_survival_function` * fix the example and adapt it to sphinx * add torch to doc deps because because pycox doesn't require it * clean example --------- Co-authored-by: LeoGrin <leo.grinsztajn@polytechnique.edu> Co-authored-by: Vincent Maladiere <maladiere.vincent@yahoo.fr>

Vincent-Maladiere

LGTM! 🎉

…competing risk incidences (#53) * removing Incidence * adding MultiIncidence to hazardosu * reinserting example for multiincidence * Hardcode uniform time sampler to simplify estimator * Remove any reference to the dataset-specific oracle scorer * Rename to SurvivalBoost * Update the .score method to work in the any event setting * Rename test file * Fix test_gradient_boosting_incidence_parameter_tuning * Simplify code * Raise error if hard-zero resampling censors too many events * adding test * Refactor survival boost tests * Fix missing assertion * fix TODO * fix linter * Fix LaTeX * More informative error message for bad value for hard_zero_fraction * Remove untested code related to Cox PH based conditional censoring estimation * Avoid using 'Est' as a short hand for 'Estimator' * FIX accept both float and array-like for time_horizon in predict_proba * DOC update predict_proba docstring * TST add a test that check the behaviour of predict_proba * DOC add docstring to test * TST check that probability sum to one for all events * DOC fix docstring type for time_horizon * Rename BaseIPCW to KaplanMeierIPCW. Adding some documentation. * API correct the API of predict_proba * Use arturo formulation * Silence 'DeprecationWarning: invalid escape sequence' by using raw docstrings with LaTeX expressions * update docstring * Simplify KaplanMeierIPCW and improve its docstring * adding some documentation to SurvivalBoost * More docstring fixes * Apply suggestions from code review Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org> * TST update test and code to follow scikit-learn convention * TST check that we override properly * Apply suggestions from code review Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org> * DOC start documenting parameter * DOC documents parameters in SurvivalBoost docstring * DOC add ipcw estimator docstring * DOC add attributes, reference, and examples * DOC document some parameters and attributes of WeightedMultiClassTargetSampler * API change the shape of the incidence curves * DOC Use data generator in marginal cumulative incidence example * FIX slice properly the prediction * fix install instructions with flit * MAINT Remove old python version in black config (#57) * MAINT activate doctest check with pytest by default * Fix a pandas warning in load_seer * FIX do not pytest collect outside of the hazardous module folder * FIX update the example to integrate on the right columns * Apply suggestions from code review Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org> * Set explicit value for n_events * Improve variable names and comments in WeightedMultiClassTargetSampler * Use n_horizons_per_observation=3 by default * Update hazardous/_survival_boost.py * Remove reference to "column" to describe the second axis of a 3D array. * Improve docstrings for SurvivalBoost * Make sure to use zero-width param ranges in marginal example * Silence AJ warning on tied events * Small improvements * Use a string in the public API of SurvivalBoost to select the censoring estimator * Keep IPCW estimator private API for now * replacing any occurence of _est to _estimator * fixing math, harmonising different notations. * Example survival analysis (#70) * survival analysis example * change the `predict_cumutive_incidence` function to `predict_survival_function` * fix the example and adapt it to sphinx * add torch to doc deps because because pycox doesn't require it * clean example --------- Co-authored-by: LeoGrin <leo.grinsztajn@polytechnique.edu> Co-authored-by: Vincent Maladiere <maladiere.vincent@yahoo.fr> * enhance documentation --------- Co-authored-by: Julie Alberge <julie.alberge@gmail.com> Co-authored-by: Guillaume Lemaitre <guillaume@probabl.ai> Co-authored-by: ArturoAmorQ <arturo.amor-quiroz@polytechnique.edu> Co-authored-by: Judith Abecassis <judithabk6@users.noreply.github.com> Co-authored-by: Jovan Stojanovic <62058944+jovan-stojanovic@users.noreply.github.com> Co-authored-by: Julie <107467312+juAlberge@users.noreply.github.com> Co-authored-by: LeoGrin <leo.grinsztajn@polytechnique.edu> Co-authored-by: Vincent Maladiere <maladiere.vincent@yahoo.fr> 5e581d5

juAlberge and others added 30 commits June 19, 2024 12:07

removing Incidence

99a4480

adding MultiIncidence to hazardosu

9b9735c

reinserting example for multiincidence

a63ef45

Hardcode uniform time sampler to simplify estimator

01b2629

Remove any reference to the dataset-specific oracle scorer

e0f9fd0

Rename to SurvivalBoost

ab06498

Update the .score method to work in the any event setting

0671d98

Rename test file

846c019

Fix test_gradient_boosting_incidence_parameter_tuning

d40906a

Simplify code

59e7edd

Raise error if hard-zero resampling censors too many events

c0e02eb

adding test

6deca3e

Refactor survival boost tests

306feb8

Fix missing assertion

f3f1925

fix TODO

7e21186

fix linter

52134bd

Fix LaTeX

f245f8a

Merge pull request #52 from soda-inria/enhance_survival_boost_test

fb78d83

Fix TODO in survival_boost_test.py

Merge remote-tracking branch 'origin/main' into multi-incidence

ca8e009

More informative error message for bad value for hard_zero_fraction

3f47477

Remove untested code related to Cox PH based conditional censoring es…

527f88d

…timation

Avoid using 'Est' as a short hand for 'Estimator'

94e7397

FIX accept both float and array-like for time_horizon in predict_proba

3c722ae

DOC update predict_proba docstring

a68a9e9

TST add a test that check the behaviour of predict_proba

c14c21a

DOC add docstring to test

9f9f750

TST check that probability sum to one for all events

7595c7f

DOC fix docstring type for time_horizon

071fe63

Rename BaseIPCW to KaplanMeierIPCW. Adding some documentation.

4376f36

API correct the API of predict_proba

3b90460

glemaitre and others added 19 commits July 3, 2024 16:10

FIX update the example to integrate on the right columns

23736de

Apply suggestions from code review

1af117b

Co-authored-by: Olivier Grisel <olivier.grisel@ensta.org>

Merge pull request #56 from glemaitre/improve_doc

40e1774

DOC documents parameters in SurvivalBoost docstring

Set explicit value for n_events

6cb17cd

Merge remote-tracking branch 'origin/main' into modify_check_predict_…

47ceb40

…survival_boost

Merge remote-tracking branch 'origin/multi-incidence' into modify_che…

8a6bcaa

…ck_predict_survival_boost

Improve variable names and comments in WeightedMultiClassTargetSampler

f8defea

Use n_horizons_per_observation=3 by default

dd7990b

Update hazardous/_survival_boost.py

0a38c24

Remove reference to "column" to describe the second axis of a 3D array.

e6f0d90

Merge pull request #61 from glemaitre/modify_check_predict_survival_b…

259aeda

…oost API change the shape of the incidence curves

Improve docstrings for SurvivalBoost

c05f601

Make sure to use zero-width param ranges in marginal example

e5e7f33

Merge branch 'multi-incidence' into use_data_generator

ee8af69

Silence AJ warning on tied events

d90cf4f

Small improvements

d0834c3

Merge pull request #62 from ArturoAmorQ/use_data_generator

5efd0ee

DOC Use data generator in marginal cumulative incidence example

Use a string in the public API of SurvivalBoost to select the censori…

139ef51

…ng estimator

Keep IPCW estimator private API for now

6c644a6

juAlberge added 2 commits July 5, 2024 12:19

replacing any occurence of _est to _estimator

012cd5e

fixing math, harmonising different notations.

176c54b

Vincent-Maladiere mentioned this pull request Jul 9, 2024

Add the c-index with IPCW #71

Open

5 tasks

This comment was marked as resolved.

Sign in to view

juAlberge and others added 2 commits August 15, 2024 18:58

enhance documentation

cc6b90c

Vincent-Maladiere approved these changes Aug 20, 2024

View reviewed changes

Vincent-Maladiere merged commit 5e581d5 into main Aug 20, 2024
9 checks passed

Vincent-Maladiere deleted the multi-incidence branch August 20, 2024 16:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrate new version of the model that jointly models the competing risk incidences #53

Integrate new version of the model that jointly models the competing risk incidences #53

ogrisel commented Jul 1, 2024

ogrisel commented Jul 4, 2024 •

edited

Loading

This comment was marked as resolved.

Vincent-Maladiere left a comment

Integrate new version of the model that jointly models the competing risk incidences #53

Integrate new version of the model that jointly models the competing risk incidences #53

Conversation

ogrisel commented Jul 1, 2024

ogrisel commented Jul 4, 2024 • edited Loading

This comment was marked as resolved.

Vincent-Maladiere left a comment

Choose a reason for hiding this comment

ogrisel commented Jul 4, 2024 •

edited

Loading