Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add weights to Kaplan Meier #396

Closed
pzivich opened this issue Jan 22, 2018 · 5 comments · Fixed by #395
Closed

Add weights to Kaplan Meier #396

pzivich opened this issue Jan 22, 2018 · 5 comments · Fixed by #395

Comments

@pzivich
Copy link
Contributor

pzivich commented Jan 22, 2018

Is there a way to add weights to individual observation in KaplanMeierFitter().fit()? This is useful for inverse probability censor weights.
For IPCW, logistic regression is used to generate probabilities of individuals being censored by a set of dependent variables (where the independent variable is censor=1). The inverse of the predicted probability is then used to weight observations.
I am currently looking up how SAS generates the weighted Kaplan Meier

@CamDavidsonPilon
Copy link
Owner

No, not atm, but there should be. All the pieces are in place for it. I'll add it to v0.14 release.

@CamDavidsonPilon CamDavidsonPilon mentioned this issue Jan 25, 2018
4 tasks
@pzivich
Copy link
Contributor Author

pzivich commented Feb 23, 2018

One item I wanted to add to this, the SE estimates from a weighed Kaplan-Meier are invalid. If confidence_intervals_ are requested on a weighted KM, an error or warning should be given to users.

Currently , the only option I am aware of for CI for a weighted KM is non-parametric bootstrapping

@CamDavidsonPilon
Copy link
Owner

Huh, interesting, can you explain why they are invalid? Any docs you recommend I read?

@pzivich
Copy link
Contributor Author

pzivich commented Feb 26, 2018

This might be my bad. It depends on the way the weights are generated. I am using inverse probability weights. Variances are invalid in this specific situations, since there is some residual correlations in the weights since they are generated via regression models. When using inverse probability weighted data, you need to use a robust variance estimator to account for these dependencies. As far as I know, there is not a version for Greenwood's variance for KM.

If you would like a source that discusses this specific issue somewhat: http://onlinelibrary.wiley.com/doi/10.1002/sim.7084/full
However, this discussion predominately focuses on Cox proportional hazards SE being invalid in IPW data. However, the same concern applies to KM

@CamDavidsonPilon
Copy link
Owner

Thanks for the advice. For now, I've included a warning to the user when they are using non-integer weights (the other use case - pre-aggregating subjects to make your dataset smaller - is valid however).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants