Add uncertainty intervalls for predictions (CET/DERT/CLV)

[mmeierer]

As a universal approach for all methods in CLVTools, implement a boostrapping procedure to be able to generate uncertainty intervalls for predictions (CET/DERT/CLV)

[tm185246]

Hello,

Wanted to check if this is implemented, specifically for the pnbd with dynamic covariates? I think this would be a highly useful feature, desirable for many different reasons.

Thanks,
Tom

[mmeierer]

This is not yet implemented.

Would you mind sharing your use cases? Depending on whether these use cases appeal to a broader audience, we might move this feature up on our priority list.

[tm185246]

I am implementing CLV predictions for a client and they have asked directly for confidence intervals for the predictions. The more (or less) confident we are about a certain prediction may influence how the client reaches out to a customer; they may choose to view a wider range for the prediction corresponding to a bigger opportunity for upselling. They also would like to have a confidence interval for how much revenue the customers will produce on aggregate over the next X years.

Hope that makes sense. Thanks!

[mmeierer]

Thanks. We might look into adding an implementation for boostrapping but this procedure is likely to be rather slow (as pointed out by previous literature). However, for a consultancy project that does not require to repeatly such an anlaysis it might be somewhat usable - depending on the size of your data.

[tm185246]

Understood. I would offer to help but am only just learning R. Anyways, thanks for creating and maintaining this package! From: Markus Meierer ***@***.***> Date: Monday, April 17, 2023 at 9:57 AM To: bachmannpatrick/CLVTools ***@***.***> Cc: Murray, Thomas ***@***.***>, Comment ***@***.***> Subject: Re: [bachmannpatrick/CLVTools] Add uncertainty intervalls for predictions (CET/DERT/CLV) (#170) *External Message* - Use caution before opening links or attachments Thanks. We might look into adding an implementation for boostrapping but this procedure is likely to be rather slow (as pointed out by previous literature). However, for a consultancy project that does not require to repeatly such an anlaysis it might be somewhat usable - depending on the size of your data. — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/bachmannpatrick/CLVTools/issues/170*issuecomment-1511404313__;Iw!!In4Qlw!pZibXpnKWW0acyaAack85RslgV-kiww2rXMzmQnKls33B1bdYiuKdH1symriXGzHkCV0oFC30SupU3X7vV9_FxVujw$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/AXHRELIDND5BDAS64HC5JR3XBVD3DANCNFSM44A4CIJQ__;!!In4Qlw!pZibXpnKWW0acyaAack85RslgV-kiww2rXMzmQnKls33B1bdYiuKdH1symriXGzHkCV0oFC30SupU3X7vV8EpFDwkQ$>. You are receiving this because you commented.Message ID: ***@***.***>

[tm185246]

Out of curiosity, would the way to bootstrap here be to resample from our cohorts with replacement, fit the model, store our predictions, and repeat many times? I'm guessing that we wouldn't have a production for each customer each time, but if we repeat many times we will end up with ample predictions for each customer anyway.

Or is there another way to bootstrap this?

[pschil]

You should be able to get confidence intervals using bootstrapping.
Randomly selecting customers and all their respective transactions with replacement should be the way to go (and not randomly selecting transactions). From the many predicted CLV values of each customer you can then make 95% CI bounds for each customer.

You are right that with this approach you will not get predictions for every customer in each sample but it should even out if there are enough repetitions as some customers are also present more than once per sample.

[tm185246]

Awesome. Thank you!