ODP IBNR bug

[rbreen2]

It seems that starting in version 0.8.5 after creating the samples for the ODP bootstrap, when you go to run cl.Chainladder().fit(samples).ibnr_.sum("origin").mean()
it would produce an oddly large negative number. However running this on data from the tutorial clrd it works. So i'm not to sure what the issue is but when I do the same thing on a lower version like 0.8.4 it works.
Thanks,

[jbogaardt]

Hi @rbreen2, can you provide more info on what "samples" is if it is not from 'clrd'? If we can reproduce the issue, we can resolve it, but I'm unable to replicate the issue currently.

[rbreen2]

Hello @jbogaardt ,
So using the same data is RAA except putting it on an excel first and reading it with the package xlwings to get the df:
raa_df = table.range('A1').options(pd.Series,index=False, header=True, expand='table').value
then producing a triangle and running the bootstrap:
raa = cl.Triangle(raa_df, origin="origin", development="development", columns="values")
samples = (cl.BootstrapODPSample(n_sims=10000).fit(raa).resampled_triangles_)
cl.Chainladder().fit(samples).ibnr_.sum("origin").mean()
Produces the value: -492582.88

But when reading RAA from the package it self:
raa = cl.load_sample("raa")
samples = (cl.BootstrapODPSample(n_sims=10000).fit(raa).resampled_triangles_)
cl.Chainladder().fit(samples).ibnr_.sum("origin").mean()
Produces the value: 53782.89

Both triangles 'raa' are the same and when I run:
cl.Chainladder().fit(samples).ultimate_.sum("origin").mean()
I get about the same answer for both ways being 214625.23

Let me know if I'm missing something or need to explain further, thank you!

[jbogaardt]

This feels like the two versions of RAA are subtlely different, though I am not sure why. Can you possibly share the version of RAA from Excel by doing raa.to_json() and copying the output here? That will help me figure out which attribute(s) are different.

[rbreen2]

Ah thank you, you're right by checking the json I realized I have do cumulative=True:
raa = cl.Triangle(raa_df, cumulative=True, origin="origin", development="development", columns="values")
After doing that it works!

Here's what the json looked like before when I pulled it from excel:
'{"metadata": "{\"is_val_tri\": false, \"is_cumulative\": null, \"is_pattern\": false, \"columns\": [\"values\"]}", "data": "{\"columns\":[\"Total\",\"origin\",\"valuation\",\"values\"],\"index\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54],\"data\":[[\"Total\",347155200000000000,378691199999999999,5012.0],[\"Total\",347155200000000000,410227199999999999,3257.0],[\"Total\",347155200000000000,441763199999999999,2638.0],[\"Total\",347155200000000000,473385599999999999,898.0],[\"Total\",347155200000000000,504921599999999999,1734.0],[\"Total\",347155200000000000,536457599999999999,2642.0],[\"Total\",347155200000000000,567993599999999999,1828.0],[\"Total\",347155200000000000,599615999999999999,599.0],[\"Total\",347155200000000000,631151999999999999,54.0],[\"Total\",347155200000000000,662687999999999999,172.0],[\"Total\",378691200000000000,410227199999999999,106.0],[\"Total\",378691200000000000,441763199999999999,4179.0],[\"Total\",378691200000000000,473385599999999999,1111.0],[\"Total\",378691200000000000,504921599999999999,5270.0],[\"Total\",378691200000000000,536457599999999999,3116.0],[\"Total\",378691200000000000,567993599999999999,1817.0],[\"Total\",378691200000000000,599615999999999999,-103.0],[\"Total\",378691200000000000,631151999999999999,673.0],[\"Total\",378691200000000000,662687999999999999,535.0],[\"Total\",410227200000000000,441763199999999999,3410.0],[\"Total\",410227200000000000,473385599999999999,5582.0],[\"Total\",410227200000000000,504921599999999999,4881.0],[\"Total\",410227200000000000,536457599999999999,2268.0],[\"Total\",410227200000000000,567993599999999999,2594.0],[\"Total\",410227200000000000,599615999999999999,3479.0],[\"Total\",410227200000000000,631151999999999999,649.0],[\"Total\",410227200000000000,662687999999999999,603.0],[\"Total\",441763200000000000,473385599999999999,5655.0],[\"Total\",441763200000000000,504921599999999999,5900.0],[\"Total\",441763200000000000,536457599999999999,4211.0],[\"Total\",441763200000000000,567993599999999999,5500.0],[\"Total\",441763200000000000,599615999999999999,2159.0],[\"Total\",441763200000000000,631151999999999999,2658.0],[\"Total\",441763200000000000,662687999999999999,984.0],[\"Total\",473385600000000000,504921599999999999,1092.0],[\"Total\",473385600000000000,536457599999999999,8473.0],[\"Total\",473385600000000000,567993599999999999,6271.0],[\"Total\",473385600000000000,599615999999999999,6333.0],[\"Total\",473385600000000000,631151999999999999,3786.0],[\"Total\",473385600000000000,662687999999999999,225.0],[\"Total\",504921600000000000,536457599999999999,1513.0],[\"Total\",504921600000000000,567993599999999999,4932.0],[\"Total\",504921600000000000,599615999999999999,5257.0],[\"Total\",504921600000000000,631151999999999999,1233.0],[\"Total\",504921600000000000,662687999999999999,2917.0],[\"Total\",536457600000000000,567993599999999999,557.0],[\"Total\",536457600000000000,599615999999999999,3463.0],[\"Total\",536457600000000000,631151999999999999,6926.0],[\"Total\",536457600000000000,662687999999999999,1368.0],[\"Total\",567993600000000000,599615999999999999,1351.0],[\"Total\",567993600000000000,631151999999999999,5596.0],[\"Total\",567993600000000000,662687999999999999,6165.0],[\"Total\",599616000000000000,631151999999999999,3133.0],[\"Total\",599616000000000000,662687999999999999,2262.0],[\"Total\",631152000000000000,662687999999999999,2063.0]]}", "sub_tris": {}, "dfs": {}}'

[jbogaardt]

Oh, thank you for finding the root cause. I'll keep this one open since we should coerce a triangle to cumulative upon fit rather than have it just fail like this. Similar to how we do in the Development estimator here:
https://github.com/casact/chainladder-python/blob/master/chainladder/development/development.py#L112-L115

[rbreen2]

Another bug I found has to do with the Mack Standard Error, it works when RAA is used but if you delete the lastest origin period being 1990 and Dev year 1990. Version 0.8.8 creates a much larger standard error compared to 0.8.2. I’m not sure why but I’m assuming it has to do with the triangle not being a perfect nxn, Here’s the code that I used after putting RAA data on excel and deleting Origin 1990:

raa_df = table.range('A1').options(pd.Series,index=False, header=True, expand='table').value
raa = cl.Triangle(raa_df, cumulative = True, origin="origin", development="development", columns="value")
mack = cl.MackChainladder().fit(raa)
mack.summary_
mack.total_mack_std_err_.values[0, 0]
Version 0.8.2 Produces : 10009.022070082632
Version 0.8.8 Produces : 76362.26020670775

[jbogaardt]

I don't quite grasp the triangle censoring you're describing. It sounds like this, but I get a different answer:

>>> import chainladder as cl
>>> cl.__version__
'0.8.8'
>>> raa = cl.load_sample('raa').iloc[..., :-1, :-1]
>>> cl.MackChainladder().fit(raa).total_mack_std_err_.values[0,0]
9859.53887525988

At a high level, it sounds like you're trying to remove the latest_diagonal, but even here I get a different answer:

>>> import chainladder as cl
>>> cl.__version__
'0.8.8'
>>> raa = cl.load_sample('raa')
>>> raa = raa[raa.valuation<raa.valuation_date]
>>> cl.MackChainladder().fit(raa).total_mack_std_err_.values[0,0]
35505.59688575841

Can you clarify the what RAA looks like, even if a pic?

[rbreen2]

Sorry, let me know if you can open this link to the pic of what I mean:
https://user-images.githubusercontent.com/44329608/138373105-9116238a-5f9f-4ce3-9fb6-fc764bfecb79.JPG
Only Origin 1990 Development 12, I removed.