CapeCod.predict discards the fitted apriori when the model was fit on exactly one fewer index level

[ppcvote]

Are you on the latest chainladder version?

• Yes, this bug occurs on the latest version (v0.10.0, verified against main at 5f34bcd).

Revised after @henrydingliu's note: this issue originally said "grain" where it meant index levels. Reworded throughout, since grain properly refers to origin and development period granularity.

Describe the bug in words

When a CapeCod model is fit on training data that has already been aggregated to fewer index levels, predict() silently discards the fitted apriori and recomputes it at the full index. Pooling sparse segments is the main reason to aggregate before fitting, so the returned apriori_ is not the quantity the caller fitted.

The guard at chainladder/methods/capecod.py:325 reads:

# If model was fit at a higher grain, then need to aggregate predicted aprioris too
if len(set(sample_weight.key_labels) - set(self.apriori_.key_labels)) > 1:

That set difference is the number of index levels the prediction carries which the fitted apriori does not, so it is >= 1 exactly when the model was fit on fewer levels. At > 1 the commonest case, one fewer level, falls through to the else and recomputes at the full index.

How can the bug be reproduced?

Using the repository's own clrd sample, whose key_labels are ['GRNAME', 'LOB']:

import chainladder as cl
import numpy as np

clrd = cl.load_sample("clrd")
tri  = clrd[clrd["LOB"] == "comauto"]["CumPaidLoss"]
prem = clrd[clrd["LOB"] == "comauto"]["EarnedPremDIR"].latest_diagonal

# aggregate to one fewer index level before fitting: LOB only, dropping GRNAME
cc = cl.CapeCod().fit(tri.groupby("LOB").sum(),
                      sample_weight=prem.groupby("LOB").sum())

print(len(set(prem.key_labels) - set(cc.apriori_.key_labels)))   # 1  -> guard is False
print(float(np.asarray(cc.apriori_).ravel()[0]))                 # 0.5689995797  (fitted)

pred = cc.predict(tri, sample_weight=prem)
print(np.asarray(pred.apriori_).ravel()[:1])                     # 1.2516635774  (recomputed)

The fitted apriori is 0.5689995797; predict() returns 1.2516635774. I checked the fitted value by hand in numpy, sum(latest) / sum(premium / cdf), and it agrees to 1e-9.

The existing predict tests cannot catch this because both fit at ['Line'] on prism, whose key_labels are six long, so the difference is always 5 and they always take the intended branch. clrd has a difference of 1 and is not covered.

What is the expected behavior?

assert np.allclose(np.asarray(pred.apriori_), np.asarray(cc.apriori_))

Changing the threshold to > 0 produces that. I checked the blast radius: with > 0 the diff == 0 case is bit-identical (still 1.2352941176 on the same data) and diff >= 2 is unchanged, so only the broken case moves.

One thing worth deciding rather than assuming: with > 0, a genuinely disjoint index turns today's ValueError: Index broadcasting is ambiguous into a less helpful KeyError. An explicit subset check would read better than a count, and I did not want to pick that shape for you.

Would you be willing to contribute this ticket?

• Yes, absolutely!

[henrydingliu]

@ppcvote thanks for catching! a few comments:

• can you revise the issue title and related wording to avoid using grain when talking about index levels? i got massively confused initially
• i'm onboard with the fix of going from >1 to >0. Also onboard with a more explicit subset check
• i'm actually not sure we should leave the current api as is. Development takes 'groupby' as a parameter. CapdCod appear to expect 'groupby' to be performed on the training data directly. this likely causes issues when trying to construct pipelines with CapeCod. interested to hear your take

[ppcvote]

Thanks @henrydingliu. Title and body reworded, "grain" now only means origin and development granularity and index levels are called index levels. That confusion was mine and it was worth fixing.

On the API, one correction that I think changes the question: CapeCod already takes groupby (capecod.py:188, documented at :28), and it does work through predict:

cc = cl.CapeCod(groupby="LOB").fit(tri, sample_weight=prem)
# apriori_ is a single pooled value across all 10 index entries: 0.5602627308
# predict() returns the same value

So the pipeline-safe path exists today. What is broken is the other way of expressing the same intent: aggregating the training data yourself and letting predict() infer it by comparing key_labels. That heuristic is what carries the > 1.

Which makes the design question sharper than "should CapeCod take groupby", and I think it is really: should that inference exist at all, now that the parameter does? Three options as I see them, in increasing order of disruption. Fix the threshold and leave both paths. Fix it and document groupby as the supported one. Or drop the inference and require groupby, which is breaking and probably wants a deprecation cycle.

One more thing that argues for a single path, and I only found it while checking your question. The two routes do not agree:

route	apriori
CapeCod(groupby="LOB")	0.5602627308
pre-aggregated training data	0.5689995797

groupby aggregates latest and reported_exposure and then takes the ratio (capecod.py:251-253), whereas pre-aggregating sums the triangles and fits on the result. Those are different estimators, not a rounding difference, and today a user can reach either one without being told which they got.

My suggestion, but it is your call: land > 0 as the bug fix so the documented behaviour works on the current release, and treat the inference-versus-parameter question as its own issue, since it is a deprecation decision rather than a defect. Happy to write either, or both.

[henrydingliu]

My suggestion, but it is your call: land > 0 as the bug fix so the documented behaviour works on the current release

i'm on board

treat the inference-versus-parameter question as its own issue

please create a separate issue. we can continue the discussion there.

[ppcvote]

Opened #1275 for the > 0 fix, and #1274 for the API question.