Skip to content

Should *_would_be_qualified columns be derivations rather than QRF targets? #416

Description

@MaxGhenis

The Build M base jetsam (fixed mechanically in #415) exposed a modeling question worth an architecture-week look.

The QBI family (estate_income_would_be_qualified, farm_operations_income_would_be_qualified, farm_rent_income_would_be_qualified, partnership_s_corp_income_would_be_qualified, rental_income_would_be_qualified, self_employment_income_would_be_qualified, sstb_self_employment_income_would_be_qualified, plus business_is_sstb) are 3–6-atom near-categorical columns imputed through the primary PUF QRF chain. A quantile forest is a strange model for a six-level ratio: it can only reproduce the atom set via leaf quantiles, the atoms look like they encode qualification-rule levels applied to the base income column, and the chain pays a full forest per target for what may be a deterministic (or simple conditional-categorical) relationship to columns drawn earlier in the chain.

Questions for the review:

  1. What defined these columns in the retired us-data pipeline — a rule over the base column, or independent survey/PUF signal? (Chesterton fence: find the origin PR before changing anything.)
  2. If rule-derived: they belong in the deterministic-derivations stage (populace#395 canonical ordering), run once on the assembled multispine, not per-source QRF.
  3. If genuinely stochastic: a categorical draw (per-level gate) is cheaper and better-specified than a QRF over atoms.

Not launch-blocking — #415's leaf bound makes the current model tractable and Build M proceeds under it. Filing so the modeling question doesn't vanish behind the memory fix.

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions