Skip to content

Verify dbt assessor weights don't dilute scores for non-dbt projects #354

@kami619

Description

@kami619

Summary

PR #334 added 4 dbt assessors with a combined 26% weight (10+10+3+3). Need to verify that non-dbt projects are unaffected when these assessors return is_applicable=False.

Question

How does the scoring engine handle assessors that return is_applicable=False?

  • If it skips them and re-normalizes weights: no impact (correct behavior)
  • If it includes them with a 0 score: every non-dbt project's overall score drops

What to check

  1. Look at the scoring aggregation logic (likely in src/agentready/services/ or the assessment pipeline)
  2. Confirm is_applicable=False assessors are excluded from weight normalization
  3. Run a before/after comparison on a non-dbt repo (e.g., any Python project on the leaderboard)

Context

The Tier 1 assessor count went from 6 to 10 with this PR. If weights aren't re-normalized, Tier 1 now sums to more than intended.

Found in

PR #334 — Feature/add dbt support

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions