Leaderboard: redhat/rhel-ai/wheels/builder (78.6/100 - Gold)

[ryanpetrello]

Leaderboard Submission

Organization: Red Hat / RHEL AI
Submitted by: @ryanpetrello

Note: This repository is hosted on GitLab, not GitHub. Submitting manually as the CLI only supports GitHub URLs.

Repos & Scores

Repository	Score	Tier
redhat/rhel-ai/wheels/builder	78.6/100	Gold

Validation Checklist

• Repository exists and is public
• Submitter has commit access
• Assessment re-run passes (±2 points tolerance)
• JSON schema valid

[coderabbitai]

Warning

.coderabbit.yaml has a parsing error

The CodeRabbit configuration file in this repository has a parsing error and default settings were used instead. Please fix the error(s) in the configuration file. You can initialize chat with CodeRabbit to get help with the configuration file.

💥 Parsing errors (1)

Validation error: String must contain at most 250 character(s) at "tone_instructions"

⚙️ Configuration instructions

• Please see the configuration documentation for more information.
• You can also validate your configuration using the online YAML validator.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Walkthrough

Adds two new assessment JSON artifacts containing agentready evaluation reports with metadata, repository/context details, overall scoring and certification, and per-attribute findings (status, measured values, thresholds, evidence, and remediation guidance).

Changes

Cohort / File(s)	Summary
Builder assessment submissions/redhat/builder/2026-03-25T12-00-00-assessment.json	New JSON assessment for builder with schema/version, timestamps, executed command, repo details, scoring/certification, and detailed findings covering project layout, dependency pinning, .gitignore, README length, docstring coverage, CI/CD visibility, and other code-quality attributes.
Rhai-pipeline assessment submissions/redhat/rhai-pipeline/2026-03-25T12-00-00-assessment.json	New JSON assessment for rhai-pipeline containing schema/metadata, repo and run context, overall score/certification, findings array with per-attribute statuses, numeric scores, evidence, remediation steps, and config/duration metadata.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (2 warnings)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The PR description claims to submit only redhat/builder (78.6/100), but the changeset includes assessment reports for both redhat/builder and redhat/rhai-pipeline (53.8/100), creating a mismatch between the stated and actual submissions.	Either update the PR description to include both repositories with their respective scores and tiers, or remove the redhat/rhai-pipeline assessment file if it was added unintentionally.
Title check	⚠️ Warning	The title references 'redhat/rhel-ai/wheels/builder' with a score of '78.6/100 - Gold', but the changeset contains assessment files for both 'redhat/builder' and 'redhat/rhai-pipeline', with the latter being Bronze-tier (53.8/100). The title is misleading about the actual scope of changes.	Update the title to accurately reflect that the PR adds leaderboard submissions for multiple Red Hat repositories, or clarify which repository is the primary focus and whether the secondary submission should be in a separate PR.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[github-actions]

📈 Test Coverage Report

Branch	Coverage
This PR	66.8%
Main	66.8%
Diff	✅ +0%

---

Coverage calculated from unit tests only

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@submissions/redhat/builder/2026-03-25T12-00-00-assessment.json`:
- Around line 201-205: The JSON fragment has inconsistent dependency_security
metadata: the "status" is "pass" while "score" is 35 and "threshold" requires
"≥60 points"; update the artifact so these three fields agree—either set
"status" to "fail" (or equivalent) to match the 35 score, or raise "score" to
meet the threshold and adjust "measured_value" if needed; ensure you modify the
same object containing the "status", "score", and "threshold" keys so
leaderboard ingestion validates consistently.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 38e45e4b-ce9e-41d0-8e66-d239e6b3b1b7

📥 Commits

Reviewing files that changed from the base of the PR and between 0022e29 and 7495a09.

📒 Files selected for processing (1)

• submissions/redhat/builder/2026-03-25T12-00-00-assessment.json

[coderabbitai]

⚠️ Potential issue | 🟠 Major

Status/score threshold is internally inconsistent and can invalidate leaderboard ingestion.

At Line 201-205, dependency_security is marked "status": "pass" while score is 35 and threshold is ≥60 points. This is a correctness issue in the submitted artifact and can cause downstream validation/reporting conflicts.

Proposed JSON fix

-      "status": "pass",
+      "status": "fail",

As per coding guidelines, "Focus on major issues impacting performance, readability, maintainability and security. Avoid nitpicks and avoid verbosity."

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@submissions/redhat/builder/2026-03-25T12-00-00-assessment.json` around lines
201 - 205, The JSON fragment has inconsistent dependency_security metadata: the
"status" is "pass" while "score" is 35 and "threshold" requires "≥60 points";
update the artifact so these three fields agree—either set "status" to "fail"
(or equivalent) to match the 35 score, or raise "score" to meet the threshold
and adjust "measured_value" if needed; ensure you modify the same object
containing the "status", "score", and "threshold" keys so leaderboard ingestion
validates consistently.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@submissions/redhat/rhai-pipeline/2026-03-25T12-00-00-assessment.json`:
- Around line 12-29: The submitted assessment file
submissions/redhat/rhai-pipeline/2026-03-25T12-00-00-assessment.json does not
match the PR target (it describes repository.name "rhai-pipeline", overall_score
53.8, certification_level "Bronze" while the PR is for the "builder" repo with a
different score/certification); fix this by either replacing the uploaded
assessment JSON with the correct builder assessment file or updating the PR to
reference the rho-pipeline change and remove the incorrect file; ensure the JSON
top-level "repository" object (fields like "name", "url", "commit_hash",
"overall_score", and "certification_level") accurately reflects the repository
the PR is modifying so validation and leaderboard ingestion succeed.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: b45fee4d-0920-4b83-9956-83a130926887

📥 Commits

Reviewing files that changed from the base of the PR and between 7495a09 and 8f9c148.

📒 Files selected for processing (1)

• submissions/redhat/rhai-pipeline/2026-03-25T12-00-00-assessment.json

[coderabbitai]

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify what assessment files exist in the PR branch
echo "=== Assessment files in submissions directory ==="
fd -e json . submissions/ --exec basename {}

echo ""
echo "=== Check for any builder-related assessment files ==="
fd -g '*builder*' submissions/

echo ""
echo "=== Repository info from all assessment JSON files ==="
fd -e json . submissions/ --exec sh -c 'echo "File: {}"; jq -r ".repository.name, .repository.url, .overall_score, .certification_level" {} 2>/dev/null | paste - - - - || echo "Failed to parse"'

Repository: ambient-code/agentready

Length of output: 2738

---

🏁 Script executed:

#!/bin/bash
# Read the file under review to extract key fields
echo "=== File under review: rhai-pipeline assessment ==="
cat submissions/redhat/rhai-pipeline/2026-03-25T12-00-00-assessment.json | head -40

echo ""
echo "=== Extract key fields from rhai-pipeline file ==="
python3 <<'EOF'
import json
import sys

try:
    with open('submissions/redhat/rhai-pipeline/2026-03-25T12-00-00-assessment.json', 'r') as f:
        data = json.load(f)
        print(f"Repository name: {data['repository']['name']}")
        print(f"Repository URL: {data['repository']['url']}")
        print(f"Overall score: {data['overall_score']}")
        print(f"Certification level: {data['certification_level']}")
except Exception as e:
    print(f"Error: {e}")
    sys.exit(1)
EOF

echo ""
echo "=== Also check the builder assessment file ==="
python3 <<'EOF'
import json
import sys

try:
    with open('submissions/redhat/builder/2026-03-25T12-00-00-assessment.json', 'r') as f:
        data = json.load(f)
        print(f"Repository name: {data['repository']['name']}")
        print(f"Repository URL: {data['repository']['url']}")
        print(f"Overall score: {data['overall_score']}")
        print(f"Certification level: {data['certification_level']}")
except Exception as e:
    print(f"Error: {e}")
    sys.exit(1)
EOF

Repository: ambient-code/agentready

Length of output: 1685

---

Critical: File under review does not match PR objectives.

The file submitted is the assessment for rhai-pipeline (score: 53.8, Bronze tier), but the PR appears to be for builder (score: 78.6, Gold tier). These are two completely different repositories assessed on the same date with different scores and certification levels.

Either the wrong assessment file was submitted in this PR, or the PR description is incorrect. This mismatch will cause validation to fail and result in the wrong repository being added to the leaderboard.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@submissions/redhat/rhai-pipeline/2026-03-25T12-00-00-assessment.json` around
lines 12 - 29, The submitted assessment file
submissions/redhat/rhai-pipeline/2026-03-25T12-00-00-assessment.json does not
match the PR target (it describes repository.name "rhai-pipeline", overall_score
53.8, certification_level "Bronze" while the PR is for the "builder" repo with a
different score/certification); fix this by either replacing the uploaded
assessment JSON with the correct builder assessment file or updating the PR to
reference the rho-pipeline change and remove the incorrect file; ensure the JSON
top-level "repository" object (fields like "name", "url", "commit_hash",
"overall_score", and "certification_level") accurately reflects the repository
the PR is modifying so validation and leaderboard ingestion succeed.

[github-actions]

🎉 This PR is included in version 2.31.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀