Update raw_score output for v2 scoring by rlronan · Pull Request #37 · 2ndSetAI/good-egg

rlronan · 2026-02-24T15:28:07Z

Fixes #36
The raw score returned with the v2 method should be the normalized score pre-normalization. For the v2 method that variable is logit, rather than raw_score. This PR updates the returned value:

        return TrustScore(
            user_login=login,
            context_repo=context_repo,
-           raw_score=raw_score,
+           raw_score=logit,
            normalized_score=normalized,
            trust_level=trust_level,
            account_age_days=user_data.profile.account_age_days,

Note that this does not effect the computations of the score (which are working correctly), it updates the returned values to be more coherent. The docs are updated to explain the difference in the returned raw_score for v1 and v2.

Diff of the output change when running v2

@@ -1,7 +1,7 @@
 {
   "user_login": "rlronan",
   "context_repo": "axolotl-ai-cloud/axolotl",
-  "raw_score": 0.0,
+  "raw_score": -0.2917947228989848,
   "normalized_score": 0.42756454607181854,
   "trust_level": "MEDIUM",
   "percentile": 0.0,

Note that sigmoid(-0.2917947228989848) = 0.42756454607181854.

Linting and tests pass

…ethod

gemini-code-assist · 2026-02-24T15:28:25Z

Summary of Changes

Hello @rlronan, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request refines the output of the v2 scoring method by ensuring the raw_score field consistently reflects the pre-normalization logit value. This change improves the coherence of the returned data structure without altering the underlying score computations. Accompanying documentation updates provide clarity on the distinction between v1 and v2 raw_score interpretations and detail the v2 normalization process.

Highlights

V2 Raw Score Output: The raw_score field in the TrustScore output for v2 scoring now correctly returns the logit value, which represents the pre-normalization score.
Documentation Clarity: Documentation has been updated to clearly explain the distinction of the raw_score field's meaning between v1 (pagerank value) and v2 (logit) scoring methods.
Methodology Explanation: A detailed explanation of the v2 normalization methodology, including the logit calculation, has been added to the methodology.md file.

Changelog

docs/library.md
- Updated the description of the raw_score field to differentiate between v1 (pagerank value) and v2 (logit).
docs/mcp-server.md
- Modified the example raw_score value in the API response to reflect a v2 logit value.
docs/methodology.md
- Clarified the v1 normalization process.
- Added a new section detailing the v2 normalization process, including the formula for calculating the logit and the final normalized score.
- Explicitly stated that raw_score in v2 output contains the pre-sigmoid logit value.
src/good_egg/scorer.py
- Modified the _score_v2 function to assign the logit variable to the raw_score field in the TrustScore object.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

github-actions · 2026-02-24T15:28:30Z

🥚 Better Egg: HIGH Trust

Score: 73%

Score Breakdown

Component	Value
Graph Score	68%
Merge Rate	87% (20/23 PRs)
Account Age	2,895 days

Top Contributions

Repository	PRs	Language	Stars
rlronan/ghp-measurement	11	Python	0
suragnair/alpha-zero-general	3	Jupyter Notebook	4381
rlronan/alpha-zero-general	3	Jupyter Notebook	0
axolotl-ai-cloud/axolotl	1	Python	11373
rlronan/SLE-GAN	1	Python	0
rlronan/cnn_vis_demo	1	Python	1

gemini-code-assist

Code Review

The pull request effectively addresses the issue of raw_score output for v2 scoring by returning the logit value instead of the raw_score. The documentation updates clearly explain this change and the difference between v1 and v2 raw_score definitions. The code change is minimal and directly targets the identified issue, improving the coherence of the returned values.

docs/library.md

gemini-code-assist · 2026-02-24T15:29:52Z

docs/mcp-server.md

  "trust_level": "HIGH",
  "normalized_score": 0.82,
-  "raw_score": 0.0045,
+  "raw_score": 0.2871,


The example output for raw_score in the get_trust_details tool has been updated to reflect the new logit value for v2. This ensures consistency between the documentation and the actual API response.

docs/methodology.md

gemini-code-assist · 2026-02-24T15:29:52Z

docs/methodology.md

+**v2:** The normalized graph score is combined with merge rate and account age into a logit, then passed through a sigmoid:
+
+```
+logit = intercept + graph_score_weight * graph_score
+      + merge_rate_weight * merge_rate
+      + account_age_weight * log(account_age_days + 1)
+
+normalized = 1 / (1 + e^(-logit))
+```
+
+`raw_score` in the v2 output contains the pre-sigmoid logit value.


The new section clearly explains the v2 normalization process and explicitly states that raw_score in v2 output contains the pre-sigmoid logit value. This is crucial for users to correctly interpret the v2 results.

jeffreyksmithjr · 2026-02-24T15:54:00Z

Please don’t use the page rank terminology.

rlronan · 2026-02-24T17:07:34Z

Please don’t use the page rank terminology.

fixed

The prior assertion (raw_score > 0) passed incidentally but didn't validate the raw_score/normalized_score relationship. Since raw_score is now the logit, it can be negative.

jeffreyksmithjr

Yeah, all seems reasonable. Made one tiny change to testing.

Change the raw score to represent the unnormalized logit for the v2 m…

f81fb6d

…ethod

rlronan requested a review from jeffreyksmithjr February 24, 2026 15:28

gemini-code-assist bot reviewed Feb 24, 2026

View reviewed changes

terminology fixes

f939337

Assert sigmoid(raw_score) == normalized_score in v2 test

786e3c7

The prior assertion (raw_score > 0) passed incidentally but didn't validate the raw_score/normalized_score relationship. Since raw_score is now the logit, it can be negative.

jeffreyksmithjr approved these changes Mar 2, 2026

View reviewed changes

jeffreyksmithjr merged commit 4f957dc into main Mar 2, 2026
2 checks passed

jeffreyksmithjr deleted the fix/raw-score-output-v2 branch March 2, 2026 13:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update raw_score output for v2 scoring#37

Update raw_score output for v2 scoring#37
jeffreyksmithjr merged 3 commits intomainfrom
fix/raw-score-output-v2

rlronan commented Feb 24, 2026 •

edited

Loading

Uh oh!

gemini-code-assist bot commented Feb 24, 2026

Uh oh!

github-actions bot commented Feb 24, 2026 •

edited

Loading

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

Uh oh!

gemini-code-assist bot Feb 24, 2026

Uh oh!

Uh oh!

gemini-code-assist bot Feb 24, 2026

Uh oh!

jeffreyksmithjr commented Feb 24, 2026

Uh oh!

rlronan commented Feb 24, 2026

Uh oh!

jeffreyksmithjr left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

rlronan commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist bot commented Feb 24, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

github-actions bot commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🥚 Better Egg: HIGH Trust

Score Breakdown

Top Contributions

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

gemini-code-assist bot Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gemini-code-assist bot Feb 24, 2026

Choose a reason for hiding this comment

Uh oh!

jeffreyksmithjr commented Feb 24, 2026

Uh oh!

rlronan commented Feb 24, 2026

Uh oh!

jeffreyksmithjr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rlronan commented Feb 24, 2026 •

edited

Loading

github-actions bot commented Feb 24, 2026 •

edited

Loading