Skip to content

🌱 cron: repair gitlab project list #4658

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

spencerschrock
Copy link
Member

What kind of change does this PR introduce?

cron update

What is the current behavior?

Many projects were missing the last character of their URL, which was causing 404 errors and resulting in empty shard files in the cron.

What is the new behavior (if this is a feature change)?**

The list was repaired using a bash script and an earlier copy of this file:

touch out.csv
while read original; do
  # remove trailing comma
  no_comma=$(echo -n $original | sed 's/,*\r*$//')
  # look for the complete line in the old file
  grep "$no_comma.,$" gitlab-projects.csv.1 >> out.csv
  # if not in the old copy, just keep the line the same
  if [ $? -eq 1 ]; then
    echo "$original" >> out.csv
  fi
done <gitlab-projects.csv

https://raw.githubusercontent.com/ossf/scorecard/8521e516d809edf2b2696e302464d3d5984816a2/cron/internal/data/gitlab-projects.csv

Here's an example diff:

diff --git a/cron/internal/data/gitlab-projects.csv b/cron/internal/data/gitlab-projects.csv
index 5e0b8ca2..70b02d0a 100755
--- a/cron/internal/data/gitlab-projects.csv
+++ b/cron/internal/data/gitlab-projects.csv
@@ -1,61 +1,62 @@
 repo,metadata
-https://gitlab.com/0100001001000010/config-loade,
-https://gitlab.com/0100001001000010/simple-gui-prompt,
-https://gitlab.com/01luisfonseca/canvas-image-resize,
-https://gitlab.com/01luisfonseca/express-reverse-prox,
-https://gitlab.com/01luisfonseca/file-image-resize,
-https://gitlab.com/01luisfonseca/lfutil,
-https://gitlab.com/06chaynes/okta-jwt-verifie,
+https://gitlab.com/0100001001000010/config-loader,
+https://gitlab.com/0100001001000010/simple-gui-prompts,
+https://gitlab.com/01luisfonseca/canvas-image-resizer,
+https://gitlab.com/01luisfonseca/express-reverse-proxy,
+https://gitlab.com/01luisfonseca/file-image-resizer,
+https://gitlab.com/01luisfonseca/lfutils,
+https://gitlab.com/06chaynes/okta-jwt-verifier,
 https://gitlab.com/09jwater/Needle,
  • Tests for the changes have been added (for bug fixes/features)

Which issue(s) this PR fixes

NONE

Special notes for your reviewer

Does this PR introduce a user-facing change?

For user-facing changes, please add a concise, human-readable release note to
the release-note

(In particular, describe what changes users might need to make in their
application as a result of this pull request.)

NONE

Many projects were missing the last character of their URL, which was
causing 404 errors and resulting in empty shard files in the cron. The
list was repaired using a bash script and an earlier copy of this file:

https://raw.githubusercontent.com/ossf/scorecard/8521e516d809edf2b2696e302464d3d5984816a2/cron/internal/data/gitlab-projects.csv

Signed-off-by: Spencer Schrock <sschrock@google.com>
@spencerschrock spencerschrock requested a review from a team as a code owner June 13, 2025 22:17
@spencerschrock spencerschrock requested review from justaugustus and raghavkaul and removed request for a team June 13, 2025 22:17
Copy link

codecov bot commented Jun 13, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.44%. Comparing base (353ed60) to head (7b01482).
⚠️ Report is 201 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4658      +/-   ##
==========================================
+ Coverage   66.80%   68.44%   +1.63%     
==========================================
  Files         230      249      +19     
  Lines       16602    19032    +2430     
==========================================
+ Hits        11091    13026    +1935     
- Misses       4808     5133     +325     
- Partials      703      873     +170     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@spencerschrock
Copy link
Member Author

Will wait to merge until we get the token issue resolved

Copy link

This pull request has been marked stale because it has been open for 10 days with no activity

Copy link

This pull request has been marked stale because it has been open for 10 days with no activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants