Skip to content

Conversation

@IsaacMilarky
Copy link
Contributor

@IsaacMilarky IsaacMilarky commented Jul 10, 2025

Remove PyGithub and Add Support for Gitlab and Bitbucket

Problem

PyGithub has less control over rate limiting than I would like it too for our purposes. We also did not previously have support for platforms other than GitHub

Solution

Remove PyGithub. Add functionality to fetch code.json files from GitLab and Bitbucket.

Testing

Ran main.py locally

Signed-off-by: Isaac Milarsky <imilarsky@gmail.com>
Signed-off-by: Isaac Milarsky <imilarsky@gmail.com>
Signed-off-by: Isaac Milarsky <imilarsky@gmail.com>
…gitlab orgs

Signed-off-by: Isaac Milarsky <imilarsky@gmail.com>
Signed-off-by: Isaac Milarsky <imilarsky@gmail.com>
@IsaacMilarky IsaacMilarky marked this pull request as ready for review July 10, 2025 18:43
Copy link
Collaborator

@sachin-panayil sachin-panayil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is great work! makes this way more modular and robust. i have a few comments / notes about stuff but overall, great work! 🔥

IDEA: we should implement pagination of API requests soon so it doesn't come back to bite us

for org in args.orgs.split(","):
org = org.strip()
indexGen.process_organization(org)
indexGen.process_github_org_files(org)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it seems like that when running the script, we call and check for Github orgs only but we should call to check all forges

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah we should update the script for that later in a separate PR.

Signed-off-by: Isaac Milarsky <imilarsky@gmail.com>
@IsaacMilarky IsaacMilarky merged commit 359ffde into main Jul 24, 2025
5 checks passed
@IsaacMilarky IsaacMilarky deleted the remove-github-lib branch July 24, 2025 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants