feat(web-ui): add private-GitHub ingestion via PAT #286
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…authentication * Accept a GitHub personal access token (PAT) from the UI and forward it through - `git_form.jinja` → new “Private Repository” checkbox + PAT field - routers (`index.py`, `dynamic.py`) and `query_processor.py` * Propagate `token` throughout the ingestion stack - `gitingest.entrypoint.parse_query` - `query_parsing` (including `try_domains_for_user_and_repo`) so we can infer the host when the user enters a bare “user/repo” slug * Tests - Added `"token": ""` to the `form_data` dict in the tests in `tests/test_flow_integration.py` **Limitation:** This PR enables PAT-protected cloning **only for GitHub**; other hosts (GitLab, Gitea, etc.) remain public-only for now.
filipchristiansen
requested review from
Copilot and
cyclotruc
and removed request for
Copilot
Contributor
There was a problem hiding this comment.
Pull Request Overview
Adds support for supplying a GitHub personal access token (PAT) so private repos can be ingested through the web UI and backend.
- UI: New “Private Repository” checkbox and masked PAT field in the ingest form
- Backend: Thread
tokenfrom HTTP form through routers,process_query, parsing, andclone_repo - Tests: Updated integration and unit tests to include
tokenin form data and clone calls
Reviewed Changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/test_repository_clone.py | Renamed test variables to clone_config and updated assertions |
| tests/test_flow_integration.py | Added "token": "" to all form_data entries |
| src/server/templates/components/git_form.jinja | Inserted PAT checkbox, input field, and toggle logic |
| src/server/routers/index.py | Accept token in form, normalize empty token to None |
| src/server/routers/dynamic.py | Same updates as index router for catch-all paths |
| src/server/query_processor.py | Forward token into clone_repo call |
| src/gitingest/query_parsing.py | Extend parse_query and _parse_remote_repo to accept token |
| src/gitingest/entrypoint.py | Pass token into parse_query |
Comments suppressed due to low confidence (2)
tests/test_flow_integration.py:66
- Consider adding test cases where a non-empty
tokenis provided to cover the private-repo ingestion path and assert that the token is passed downstream correctly.
"token": "",
src/gitingest/query_parsing.py:26
- Add validation to ensure the provided token matches the expected GitHub PAT format (e.g. starts with 'github_pat_' or 'gph_') and reject or warn on invalid values.
async def parse_query(
cyclotruc
approved these changes
Jun 21, 2025
Member
cyclotruc
left a comment
cyclotruc
left a comment
There was a problem hiding this comment.
Tested and works wonderfully!
added a little link to help generate a token
This PR is a big step in private repo support, until we are able to roll a proper Oauth integration
One idea for future improvement:
Store the PAT in some http only cookies so users can re-use them in subsequent visits to gitingest.com
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why
Until now the gitingest.com web UI could ingest only public repositories.
Teams often need to analyse private GitHub repos, so we’re adding a way to supply a personal access token (PAT).
What’s inside
git_form.jinjathat reveals a masked PAT field.• Checkbox state + token are posted with the form.
index.py,dynamic.py,query_processor.pyaccepttokenfrom the form and thread it downstream.gitingest.entrypoint.parse_queryandquery_parsingnow carrytoken, lettingtry_domains_for_user_and_repo()resolve bareuser/reposlugs against GitHub also when a PAT is present."token": ""toform_dataintests/test_flow_integration.pyto keep path-coverage intact.Limitation: This PR enables PAT-protected cloning only for GitHub; other hosts (GitLab, Gitea, etc.) remain public-only for now.