GithubGPT

AI-powered tools to facilitate pull request reviews and enable semantic search throughout GitHub repositories.

---

Table of Contents

• Overview
• Features
• Architecture
• Prerequisites
• Setup
  • 1. Clone the Repository
  • 2. Install Dependencies
  • 3. Configure Environment Variables
  • 4. Set Up a GitHub OAuth App
  • 5. Set Up Pinecone
• Running the Application
• Usage
  • 1. Log In with GitHub
  • 2. Fetch and Index a Repository
  • 3. Query the Repository
• Project Structure
• Environment Variables Reference

---

Overview

GithubGPT is a Flask web application that uses AI to make GitHub repositories searchable through natural language. Authenticate with your GitHub account, point the app at any repository you have access to, and then ask questions about the code in plain English. Answers are generated by combining semantic vector search (powered by Pinecone) with OpenAI's language models via LangChain.

---

Features

• GitHub OAuth Authentication — Securely log in with your GitHub account; no personal access tokens required.
• Repository Ingestion — Fetch source files from any GitHub repository and branch, filtered by file extension.
• Semantic Embeddings — File contents are split into chunks and embedded using OpenAI's text-embedding-ada-002 model, then stored in a Pinecone vector index.
• Natural Language Search — Ask questions about the codebase and receive AI-generated answers backed by the most relevant code snippets.
• Pull Request Review (in development) — Inline code review tooling driven by AI analysis of pull request diffs.

---

Architecture

Browser ──► Flask App (app/)
               │
               ├── GitHub OAuth (commonly_used/auth.py)
               │      └── Exchanges OAuth code for access token
               │
               ├── Data Fetching (commonly_used/data_fetching.py)
               │      └── Uses GitHub API to fetch repository file tree & content
               │
               └── Semantic Search (semantic_search/)
                      ├── search_engine.py  – indexes files into Pinecone, runs QA chain
                      └── embedding_generator.py – generates OpenAI embeddings

Key technologies:

Layer	Technology
Web framework	Flask
AI / LLM	OpenAI GPT (via LangChain)
Embeddings	OpenAI text-embedding-ada-002
Vector store	Pinecone (Serverless)
GitHub integration	GitHub OAuth 2.0 + REST API

---

Prerequisites

• Python 3.9+
• An OpenAI API key
• A Pinecone account with a serverless index (dimension 1536, metric cosine)
• A GitHub OAuth App

---

Setup

1. Clone the Repository

git clone https://github.com/Nzouh/GithubGPT.git
cd GithubGPT

2. Install Dependencies

It is recommended to use a virtual environment:

python -m venv venv
source venv/bin/activate   # Windows: venv\Scripts\activate
pip install -r requirements.txt

3. Configure Environment Variables

Create a .env file in the project root (this file is git-ignored):

OPEN_AI_KEY=your_openai_api_key
PINECONE_KEY=your_pinecone_api_key
PINECONE_INDEX_HOST=your_pinecone_index_host_url
GITHUB_CLIENT_ID=your_github_oauth_app_client_id
GITHUB_CLIENT_SECRET=your_github_oauth_app_client_secret
GITHUB_REDIRECT_URI=http://localhost:5000/callback

4. Set Up a GitHub OAuth App

1. Go to GitHub → Settings → Developer settings → OAuth Apps → New OAuth App.
2. Set the Authorization callback URL to http://localhost:5000/callback (or your production URL).
3. Copy the Client ID and Client Secret into your .env file.

5. Set Up Pinecone

1. Create a free account at pinecone.io.
2. Create a Serverless index named my-pinecone-index with:
   • Dimensions: 1536
   • Metric: cosine
   • Cloud / Region: AWS us-west-2 (or update the values in semantic_search/search_engine.py)
3. Copy your API key and index host URL into your .env file.

---

Running the Application

python main.py

The app will start on http://localhost:5000 in debug mode.

---

Usage

1. Log In with GitHub

Open http://localhost:5000 in your browser and click Login with GitHub. You will be redirected to GitHub to authorize the app, then sent back to the application.

2. Fetch and Index a Repository

After logging in, navigate to /fetch-files (you are redirected there automatically after login). Fill in the form:

Field	Description
Repository Owner	GitHub username or organization (e.g., your-username)
Repository Name	Repository name (e.g., your-repo)
Branch Name	Branch to index (default: main)
File Extensions	Comma-separated list of extensions to include (default: .py,.md,.txt)

Click Fetch and Process Files. The app will download each matching file, chunk it, generate embeddings, and store them in Pinecone.

3. Query the Repository

After indexing, you are redirected to the query page at /query. Type a natural language question about the repository (e.g., "Where is the authentication logic?" or "How are files fetched from GitHub?") and click Search. The app retrieves the most relevant code chunks from Pinecone and uses OpenAI to generate a human-readable answer.

---

Project Structure

GithubGPT/
├── main.py                        # Application entry point
├── requirements.txt               # Python dependencies
├── .gitignore
│
├── app/                           # Flask application
│   ├── __init__.py                # App factory (create_app)
│   ├── routes.py                  # URL routes: /, /login, /callback, /fetch-files, /query
│   └── templates/                 # Jinja2 HTML templates
│       ├── index.html
│       ├── fetch_files.html
│       ├── query.html
│       ├── query_results.html
│       ├── process_file.html
│       └── error.html
│
├── semantic_search/               # Embedding & search logic
│   ├── search_engine.py           # Repository indexing, semantic search, QA chain
│   └── embedding_generator.py     # Alternative embedding pipeline
│
├── commonly_used/                 # Shared utilities
│   ├── auth.py                    # GitHub OAuth URL generation & token exchange
│   └── data_fetching.py           # GitHub API: repo tree & file content fetching
│
├── pull_request_review/           # PR review module (in development)
│   └── code_review.py
│
└── tests/                         # Unit tests
    ├── test_code_review.py
    └── test_data_fetching.py

---

Environment Variables Reference

Variable	Description
OPEN_AI_KEY	OpenAI API key used for embeddings and language model inference
PINECONE_KEY	Pinecone API key
PINECONE_INDEX_HOST	Full host URL of the Pinecone index (e.g., https://my-pinecone-index-xxxx.svc.pinecone.io)
GITHUB_CLIENT_ID	GitHub OAuth App client ID
GITHUB_CLIENT_SECRET	GitHub OAuth App client secret
GITHUB_REDIRECT_URI	OAuth callback URL registered in your GitHub OAuth App

---

Demo

• Demo Video 1
• Demo Video 2