GitNews is an automated blog article generator that analyzes commits from GitHub repositories, identifies technical novelties (new packages, design patterns, architectural changes), and generates detailed articles using OpenAI's GPT-4. Articles are stored in PostgreSQL with pgvector embeddings to prevent duplicate content. Built with .NET 8 following Clean Architecture.
Available as a one-shot CLI tool or a scheduled background worker that runs daily.
- 🔍 Automatic Repository Scanning - Lists and processes all repositories from a GitHub account
- 🤖 AI-Powered Article Generation - Uses GPT-4 to analyze commit diffs and generate technical blog posts
- 🧠 Embedding-Based Deduplication - Generates vector embeddings to detect and skip similar articles
- 📊 Smart Commit Filtering - Tracks processed commits in PostgreSQL to avoid reprocessing
- 🆕 New Project Detection - Projects with ≤3 commits get a project description; others focus on technical novelties
- ⏰ Scheduled Worker - Background service that runs daily at a configurable time
- 🐳 Docker Ready - Full Docker Compose setup with PostgreSQL and the Worker service
- .NET 8 - Target framework for all projects
- PostgreSQL 16 - Primary data store with pgvector extension
- Entity Framework Core 8 - ORM with Code First migrations
- Pgvector - Vector similarity search for article deduplication
- Octokit 13 - GitHub API client for repository and commit data
- OpenAI API - GPT-4 for article generation, text-embedding-3-small for embeddings
- Docker / Docker Compose - Containerized deployment
- GitVersion - Semantic versioning from commit messages
- GitHub Actions - Automated tagging and release creation
GitNews/
├── GitNews.DTO/ # Data Transfer Objects, settings classes
├── GitNews.Infra.Interfaces/ # Repository & AppService interfaces (generics)
│ ├── AppServices/ # IGitHubAppService, IBlogGeneratorAppService, IEmbeddingAppService
│ └── Repository/ # IProcessedCommitRepository, IArticleRepository
├── GitNews.Domain/ # Entity models, service interfaces & implementations
│ ├── Interfaces/ # IGitNewsProcessorService
│ ├── Models/ # Article, ProcessedCommit
│ └── Services/ # GitNewsProcessorService (orchestration)
├── GitNews.Infra/ # Infrastructure implementations
│ ├── AppServices/ # GitHubAppService, BlogGeneratorAppService, EmbeddingAppService
│ ├── Context/ # GitNewsDbContext, design-time factory
│ ├── Migrations/ # EF Core migrations
│ └── Repository/ # ProcessedCommitRepository, ArticleRepository
├── GitNews.Application/ # DI composition root (Startup.cs)
├── GitNews.Console/ # CLI entry point (one-shot execution)
├── GitNews.Worker/ # Background worker (scheduled daily)
├── docker-compose.yml # PostgreSQL + Worker services
├── .env.example # Environment variable template
└── CLAUDE.md # AI assistant instructions
GitNews.Console ─┐
GitNews.Worker ─┤→ GitNews.Application → GitNews.Infra → GitNews.Domain
│ → GitNews.DTO
│ → GitNews.Infra.Interfaces → GitNews.DTO
cp .env.example .envPOSTGRES_DB=gitnews
POSTGRES_USER=postgres
POSTGRES_PASSWORD=your_secure_password_here
POSTGRES_PORT=5432
GITHUB_TOKEN=your_github_token_here
GITHUB_OWNER=your_github_username
GITHUB_MAXCOMMITS=30
GITHUB_INCLUDEFORKS=false
OPENAI_APIKEY=your_openai_api_key_here
OPENAI_MODEL=gpt-4
OPENAI_BASEURL=https://api.openai.com/v1
WORKER_SCHEDULETIME=19:00- Never commit the
.envfile with real credentials - You need a GitHub Personal Access Token with
reposcope - You need an OpenAI API Key with access to GPT-4 and embeddings
cp .env.example .env
# Edit .env with your credentialsdocker-compose up -d --buildThis starts:
- PostgreSQL 16 with pgvector on port 5432
- GitNews Worker scheduled to run daily at the configured time
docker-compose ps
docker-compose logs -f worker| Action | Command |
|---|---|
| Start services | docker-compose up -d |
| Start with rebuild | docker-compose up -d --build |
| Stop services | docker-compose stop |
| View status | docker-compose ps |
| View worker logs | docker-compose logs -f worker |
| Remove containers | docker-compose down |
| Remove containers and volumes ( |
docker-compose down -v |
- .NET 8 SDK
- PostgreSQL 16 with pgvector extension
- GitHub Personal Access Token
- OpenAI API Key
docker-compose up -d postgrescp GitNews.Console/appsettings.example.json GitNews.Console/appsettings.json
# Edit appsettings.json with your credentialsdotnet builddotnet run --project GitNews.ConsoleOr with CLI arguments:
dotnet run --project GitNews.Console -- \
--owner your-github-username \
--github-token ghp_xxx \
--openai-key sk-xxx \
--connection-string "Host=localhost;Port=5432;Database=gitnews;Username=postgres;Password=postgres"cp GitNews.Worker/appsettings.example.json GitNews.Worker/appsettings.json
# Edit appsettings.json
dotnet run --project GitNews.Worker| Option | Description | Default |
|---|---|---|
-o, --owner |
GitHub account owner | — |
--github-token |
GitHub access token | — |
--openai-key |
OpenAI API key | — |
-m, --model |
ChatGPT model | gpt-4 |
--max-commits |
Max commits per repo | 30 |
--include-forks |
Include forked repositories | false |
--connection-string |
PostgreSQL connection string | — |
-h, --help |
Show help | — |
All settings can be configured via environment variables using the ASP.NET Core __ convention:
| Variable | Description |
|---|---|
GitHub__Token |
GitHub access token |
GitHub__Owner |
Account owner |
OpenAI__ApiKey |
OpenAI API key |
OpenAI__Model |
ChatGPT model |
Database__ConnectionString |
PostgreSQL connection string |
1. GitHub API → List all repositories for an owner
2. For each repo → Fetch README, commit count, recent commits with diffs
3. PostgreSQL → Filter out already-processed commits
4. OpenAI GPT-4 → Analyze diffs, generate article (title, content, category, tags)
5. OpenAI Embeddings → Generate vector embedding for the article
6. pgvector → Check similarity against existing articles (skip duplicates)
7. PostgreSQL → Save article and mark commits as processed
articles- Generated blog articles with pgvector embedding column (1536 dimensions)processed_commits- Tracks processed commits with unique index on(repository, sha)
Migrations are auto-applied on startup. To create a new migration:
dotnet ef migrations add <MigrationName> \
--project GitNews.Infra \
--startup-project GitNews.Consoledocker-compose exec postgres pg_dump -U postgres gitnews > backup.sqldocker-compose exec -T postgres psql -U postgres gitnews < backup.sqlVersion and Tag (version-tag.yml):
- Triggers on push to
main - Uses GitVersion for semantic versioning based on commit messages
- Creates and pushes Git tags automatically
Create Release (create-release.yml):
- Triggers after successful tagging
- Creates GitHub Releases for major/minor changes
- Generates changelog from commit messages
| Prefix | Version Bump |
|---|---|
major: or breaking: |
Major (x.0.0) |
feat: or feature: |
Minor (0.x.0) |
fix: or patch: |
Patch (0.0.x) |
Check:
docker-compose ps postgres
docker-compose logs postgresCommon causes:
- PostgreSQL container not running
- Wrong connection string or port
- pgvector extension not installed
Common causes:
- Too many repositories being processed
- Token without proper permissions
Solutions:
- Reduce
MaxCommitssetting - Use a token with
reposcope - The app has built-in retry with exponential backoff for rate limits
Common causes:
- Invalid API key
- Insufficient credits
- Model not available
Solutions:
- Verify API key at platform.openai.com
- Check billing and usage limits
- Try a different model (e.g.,
gpt-3.5-turbo)
Contributions are welcome! Please feel free to submit a Pull Request.
- Fork the repository
- Create a feature branch (
git checkout -b feature/AmazingFeature) - Make your changes
- Build and verify (
dotnet build) - Commit your changes (
git commit -m 'feat: add some AmazingFeature') - Push to the branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Developed by Rodrigo Landim Carneiro
This project is licensed under the MIT License - see the LICENSE file for details.
- Issues: GitHub Issues
⭐ If you find this project useful, please consider giving it a star!