Git Repositories Manager

An interactive Python script for managing multiple Git repository branch pulls and synchronization. Supports batch processing, parallel execution, automatic stashing, and comprehensive logging.

Features

• Automatic Repository Discovery: Scans a directory tree for Git repositories
• Multi-Provider Support: Works with GitHub, GitLab, Bitbucket, and any Git provider
• Branch Management: Lists and manages multiple remote branches
• Interactive & Batch Modes: Choose between interactive selection or automated processing
• Persistent Interactive Session: After each operation, returns to the main menu until you quit with q
• Parallel Execution: Process multiple repositories concurrently for faster processing
• Safe Operations: Dry-run mode for testing before execution
• Local Changes Handling: Automatic stash/restore for repositories with uncommitted changes
• Flexible Remotes: Support for custom remote names (origin, upstream, etc.)
• Comprehensive Logging: Console output, file logging, and debug verbosity
• Comprehensive Test Suite: Unit tests, CLI tests, and edge case coverage

Requirements

• Python 3.6+: Core requirement for running the script
• Git: Must be installed and accessible from command line
• click: CLI framework for argument parsing (automatically installed)
• pytest: Testing framework (optional, for running tests)

Installation

Quick Start

# Clone or download the repository
cd repomanager

# Create a virtual environment (recommended)
python -m venv .venv

# Activate the virtual environment
# On Windows:
.venv\Scripts\activate
# On Linux/macOS:
source .venv/bin/activate

# Install dependencies
pip install -r requirements.txt

Manual Setup without Virtual Environment

# Install only required dependencies
pip install click

Usage

Basic Usage

Run the script from the directory containing your Git repositories:

# Interactive mode (default) - select repositories and branches manually
python repoManager.py

# Batch mode - process all repositories and branches automatically
python repoManager.py --all

# Dry-run simulation without making changes
python repoManager.py --all --dry-run

In interactive mode, the script does not exit after a single pull: it returns to the main repository menu so you can continue with other operations. Use q to exit.

Command-Line Options

General Options

--root <path>

Specify the root directory to scan for repositories (default: current directory)

# Scan repositories in a specific directory
python repoManager.py --root /path/to/repos

# Example with batch mode
python repoManager.py --root ~/projects --all

--remote <name>

Specify the remote to use for fetch and pull operations (default: origin)

# Use upstream remote instead of origin
python repoManager.py --remote upstream

# Combine with batch mode
python repoManager.py --all --remote upstream

Execution Mode Options

--all

Enable batch/non-interactive mode. Processes all discovered repositories and their branches without prompting for user input.

# Process all repositories silently
python repoManager.py --all

# Useful for automated workflows
python repoManager.py --all --root /workspace/repos

--dry-run

Simulate operations without executing destructive Git commands (checkout/pull). Safe for testing configurations and workflows.

# Preview what would happen without making changes
python repoManager.py --all --dry-run

# Dry-run with verbose output to see details
python repoManager.py --all --dry-run --verbose

--yes / -y

Automatically answer "yes" to all interactive prompts. Useful when you want to proceed without confirmations.

# Skip confirmation prompts
python repoManager.py --yes

# Combine with local changes handling
python repoManager.py --all --yes --stash

Parallel Execution

--parallel <n>

Execute repository processing in parallel using N worker threads (default: 1, sequential execution). In batch mode, each repository is handled independently to avoid concurrent operations on the same working tree.

# Process 4 repositories in parallel
python repoManager.py --all --parallel 4

# Maximum parallelism (careful with system resources)
python repoManager.py --all --parallel 8

Note: Actual performance depends on network bandwidth and system resources.

--git-timeout <seconds>

Set timeout (seconds) for each git command before failing/retrying.

python repoManager.py --all --git-timeout 90

--git-retries <n>

Retry transient network git failures up to N times.

python repoManager.py --all --git-retries 2

--summary-file <path>

Write a structured JSON summary of every processed repository/branch operation.

python repoManager.py --all --summary-file logs/summary.json

Local Changes Management

--stash

Automatically handle uncommitted local changes by stashing them before checkout and restoring after pull. Ensures clean working directory for operations.

# Auto-stash before operations
python repoManager.py --all --stash

# Combine with parallel execution
python repoManager.py --all --stash --parallel 4

How it works:

1. Detects uncommitted changes with git status --porcelain
2. Creates a stash with git stash create
3. Performs checkout and pull
4. Restores stash with git stash pop

--stash-keep

When used with --stash, preserves the created stash instead of automatically restoring it. Useful when you want to review changes manually.

# Stash changes but keep them for manual inspection
python repoManager.py --all --stash --stash-keep

# Combined with assume-yes for unattended execution
python repoManager.py --all --stash --stash-keep --yes

Logging Options

--log-file <path>

Save all log output to a file in addition to console output. Useful for record-keeping and debugging.

# Save logs to file
python repoManager.py --all --log-file logs/pull.log

# Combine with verbose for detailed logging
python repoManager.py --all --verbose --log-file logs/debug.log

File locations are created automatically if they don't exist.

--verbose

Enable verbose output with DEBUG-level logging. Shows detailed information about script operations.

# See detailed debug information
python repoManager.py --all --verbose

# Combine with file logging for full records
python repoManager.py --all --verbose --log-file logs/full.log

Usage Examples

Example 1: Interactive Mode

python repoManager.py

What happens:

• Scans current directory for repositories
• Shows list of found repositories
• Prompts user to select a repository
• Shows available remote branches
• Prompts user to select a branch
• Performs selected pull
• Returns to the main repository menu for more operations
• Exits only when user enters q

Example 2: Batch Mode with All Defaults

python repoManager.py --all

What happens:

• Scans current directory
• Pulls all branches from all repositories
• Uses 'origin' remote
• Processes sequentially (no parallelism)

Example 3: Parallel Batch Mode

python repoManager.py --all --parallel 4

What happens:

• Scans current directory
• Processes up to 4 repositories in parallel
• Significantly faster for large numbers of repos

Example 4: Custom Root with Specific Remote

python repoManager.py --root ~/projects --all --remote upstream

What happens:

• Scans ~/projects for repositories
• Processes all in batch mode
• Uses upstream remote instead of origin

Example 5: Safe Testing with Dry-Run

python repoManager.py --all --dry-run --verbose

What happens:

• Scans repositories
• Shows what would be executed
• Doesn't perform any actual git operations
• Useful for testing configuration

Example 6: Handle Local Changes Automatically

python repoManager.py --all --stash

What happens:

• Automatically stashes uncommitted changes
• Performs pull operations
• Restores stashed changes
• If stash pop conflicts occur, manual resolution is still required

Example 7: Safe Batch with Logging

python repoManager.py --all --stash --parallel 4 --log-file logs/sync.log --summary-file logs/summary.json

What happens:

• Scans and pulls all repositories
• Handles local changes automatically
• Executes repository processing in parallel
• Saves detailed logs to logs/sync.log
• Saves structured results to logs/summary.json

Example 8: Production-Ready Setup

python repoManager.py --all --stash --stash-keep --yes --parallel 4 \
  --git-timeout 90 --git-retries 2 \
  --log-file logs/production.log --summary-file logs/summary.json --verbose

What happens:

• Runs fully unattended (no prompts)
• Preserves stashed changes for review
• Fast parallel execution
• Comprehensive logging for auditing
• Structured output for automation/reporting

Project Structure

repomanager/
├── repoManager.py              # Main script with GitRepoManager class
├── requirements.txt            # Python dependencies
├── README.md                   # This documentation file
├── .gitignore                 # Git ignore rules
├── .github/
│   ├── workflows/             # CI/CD workflows
│   │   ├── ci.yml            # Continuous integration tests
│   │   ├── workflow-main.yml  # Production release workflow
│   │   ├── workflow-develop.yml # Development release workflow
│   │   └── workflow-dependabot-automerge.yml
│   └── dependabot.yml         # Dependabot configuration
└── tests/                      # Test suite
    ├── test_repo_manager.py    # Unit tests for core functionality
    ├── test_cli.py            # CLI argument tests
    └── test_edge_cases.py     # Edge cases and error scenarios

Running Tests

Test Suite Overview

The project includes comprehensive tests covering:

• Unit tests: Core GitRepoManager functionality
• CLI tests: Command-line argument handling
• Edge cases: Error conditions and special scenarios

Running Tests (commands)

# Run all tests
pytest

# Run tests with verbose output
pytest -v

# Run tests with coverage report
pytest --cov=repoManager tests/

# Run specific test file
pytest tests/test_repo_manager.py

# Run specific test
pytest tests/test_repo_manager.py::test_find_git_repositories

# Run with full output and stop on first failure
pytest -vv -x

Test Coverage

# Generate and display coverage report
pytest --cov=repoManager --cov-report=html tests/

# Open coverage report in browser (after generating HTML)
# On Windows: start htmlcov/index.html
# On macOS: open htmlcov/index.html
# On Linux: xdg-open htmlcov/index.html

Handling Stash Conflicts

When using --stash, conflicts may occur during git stash pop. Here's how to handle them:

Conflict Scenario

If the git stash pop operation encounters conflicts:

1. The stash remains saved - You won't lose any work
2. The working tree shows conflicts - Git marks conflicted files with conflict markers
3. The script reports an error - Details are logged for troubleshooting
4. Manual resolution needed - You can resolve conflicts and retry

Resolution Steps

# 1. Check the conflicted files
git status

# 2. Edit conflicted files to resolve conflicts
# Look for markers like: <<<<<<, ======, >>>>>>

# 3. Stage the resolved files
git add <resolved-files>

# 4. Complete the stash pop
git stash drop

# 5. Commit the resolved changes
git commit -m "Resolved stash conflicts"

Preventing Conflicts

• Use --stash-keep to manually review changes before applying
• Use --dry-run first to preview what will happen
• Consider using --yes with --stash only when you're confident

Troubleshooting

Error: "git: command not found"

Problem: Git is not installed or not in your system PATH

Solutions:

# Check if git is installed
git --version

# Install git from https://git-scm.com
# Then restart your terminal

Error: "ModuleNotFoundError: No module named 'repoManager'"

Problem: Running pytest from wrong directory

Solutions:

# Run pytest from repository root
cd /path/to/repomanager
pytest tests/

# Or run with Python module syntax
python -m pytest tests/

Error: "ModuleNotFoundError: No module named 'click'"

Problem: Dependencies not installed

Solutions:

# Reinstall requirements
pip install -r requirements.txt

# Or install click directly
pip install click

# Verify installation
python -c "import click; print(click.__version__)"

Error: "PermissionError" when writing log files

Problem: No write permissions to log directory

Solutions:

# Create logs directory with proper permissions
mkdir -p logs
chmod 755 logs

# Or specify log file in current directory
python repoManager.py --all --log-file pull.log

Repositories Not Found

Problem: Script doesn't find expected repositories

Solutions:

# Verify correct root directory
python repoManager.py --root /correct/path --all --dry-run

# Check directory structure
ls -la /path/to/repos

# Ensure .git directories exist
find /path -type d -name ".git" | head -5

Script Hangs During Execution

Problem: Script seems to hang or freeze

Solutions:

# Reduce parallel workers
python repoManager.py --all --parallel 2

# Add verbose output to see progress
python repoManager.py --all --verbose

# Try with a smaller subset
python repoManager.py --root /small/subset --all --verbose

Advanced Usage

Automated Daily Pulls

# Create a cron job (Linux/macOS)
0 9 * * * cd /path/to/repos && python /path/to/repoManager.py --all --stash \
  --log-file /path/to/logs/daily.log 2>&1

Integration with CI/CD

# Use in GitHub Actions
- name: Sync repositories
  run: |
    python repoManager.py --all --parallel 4 --stash \
      --log-file sync.log --verbose

Batch Processing Multiple Locations

# Create a wrapper script (sync-all.sh)
#!/bin/bash
for dir in ~/projects ~/work ~/personal; do
  python repoManager.py --root "$dir" --all --parallel 4
done

Performance Tips

1. Use --parallel for many repositories: Start with --parallel 4 and adjust based on system resources
2. Use --stash for unattended operation: Automatically handles local changes
3. Monitor with --log-file and --summary-file: Keep records of operations for debugging and automation
4. Test with --dry-run first: Before running on production repositories

Limitations

• Requires Git to be accessible from command line
• Does not handle complex merge conflicts automatically (requires manual resolution)
• Parallel execution may stress network bandwidth with many repos
• Authentication must be configured in Git credentials

Contributing

For bug reports, feature requests, or improvements:

1. Test your changes locally: pytest tests/
2. Ensure code quality: flake8 repoManager.py
3. Document changes in comments
4. Submit pull requests with clear descriptions

License

MIT License - See LICENSE file for details

---

Last Updated: November 2025
Version: 1.0
Python Compatibility: 3.6+