GitClone

An educational, first-principles reimplementation of core Git internals in modern C++20.

Overview

GitClone is a ground-up recreation of the foundational components of Git. It was built with the singular goal of deeply understanding version control internals, content-addressable storage, and distributed system architectures.

Rather than being a conceptual mockup, GitClone is designed for strict binary interoperability with the official Git executable. A repository created by GitClone can be seamlessly used by native Git, and vice versa. It is built in C++20, emphasizing rigorous systems engineering, correct abstraction boundaries, and modern language features over development speed.

Note: GitClone is an educational exploration. It is not intended to replace Git, nor does it support Git's complete surface area or network protocols.

Engineering Highlights

• Content-Addressable Storage: Full loose object database (.git/objects) implementation handling SHA-1 identity and zlib compression.
• Git Object Parsing: Strict parsing and serialization of blob, tree, and commit binary formats.
• Binary Git Index (V2): Custom parser/serializer for the .git/index staging area, compliant with Git's V2 8-byte alignment constraints.
• Commit Graph & Merging: True DAG-based branch history with recursive three-way merge logic and two-parent merge commits.
• Packfile Reading: Support for unpacking, delta resolution, and traversing packed repository histories.
• Native Git Interoperability: Validated index formats, commit signatures, and tree topologies against native Git.
• Modern C++20: Extensive use of std::filesystem, std::span, std::byte, and modern resource management (RAII).
• Error Handling: Leverages tl::expected for monadic, exception-free error propagation.
• Robust Quality Assurance: Over 240 assertions across 15 integration and unit test suites, continuously verified against Clang AddressSanitizer (ASAN) and UndefinedBehaviorSanitizer (UBSAN).

Architecture

GitClone utilizes a strictly layered architecture to separate CLI parsing, domain logic, and core infrastructure:

graph TD
    CLI[CLI Dispatcher] --> Domain
    
    subgraph Domain [Domain Subsystems]
        Repo[Repository / Config]
        Index[Index / Staging]
        Refs[References / HEAD]
        ODB[Object Database]
        DiffMerge[Diff & Merge Engine]
    end
    
    Domain --> Core
    
    subgraph Core [Core Infrastructure]
        Crypto[Crypto / SHA-1]
        Compress[Zlib Deflate/Inflate]
        IO[Binary I/O]
    end

Loading

The CLI primarily interprets arguments and coordinates domain operations. The domain layer enforces structural invariants (e.g., repository boundaries, index formatting) while delegating to the core infrastructure layer for low-level cryptographic hashing and filesystem manipulation.

How GitClone Works

The version control pipeline maps directly to Git's core internal design:

1. Working Tree: The user modifies files on the filesystem.
2. Index: Using gitclone add, the file content is hashed, compressed, and written to the Object Database as a Blob. The .git/index binary file is updated to map the file path to its SHA-1 hash, alongside its filesystem mtime and size.
3. Tree: gitclone write-tree converts the flat Index into a hierarchical tree of Tree objects, recursively writing them to the Object Database.
4. Commit: gitclone commit generates a Commit object pointing to the root Tree and the current parent Commit, securing the history snapshot.
5. References: The .git/refs/heads/<branch> file is advanced to point to the new Commit SHA-1, moving the branch tip forward.

All objects reside in the Object Database (.git/objects). To optimize space and performance, Git packs loose objects into Packfiles, which GitClone's PackReader subsystem can read and resolve using delta decompression.

Supported Commands

GitClone currently supports a focused subset of plumbing and porcelain commands:

• gitclone init: Initializes an empty repository structure.
• gitclone hash-object [-w] [--stdin] <file>: Computes the SHA-1 of a file. The -w flag writes it as a blob to the object database.
• gitclone cat-file (-p|-t|-s) <sha1>: Inspects an object in the database (pretty-print, type, or size).
• gitclone add <file>: Hashes a file and stages it into the index.
• gitclone ls-files [-s]: Lists files currently tracked in the index.
• gitclone status: Identifies modified, staged, and untracked files by comparing the working tree against the index. Utilizes a stat-cache for fast comparisons.
• gitclone diff: Computes line-level unified diffs between the working tree and the index.
• gitclone write-tree: Serializes the current index into a hierarchy of tree objects.
• gitclone commit -m "<message>": Creates a commit object linking to the current tree and parent history.
• gitclone branch <name>: Creates a new branch reference.
• gitclone log: Traverses the commit graph and prints the history.
• gitclone merge <branch>: Performs a three-way merge between the current branch and the target branch. Handles fast-forwards and divergent branches, generating merge commits or conflict markers as appropriate.

Git Compatibility

GitClone is continuously tested against the actual git executable to guarantee interoperability:

• Repositories created by gitclone init are instantly recognized by git status.
• Blobs, trees, and commits written by gitclone can be inspected by native git cat-file.
• Binary indexes generated by gitclone add are verified by git ls-files --stage.
• Merge commits produced by gitclone merge are traversed seamlessly by native git log.
• Packfiles generated by native git gc are seamlessly read and unpacked by gitclone log.

Testing

GitClone uses the Catch2 framework for exhaustive quality assurance:

• Unit Tests: Granular coverage for cryptographic hashing, compression, and object deserialization.
• Integration Tests: End-to-end command execution utilizing real temporary directories for filesystem fidelity.
• Compatibility Tests: CLI integration tests directly invoke the system git binary to independently verify GitClone's output logic (e.g. comparing gitclone merge behavior against native git log).
• Memory Safety: The entire 15-suite test harness compiles and runs successfully under Clang's AddressSanitizer (ASAN) and UndefinedBehaviorSanitizer (UBSAN).

Build & Installation

Prerequisites

• CMake 3.20+
• A C++20 compliant compiler (GCC 10+, Clang 11+, Apple Clang 13+)
• Zlib (zlib1g-dev on Linux)

Build Instructions

# 1. Clone the repository
git clone https://github.com/yourusername/gitclone.git
cd gitclone

# 2. Configure with CMake
mkdir build && cd build
cmake ..

# 3. Compile
cmake --build .

# 4. Run the test suite
./gitclone_tests

# 5. Use the CLI
./gitclone init

Sanitizers

To build with memory and undefined behavior sanitizers enabled, simply toggle the CMake option:

cmake -DENABLE_SANITIZERS=ON ..
cmake --build .
./gitclone_tests

Project Structure

• src/: Implementation of core logic, domain subsystems, and CLI commands.
• include/gitclone/: Public API headers for subsystem boundaries.
• tests/: Catch2 unit and integration test suites.
• docs/: Extensive project documentation, architecture diagrams, and theoretical workbooks.
• external/: Bundled dependencies (tl::expected, TinySHA1, picosha2).

Engineering Decisions

• C++20: Chosen for its robust memory management (std::span, std::unique_ptr), native filesystem API (std::filesystem), and byte-level manipulation (std::byte), which are critical for parsing binary object formats safely.
• tl::expected: Used universally for error handling to avoid the overhead and unpredictable control flow of C++ exceptions, ensuring explicit and monadic error propagation across subsystem boundaries.
• Native Filesystem IO: We deliberately rejected virtual filesystem abstractions (e.g. mocking in memory). Given the heavy reliance on POSIX filesystem semantics (stat, permissions, inode data), running tests on actual temporary directories provides significantly higher fidelity.
• Strict Architecture Boundaries: The CLI layer is forbidden from accessing the filesystem directly (core::io); all environment interactions must route through the repo::Repository domain layer to guarantee environment awareness (e.g. correctly isolating .git/).

What I Learned

Implementing Git from first principles was a profound exercise in systems design:

• Merkle Trees and DAGs: Deepened my understanding of how cryptographic hashes intrinsically secure hierarchical data and branch histories.
• Binary Format Parsing: Managing 8-byte alignment padding in the .git/index format and navigating complex delta instructions within Packfiles significantly sharpened my skills in memory-safe binary traversal.
• Three-Way Merging: Implementing divergent branch merging exposed the complexities of Least Common Ancestor (LCA) algorithms and the necessity of handling file-level conflicts robustly.
• Interoperability: Reverse-engineering an undocumented spec (by testing against the reference implementation) demonstrated the value of strict, cross-binary integration tests.

AI-Assisted Development

GitClone was developed with substantial assistance from AI coding agents. AI was used as an implementation and engineering-assistance tool for code generation, debugging, documentation, test development, and iterative review.

However, the core project architecture, subsystem boundaries, engineering requirements, milestones, and rigorous verification criteria were explicitly defined and directed by the author. AI-generated implementations were heavily scrutinized, reviewed, and iteratively corrected rather than blindly accepted.

During development, the project underwent comprehensive independent engineering audits. These audits successfully identified real, subtle defects—such as index serialization padding alignment, CLI architecture violations, and incomplete divergent merge handling—despite the project's own internal tests passing.

AI-assisted does not mean AI-verified. The most critical engineering lesson of this project was that passing a project's own internal test suite does not establish correctness. Real-world correctness was ultimately established through rigorous native Git interoperability checks, independent audits, regression testing, and sanitizer validation.

Limitations

GitClone is an educational subset of Git. It intentionally omits:

• Network protocols (clone, fetch, push, HTTP/SSH transports)
• Advanced merging strategies (e.g., recursive rename detection, octopus merges)
• Packfile writing (it can read packs, but does not pack loose objects via gc)
• Advanced reference management (e.g., packed-refs)
• Advanced config parsing (.gitconfig rules)

Future Work

• SHA-256 Support: Adapting the Object Database to support Git's newer object format transition.
• Packed-Refs: Implementing support for reading and writing compressed reference lists.
• Network Protocol Foundations: Exploring the implementation of a basic git-upload-pack client to enable remote fetching.

Documentation

GitClone contains extensive internal documentation detailing its theory, design, and execution.

• Architecture Guide: Detailed component interactions.
• Git Internals Guide: An educational dive into Git's object model and binary structures.
• Project State & Milestones: The living roadmap of the project's development.

License

This project is open-source and available under the standard MIT License.