go-vcsfetch

vcs fetcher and cloner for Go.

A Go library for fetching files from version control systems (vcs).

---

Easily retrieve individual files or repositories over a vcs location.

• Support git repositories
• Support SPDX Locators (spdx downloadLocation attribute)
• Support common git-url schemes

All fetched resources are exposed for read-only operations only.

If you're looking for general purpose vcs support in Go for read/write or other git-heavy operations, consider using github.com/go-git/go-git instead.

Status

Work in progress. Unreleased.

Use-cases

• retrieve a single file over a remote repo (e.g. config file)
• retrieve an entire folder at a specific version
• ...

Not intended to work with local resources (e.g. file://...).

Features

VCS (Version Control System)

• Works without git installed
• Supported schemes: http, https, ssh, git TCP
• Authentication (basic, ssh)
• Fetch (single file) or Clone (folder or entire repo)
• Fetch optimized for common SCMs (github.com, gitlab), with https raw content download to bypass pure-git operations
• In memory or filesystem-backed
• Supports sparse-cloning
• Auto-detects the presence of the git binary for faster fetching using the git command line

Resolving versions

• Ref as commit sha, branch or tag, with exact match
• Semver tag resolution with incomplete semver: e.g. resolve v2 as the latest tag <v3, and 2.1 as the latest tag <v2.2

SCM-specific URLs

• git-url parses resource locators for well-known schemes
  • azure
  • bitbucket
  • gitea
  • github
  • gitlab
• know how to transform a resource locator into a raw-content URL

Quick start

go get github.com/fredbi/go-vcsfetch

Usage

Basic usage

import (
    "bytes"
    "context"
    "log"

    "github.com/fredbi/go-vcsfetch"
)

...

vf := vcsfetch.NewFetcher()
w := new(bytes.Buffer)
ctx := context.Background()

const spdxDownloadLocation = "https://github.com/fredbi/go-vcsfetch@HEAD#.golangci.yml"

if err := vf.Fetch(ctx, w, spdxDownloadLocation); err != nil {
    ...
}

log.Println(w.String())

Advanced usage with options

Example use cases:

1. authentication
2. git over TCP
3. repo cloning
4. git-urls
5. folder retrieval and repeated fetches
6. exact vs semver tag resolution
7. using shorthand slugs
8. git-archive (with benchmark)
9. TLS settings

Take a tour of the .

Dependencies

This library is built on top of github.com/go-git/go-git, a pure Go git implementation. It does not require runtime dependencies (e.g. like when using go-git bindings from git2go).

It does not require the git binary to be installed.

However, when the git binary is present and auto-detection is not disabled, the library may chose to perform some operations using the native git implementation, which is usually faster than the native go port.

Documentation

.

Resource usage and performances

TODO

Roadmap

• Support for git-archive download, when well-known SCM will start support this protocol
• Support for mercurial, with a runtime dependency on hg.
• native go git-archive support (or from go-git/v6?)
• support semver version constraint such as ^v1.2.3 or ~v1.2.3
• mock git server

License

This library is distributed under the Apache 2.0 license.

SPDX-FileCopyrightText: Copyright 2025 Frédéric BIDON SPDX-License-Identifier: Apache-2.0

Credits and acknowledgments

Initially, my intent was to enable a shared golangci-lint config file on a repository common to all repos within an organization.

Doing a little research on how to work with vcs resources in Go, I stumbled over this little package github.com/carabiner-dev/vcslocator.

I started to use it right away, but was quickly hindered by a number of limitations. My requirements departed quite a bit from that implementation, to the point that forking wasn't an option. And so started this implementation.

Thank you to the guys at carabiner-dev, who provided me the inspiration to use a SPDX locator on top of go-git.

Notice that this implementation is 100% original code and not a plagiarism of the above.