A curated list of awesome Go storage projects and libraries
Switch branches/tags
Nothing to show
Clone or download
Pull request Compare This branch is 19 commits behind gostor:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.github
CONTRIBUTING.md
LICENSE
README.md

README.md

Awesome Go Storage Awesome

A curated list of awesome Go storage projects and libraries. Inspired by awesome-go.

Contributing

Please take a quick gander at the contribution guidelines first. Thanks to all contributors; you rock!

If you see a package or project here that is no longer maintained or is not a good fit, please submit a pull request to improve this file. Thank you!

Contents

Storage Server

Storage Servers implemented in Go.

  • minio - Minio is an open source object storage server compatible with Amazon S3 APIs.
  • rclone - "rsync for cloud storage" - Google Drive, Amazon Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Cloudfile…
  • camlistore - Camlistore is your personal storage system for life: a way of storing, syncing, sharing, modelling and backing up content.
  • torus - A modern distributed storage system by CoreOS.
  • s3git - Git for Cloud Storage. Distributed Version Control for Data.
  • rook - Open, Cloud Native, and Universal Distributed Storage.
  • longhorn Longhorn is an open source persistent block storage server delivered via containers.

Key-Value Store

Key-Value Store implemented in Go.

  • consul - Distributed consistent replicated key-value store for service discovery and configuration.
  • etcd - Distributed reliable key-value store for the most critical data of a distributed system.
  • go-cache - An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.
  • biscuit - Biscuit is a multi-region HA key-value store for your AWS infrastructure secrets.
  • diskv - A disk-backed key-value store.

File System

File Systems implemented in Go.

  • git-lfs - Git extension for versioning large files.
  • seaweedfs - SeaweedFS is a simple and highly scalable distributed file system for small files.
  • fsnotify - Cross-platform file system notifications for Go.
  • goofys - A high-performance, POSIX-ish Amazon S3 file system written in Go.
  • go-systemd - Go bindings to systemd socket activation, journal, D-Bus, and unit files.
  • gcsfuse - A user-space file system for interacting with Google Cloud Storage.
  • svfs - A virtual file system over Openstack Swift built upon fuse.

Database

Databases implemented in Go.

  • BigCache - Efficient key/value cache for gigabytes of data.
  • bolt - A low-level key/value database for Go.
  • buntdb - A fast, embeddable, in-memory key/value database for Go with custom indexing and spatial support.
  • cache2go - An in-memory key:value cache which supports automatic invalidation based on timeouts.
  • cockroach - A Scalable, Geo-Replicated, Transactional Datastore
  • couchcache - A RESTful caching micro-service backed by Couchbase server.
  • dgraph - Scalable, Distributed, Low Latency, High Throughput Graph Database.
  • diskv - A home-grown disk-backed key-value store.
  • eliasdb - Dependency-free, transactional graph database with REST API, phrase search and SQL-like query language.
  • forestdb - Go bindings for ForestDB.
  • GCache - Cache library with support for expirable Cache, LFU, LRU and ARC.
  • geocache - An in-memory cache that is suitable for geolocation based applications.
  • go-cache - An in-memory key:value store/cache (similar to Memcached) library for Go, suitable for single-machine applications.
  • goleveldb - An implementation of the LevelDB key/value database in the Go.
  • groupcache - Groupcache is a caching and cache-filling library, intended as a replacement for memcached in many cases.
  • influxdb - Scalable datastore for metrics, events, and real-time analytics
  • ledisdb - Ledisdb is a high performance NoSQL like Redis based on LevelDB.
  • levigo - Levigo is a Go wrapper for LevelDB.
  • moss - Moss is a simple LSM key-value storage engine written in 100% Go.
  • piladb - Lightweight RESTful database engine based on stack data structures.
  • pREST - Serve a RESTful API from any PostgreSQL database.
  • prometheus - Monitoring system and time series database.
  • rqlite - The lightweight, distributed, relational database built on SQLite.
  • scribble - A tiny flat file JSON store.
  • tidb - TiDB is a distributed SQL database. Inspired by the design of Google F1.
  • tiedot - Your NoSQL database powered by Golang.
  • Tile38 - A geolocation DB with spatial index and realtime geofencing.

Database schema migration.

  • darwin - Database schema evolution library for Go
  • goose - Database migration tool. You can manage your database's evolution by creating incremental SQL or Go scripts.
  • gormigrate - Database schema migration helper for Gorm ORM.
  • migrate - Database migration handling in Golang support MySQL, PostgreSQL, Cassandra, and SQLite.
  • pravasan - Simple Migration tool - currently for MySQL but planning to support soon for Postgres, SQLite, MongoDB, etc.,
  • soda - Database migration, creation, ORM, etc... for MySQL, PostgreSQL, and SQLite.
  • sql-migrate - Database migration tool. Allows embedding migrations into the application using go-bindata.

Database tools.

  • go-mysql - A go toolset to handle MySQL protocol and replication.
  • go-mysql-elasticsearch - Sync your MySQL data into Elasticsearch automatically.
  • kingshard - kingshard is a high performance proxy for MySQL powered by Golang.
  • myreplication - MySql binary log replication listener. Support statement and row based replication.
  • orchestrator - MySQL replication topology manager & visualizer
  • pgweb - A web-based PostgreSQL database browser
  • vitess - vitess provides servers and tools which facilitate scaling of MySQL databases for large scale web services.

SQL query builder, libraries for building and using SQL.

  • dat - Go Postgres Data Access Toolkit
  • Dotsql - Go library that helps you keep sql files in one place and use it with ease.
  • goqu - An idiomatic SQL builder and query library.
  • igor - Abstraction layer for PostgreSQL that supports advanced functionality and uses gorm-like syntax.
  • ozzo-dbx - Powerful data retrieval methods as well as DB-agnostic query building capabilities.
  • scaneo - Generate Go code to convert database rows into arbitrary structs.
  • sqrl - SQL query builder, fork of Squirrel with improved performance.
  • Squirrel - Go library that helps you build SQL queries.
  • xo - Generate idiomatic Go code for databases based on existing schema definitions or custom queries supporting PostgreSQL, MySQL, SQLite, Oracle, and Microsoft SQL Server.

Database Drivers

Libraries for connecting and operating databases.

  • Relational Databases

    • bgc - Datastore Connectivity for BigQuery for go.
    • firebirdsql - Firebird RDBMS SQL driver for Go
    • go-adodb - Microsoft ActiveX Object DataBase driver for go that using database/sql.
    • go-bqstreamer - BigQuery fast and concurrent stream insert.
    • go-mssqldb - Microsoft MSSQL driver in go language.
    • go-oci8 - Oracle driver for go that using database/sql.
    • go-sql-driver/mysql - MySQL driver for Go.
    • go-sqlite3 - SQLite3 driver for go that using database/sql.
    • gofreetds Microsoft MSSQL driver. Go wrapper over FreeTDS.
    • pgx - PostgreSQL driver supporting features beyond those exposed by database/sql.
    • pq - Pure Go Postgres driver for database/sql.
  • NoSQL Databases

    • aerospike-client-go - Aerospike client in Go language.
    • arangolite - Lightweight golang driver for ArangoDB.
    • asc - Datastore Connectivity for Aerospike for go.
    • cayley - A graph database with support for multiple backends.
    • dsc - Datastore connectivity for SQL, NoSQL, structured files.
    • dynago - Dynago is a principle of least surprise client for DynamoDB
    • go-couchbase - Couchbase client in Go
    • go-couchdb - Yet another CouchDB HTTP API wrapper for Go
    • gocb - Official Couchbase Go SDK
    • gocql - A Go language driver for Apache Cassandra.
    • gomemcache - memcache client library for the Go programming language.
    • gorethink - Go language driver for RethinkDB
    • goriak - Go language driver for Riak KV
    • mgo - MongoDB driver for the Go language that implements a rich and well tested selection of features under a very simple API following standard Go idioms.
    • neo4j - Neo4j Rest API Bindings for Golang
    • Neo4j-GO - Neo4j REST Client in golang.
    • neoism - Neo4j client for Golang
    • redigo - Redigo is a Go client for the Redis database.
    • redis - Redis client for Golang
    • redis - A simple, powerful Redis client for Go.
    • redis - Redis-protocol compatible TCP servers/services.
  • Search and Analytic Databases

    • bleve - A modern text indexing library for go.
    • elastic - Elasticsearch client for Go.
    • elastigo - A Elasticsearch client library.
    • goes - A library to interact with Elasticsearch.
    • skizze - A probabilistic data-structures service and storage.