LFS server with multiple file stores and backing stores
Switch branches/tags
Nothing to show
Clone or download
Latest commit 9238067 Sep 14, 2016
Permalink
Failed to load latest commit information.
Godeps Added MySQL meta object store support Aug 22, 2016
log add in user service and ldap Jul 16, 2015
mgmt Added MySQL meta object store support Aug 22, 2016
scripts update start script - remove dup Sep 12, 2016
test_ldap_server removing local src Mar 7, 2016
.gitignore Adds env var config Mar 8, 2016
CONTRIBUTING.md Changes after merge (#29) Sep 12, 2016
Dockerfile updates docker and start script Sep 12, 2016
LICENSE add MIT license Aug 13, 2015
Procfile Adds env var config Mar 8, 2016
README.md updates readme Sep 14, 2016
aws_content_store.go Rebuild templates (#31) Sep 12, 2016
aws_content_store_test.go fixes a bunch of broken tests Sep 10, 2016
cassandra_meta_store.go Rebuild templates (#31) Sep 12, 2016
cassandra_meta_store_test.go fixes a bunch of broken tests Sep 10, 2016
cassandra_service.go Cassandra version selection (#30) Sep 12, 2016
common.go removing local src Mar 7, 2016
config.go Cassandra version selection (#30) Sep 12, 2016
config.ini.example Cassandra version selection (#30) Sep 12, 2016
config.ini.localdata.example add local-only example Mar 3, 2016
consumers_spec.md add in user service and ldap Jul 16, 2015
content_store.go add aws content store support Jul 31, 2015
content_store_test.go add aws content store support Jul 31, 2015
downloader.go update config template Aug 3, 2015
downloader_test.go removing local src Mar 7, 2016
errors.go Rebuild templates (#31) Sep 12, 2016
harbour_test.go fixes a bunch of broken tests Sep 10, 2016
kvlogger.go adding functionality to the base git-lfs-test server Jul 15, 2015
main.go Merge pull request #26 from unixworld/master Sep 12, 2016
meta_store.go Rebuild templates (#31) Sep 12, 2016
meta_store_auth.go removing local src Mar 7, 2016
meta_store_auth_test.go removing local src Mar 7, 2016
meta_store_test.go fix a ldap issues Aug 15, 2015
mgmt-css.rice-box.go Rebuild templates (#31) Sep 12, 2016
mgmt-js.rice-box.go Rebuild templates (#31) Sep 12, 2016
mgmt-templates.rice-box.go Rebuild templates (#31) Sep 12, 2016
mgmt.go Fix issue with add project Aug 22, 2016
mgmt_test.go fixes a bunch of broken tests Sep 10, 2016
mysql_meta_store.go Changes after merge (#29) Sep 12, 2016
mysql_meta_store_test.go Changes after merge (#29) Sep 12, 2016
mysql_service.go Added MySQL meta store test Sep 11, 2016
server.go Added MySQL meta object store support Aug 22, 2016
shared_content_store.go add aws content store support Jul 31, 2015
shared_test.go go fmt Jul 17, 2015
tracking_listener.go adding functionality to the base git-lfs-test server Jul 15, 2015
user_service.go updates scripts to use godeps Jan 21, 2016
user_service_test.go removing local src Mar 7, 2016

README.md

LFS Server Go!

NOT yet ready for primetime and is going through some iterations, specifically regarding access control.

LFS Server Go! a server that implements the Git LFS API

This is based off of lfs-test-server

  1. This server provides access to
  2. The meta store is offloaded to
  • BoltDB
  • Cassandra
  1. There is a notion of project -> OID membership, which is lacking from the original. This is wired up but still a WIP. It will allow for validating a user's membership to a project and the project's associated OID to the user, thus ensuring a user's access to a project will allow for access to an OID

##TODO:

  1. Update access
  • Rename user to namespace
  • Implement namespace and project based access
  1. Remove/Clean up old objects on delete
  2. When an object is public and AWS is enabled, offload GETs directly to AWS
  3. Adopt verification of uploads
  4. Redo the UI so it is abstracted into its own app

Installing

Use the Go installer, this will install all dependencies tracked:

  $ go install github.com/cloudmazing/lfs-server-go

To use a specific config file, set LFS_SERVER_GO_CONFIG=/path/to/config

Then start with ./scripts/start

Stop with ./scripts/stop

Running

Set your GO_ENV. Options are prod or dev or test

Running the binary will start an LFS server on localhost:8080 by default. All of the configuration settings are stored in config.ini.

You'll want to copy config.ini.example to config.ini

A running database server, if desired. One of MySQL or Cassandra are the external database options. BoltDB is the local option and is not suggested for production use

An example usage:

Generate a key pair

openssl req -x509 -sha256 -nodes -days 2100 -newkey rsa:2048 -keyout mine.key -out mine.crt

Update the config to point at your new keys

Build the server

go build

S3

If using S3, you'll need to set AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY in your environment AWS_ACCESS_KEY is your AWS access key ID AWS_SECRET_ACCESS_KEY is your AWS secret key

Example:

export AWS_ACCESS_KEY_ID=someLongString
export AWS_SECRET_ACCESS_KEY=someLongerString/withMoreStuff

Or set it in the config file via the AccessKeyId and SecretAccessKey config settings

Start it

./scripts/start

Client

Further client documentation on the client is available at https://git-lfs.github.com/

To use the LFS test server with the Git LFS client, configure it in the repository's .gitconfig file:

  [lfs]
    url = "http://localhost:8080/janedoe/lfsrepo"

This file MUST be checked into git inside of your project.

HTTPS:

NOTE: If using https with a self signed cert also disable cert checking in the client repo.

	[lfs]
		url = "https://localhost:8080/jimdoe/lfsrepo"

	[http]
		sslfverify = false

Security Design

Namespaces -> projects Users are given access to a namespace: read, write, or both Users are given access to a project: read, write, or both

Building

To build from source, use the Go tools + godep:

  $ go get github.com/cloudmazing/lfs-server-go
  $ go install ./...
  $ godep restore

Making changes

To build from source, use the Go tools:

  $ go get github.com/cloudmazing/lfs-server-go
  $ go install ./...
  <edit files>
  $ godep save ./...
  $ godep update ./...

Testing

MUST have AWS S3 credentials (public and secret keys)

MUST create a database and user in mysql used for testing:

create database lfs_server_go_test;
grant all privileges on lfs_server_go_test.* to 'lfs_server'@'localhost' identified by 'pass123';

MUST have Cassandra

brew install cassandra