/sequins

A key/value store for serving static batch data
  1. Go 98.6%
  2. Makefile 1.3%
  3. Shell 0.1%
Go Makefile Shell
Clone or download

Clone with HTTPS

Use Git or checkout with SVN using the web URL.

Open in Desktop Download ZIP
Find file
Switch branches/tags
Nothing to show
Latest commit 8f77884 Nov 28, 2017 @jerry-stripe jerry-stripe Merge pull request #148 from stripe/jerry-symlink 
allow s3_backend to handle _SYMLINK
Permalink
Failed to load latest commit information.
backend allow s3_backend to handle _SYMLINK Nov 28, 2017
blocks Allow ingestion of partitioned Sparkey files Oct 20, 2017
doc add max_replication to config to allow for a partition replication limit Nov 16, 2017
sharding add max_replication to config to allow for a partition replication limit Nov 16, 2017
test_databases Allow ingestion of partitioned Sparkey files Oct 20, 2017
vendor Begin to integrate datadog to report metrics. Aug 11, 2017
workqueue Parallelize by individual file, rather than dataset, when downloading. Sep 13, 2017
zk Allow faster startup, by deleting ZK nodes in parallel Nov 1, 2017
.gitignore Ignore intellij projects and sequins configs. Aug 16, 2017
.travis.yml update zk url as the version we use got moved to archives Nov 8, 2017
Dockerfile bump golang version to 1.8.3 Aug 4, 2017
LICENSE.txt initial open-source release Nov 20, 2014
Makefile Run `go vet` as part of CI, and fix all violations. Aug 3, 2017
README.md Add chat to the landing page and readme Oct 26, 2016
build.go Document input formats Oct 30, 2017
cluster_test.go add max_replication to config to allow for a partition replication limit Nov 16, 2017
config.go add max_replication to config to allow for a partition replication limit Nov 16, 2017
config_test.go Add a bunch of path validations into validateConfig Aug 17, 2016
db.go emit an event when a version is set to active in a db Nov 16, 2017
debug.go Log requests to a file, for future replaying Nov 1, 2017
debug_test.go Log requests to a file, for future replaying Nov 1, 2017
hdfs_sequins_test.go refactor test source directories Oct 3, 2017
healthcheck_test.go include more info in healthcheck, return 200 if any versions available Oct 9, 2017
jenkins_build.sh Run `go vet` as part of CI, and fix all violations. Aug 3, 2017
main.go add retries to s3backend::Open function Aug 2, 2017
proxy.go Run `go vet` as part of CI, and fix all violations. Aug 3, 2017
proxy_test.go Increase sleep times in proxy tests Nov 15, 2016
s3_sequins_test.go allow s3_backend to handle _SYMLINK Nov 28, 2017
sequins.conf.example add max_replication to config to allow for a partition replication limit Nov 16, 2017
sequins.go remove /healthcheck in favor of /healthz Nov 9, 2017
sequins_test.go Log requests to a file, for future replaying Nov 1, 2017
serve.go Move error handling up in the stack Oct 13, 2017
status.go update replication status for versionStatus struct on initialization Nov 14, 2017
status.tmpl increase css width some more Nov 14, 2017
utils.go Filter out DBs and versions with invalid characters Jun 28, 2017
version.go add max_replication to config to allow for a partition replication limit Nov 16, 2017
version_mux.go Add some comments Nov 2, 2016

README.md

Sequins is a key/value database for serving static batch data.

It's well-suited to making data generated by Hadoop available to processes that need low-latency, reliable access.

It's designed to be:

  • Horizontally scalable: replicate a hot dataset over a few nodes, or partition a large one over many.
  • Reliable: serve your data without an online dependency on Hadoop or HDFS. Sequins is built to be resilient to multi-node failures.
  • Interoperable: load data from HDFS or S3 in Hadoop's SequenceFile format. Tools like Spark or Impala can also be used to generate data.
  • Accessible: fetch values with HTTP GET; no client library required.

See the manual to get started.

build Gitter