A high performance replicated log service. (The development is moved to Apache Incubator)
Clone or download
sijie Blacklist branches on travis
Author: Sijie Guo <sijie@apache.org>

Reviewers: Franck Cuny <franck.cuny@gmail.com>

Closes #49 from sijie/sijie/fix_travis_build
Latest commit 8e619cd Dec 10, 2016
Permalink
Failed to load latest commit information.
dev-support DL-79: DL eclipse formatter file Dec 10, 2016
distributedlog-benchmark Update a couple of scripts Dec 3, 2016
distributedlog-client DL-80: synchronized getLogRecordSetWriter() Dec 3, 2016
distributedlog-core DL-49: Support building project with scala 2.10 and 2.11 Dec 3, 2016
distributedlog-protocol DL-32: Fix Findbug warnings Oct 13, 2016
distributedlog-service Update a couple of scripts Dec 3, 2016
distributedlog-tutorials DL-49: Support building project with scala 2.10 and 2.11 Dec 3, 2016
docker DL-1: Clean up the distributedlog pom files Aug 23, 2016
docs Add documentation for distributedlog-benchmark Nov 17, 2016
scripts DL-49: Support building project with scala 2.10 and 2.11 Dec 3, 2016
src DL-36: travis ci build failed with assembly plugin Aug 27, 2016
vagrant DL-1: Clean up the distributedlog pom files Aug 23, 2016
website Fix the build scripts Nov 9, 2016
.gitignore use dlog-daemon.sh to start writeproxy, use the newly created my_name… May 17, 2016
.travis.yml Blacklist branches on travis Dec 10, 2016
ChangeLog Initial Check-in for distributedlog oss May 9, 2016
Dockerfile DL-1: Clean up the distributedlog pom files Aug 23, 2016
LICENSE Initial Check-in for distributedlog oss May 9, 2016
NOTICE DL-4: Add NOTICE file. Jul 31, 2016
README.md The CONFIG.ini file has no purpose outside of Twitter Dec 3, 2016
Vagrantfile DL-1: Clean up the distributedlog pom files Aug 23, 2016
pom.xml DL-32: Fix Findbug warnings Oct 13, 2016

README.md

logo

Build Status Build Status Coverage Status

Apache DistributedLog (incubating)

Apache DistributedLog (DL) is a high-throughput, low-latency replicated log service, offering durability, replication and strong consistency as essentials for building reliable real-time applications.

Status

The Apache DistributedLog project is in the process of incubating. This includes the creation of project resources, the refactoring of the initial code submissions, and the formulation of project documentation, planning and the improvements of existing user and operation documents. Any feedback and contributions are welcome.

Features

High Performance

DL is able to provide milliseconds latency on durable writes with a large number of concurrent logs, and handle high volume reads and writes per second from thousands of clients.

Durable and Consistent

Messages are persisted on disk and replicated to store multiple copies to prevent data loss. They are guaranteed to be consistent among writers and readers in terms of strict ordering.

Efficient Fan-in and Fan-out

DL provides an efficient service layer that is optimized for running in a multi- tenant datacenter environment such as Mesos or Yarn. The service layer is able to support large scale writes (fan-in) and reads (fan-out).

Various Workloads

DL supports various workloads from latency-sensitive online transaction processing (OLTP) applications (e.g. WAL for distributed database and in-memory replicated state machines), real-time stream ingestion and computing, to analytical processing.

Multi Tenant

To support a large number of logs for multi-tenants, DL is designed for I/O isolation in real-world workloads.

Layered Architecture

DL has a modern layered architecture design, which separates the stateless service tier from the stateful storage tier. To support large scale writes (fan- in) and reads (fan-out), DL allows scaling storage independent of scaling CPU and memory.

First Steps

Next Steps

Get In Touch

Report a Bug

For filing bugs, suggesting improvements, or requesting new features, help us out by opening a jira.

Need Help?

Subscribe or mail the dev@distributedlog.incubator.apache.org list - Ask questions, find answers, join developement discussions and also help other users.

Contributing

We feel that a welcoming open community is important and welcome contributions.

Contributing Code

  1. See Developer Guide to get your local environment setup.

  2. Take a look at our open issues.

  3. Review our coding style and follow our code reviews to learn about our conventions.

  4. Make your changes according to our code review workflow.

  5. Checkout the list of project ideas to contribute more features or improvements.

Improving Website and Documentation

  1. See website/README.md on how to build the website.

  2. See docs/README.md on how to build the documentation.

About

Apache DistributedLog is an open source project of The Apache Software Foundation (ASF). The Apache DistributedLog project originated from Twitter.