Skip to content
Branch: master
Find file History
Latest commit daa0ba5 May 6, 2019
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
query Initial commit Apr 4, 2019
README.md Initial commit Apr 4, 2019
booting_all_the_views.png Initial commit Apr 4, 2019
booting_beam_views.md Initial commit Apr 4, 2019
booting_carousel.png Initial commit Apr 4, 2019
booting_conductor.png Initial commit Apr 4, 2019
booting_consumers.png Initial commit Apr 4, 2019
booting_graph.png Initial commit Apr 4, 2019
booting_partitions_4_6.png Initial commit Apr 4, 2019
booting_partitions_8_4.png Initial commit Apr 4, 2019
booting_partitions_all.png Initial commit Apr 4, 2019
central_log_arch.md Initial commit Apr 4, 2019
central_log_arch.png Initial commit Apr 4, 2019
control_plane.md Initial commit Apr 4, 2019
ha_dr_beam.png Initial commit Apr 4, 2019
high_availability.md Initial commit Apr 4, 2019
kafka.md
kafka_fetch_seq.png
kafka_fsync.png
kafka_no_fsync.png Initial commit Apr 4, 2019
kafka_read_seq.png Initial commit Apr 4, 2019
protobeam_v1.md Initial commit Apr 4, 2019
protobeam_v2.md Initial commit Apr 4, 2019
protobeam_v3.md Initial commit Apr 4, 2019
query.md Update query.md May 6, 2019
rfc_update.md Initial commit Apr 4, 2019
rfc_update_plan.png Initial commit Apr 4, 2019
v1_arch.png Initial commit Apr 4, 2019
v1_kv.png
v1_tx.png Initial commit Apr 4, 2019
v2_components.png Initial commit Apr 4, 2019
v3_components.png Initial commit Apr 4, 2019
v3_exec_join.png Initial commit Apr 4, 2019
v3_exec_loop.png Initial commit Apr 4, 2019
v3_fact_as_edge.png Initial commit Apr 4, 2019
v3_hash_3_4.png Initial commit Apr 4, 2019
v3_hashplan.png Initial commit Apr 4, 2019
v3_plan_final.png
v3_plan_space.png Initial commit Apr 4, 2019
v3_query.png Initial commit Apr 4, 2019
v3_tree.png Initial commit Apr 4, 2019

README.md

This directory contains point-in-time snapshots of what we thought about a certain topic at a certain time. While many of these documents build on each other, it should be possible to read these in isolation and to approach them in any order. We suggest reviewing this chronological list first, then jumping around based on your interests.

  • ProtoBeam v1 (November 2017): This document introduces Beam's basic architecture, which is based around a central log. It describes the expected benefits and challenges of a central log, as well as Beam's approach to distributed transactions.

  • ProtoBeam v2 (March 2018): This document summarized the state of the system back then as a property graph store. It also includes some discussion of storing data in RocksDB compared to other embedded key-value stores.

  • Booting Beam Views (March 2018): Part of the ProtoBeam v2 series, this document describes how view servers in Beam start up once the oldest part of the log has been discarded.

  • Using Apache Kafka in Beam (March 2018): Part of the ProtoBeam v2 series, this document describes challenges with using Kafka as Beam's log, and why it might not be a great fit.

  • ProtoBeam v3 (August 2018): This document summarized the state of the system back then, after adapting it from a property graph store to a knowledge graph store. It describes the overall system components (which are still accurate) and the query language at that time (which has evolved significantly since; see the Query doc below). It includes details on the query processor, including the parser, cost-based query planner, parallel query execution engine, and RPC fanout mechanism.

  • Control Plane Requirements (September 2018): This document explores the various scenarios that a control plane (or manual operator) will need to handle for a cross-datacenter production Beam deployment.

  • Building Feature-Rich Data Stores Using a Central Log (January 2019): This short document argues that a central log forms the basis of an architecture that is well-suited for systems like Beam.

  • High Availability and Disaster Recovery (March 2019): This short document explains how fault-tolerance is built into Beam's architecture.

  • Update RFC (March 2019): This long document proposes a new mechanism for inserting, deleting, and updating facts in Beam, based on the SPARQL Update specification. Only part of this RFC has been implemented at this time.

  • Query (March 2019): This document describes Beam's current query language, which is a cross between SPARQL and the version described in the ProtoBeam v3 doc.

You can’t perform that action at this time.