Skip to content
master
Switch branches/tags
Code

Latest commit

… for very large tablets

Summary:
Renames the flag controlling the terminal tablet splitting threshold to more clearly delineate its purpose as a forced catch-all for tablets that grow unchecked after both split phases have been passed. The purpose of FLAGS_tablet_force_split_threshold_bytes is to prevent massive "whale tablets" from growing in the cluster. This threshold is meant to be set fairly high relative to the expected tablet size in the cluster.

Also rename tablet_split_low_phase_tablet_count_per_node/tablet_split_high_phase_tablet_count_per_node to use the term "shard" instead, which is more well-defined. Whereas tablet could refer to tablet peer (replica) or logical tablet (shard, from the table's perspective), using the term "shard" more unambiguously identifies our intended semantics.

Test Plan: Jenkins

Reviewers: timur, rao, bogdan

Reviewed By: bogdan

Subscribers: kannan, ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D11965
cd888d7

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time

YugabyteDB


License Documentation Status Ask in forum Slack chat Analytics

What is YugabyteDB?

YugabyteDB is a high-performance, cloud-native distributed SQL database that aims to support all PostgreSQL features. It is best to fit for cloud-native OLTP (i.e. real-time, business-critical) applications that need absolute data correctness and require at least one of the following: scalability, high tolerance to failures, or globally-distributed deployments.

The core features of YugabyteDB include:

  • Powerful RDBMS capabilities Yugabyte SQL (YSQL for short) reuses the query layer of PostgreSQL (similar to Amazon Aurora PostgreSQL), thereby supporting most of its features (datatypes, queries, expressions, operators and functions, stored procedures, triggers, extensions, etc). Here is a detailed list of features currently supported by YSQL.

  • Distributed transactions The transaction design is based on the Google Spanner architecture. Strong consistency of writes is achieved by using Raft consensus for replication and cluster-wide distributed ACID transactions using hybrid logical clocks. Snapshot and serializable isolation levels are supported. Reads (queries) have strong consistency by default, but can be tuned dynamically to read from followers and read-replicas.

  • Continuous availability YugabyteDB is extremely resilient to common outages with native failover and repair. YugabyteDB can be configured to tolerate disk, node, zone, region, and cloud failures automatically. For a typical deployment where a YugabyteDB cluster is deployed in one region across multiple zones on a public cloud, the RPO is 0 (meaning no data is lost on failure) and the RTO is 3 seconds (meaning the data being served by the failed node is available in 3 seconds).

  • Horizontal scalability Scaling a YugabyteDB cluster to achieve more IOPS or data storage is as simple as adding nodes to the cluster.

  • Geo-distributed, multi-cloud YugabyteDB can be deployed in public clouds and natively inside Kubernetes. It supports deployments that span three or more fault domains, such as multi-zone, multi-region, and multi-cloud deployments. It also supports xCluster asynchronous replication with unidirectional master-slave and bidirectional multi-master configurations that can be leveraged in two-region deployments. To serve (stale) data with low latencies, read replicas are also a supported feature.

  • Multi API design The query layer of YugabyteDB is built to be extensible. Currently, YugabyteDB supports two distributed SQL APIs: Yugabyte SQL (YSQL), a fully relational API that re-uses query layer of PostgreSQL, and Yugabyte Cloud QL (YCQL), a semi-relational SQL-like API with documents/indexing support with Apache Cassandra QL roots.

  • 100% open source YugabyteDB is fully open-source under the Apache 2.0 license. The open-source version has powerful enterprise features such as distributed backups, encryption of data-at-rest, in-flight TLS encryption, change data capture, read replicas, and more.

Read more about YugabyteDB in our Docs.

Get Started

Cannot find what you are looking for? Have a question? Please post your questions or comments on our Community Slack or Forum.

Build Apps

YugabyteDB supports several languages and client drivers. Below is a brief list.

Language ORM YSQL Drivers YCQL Drivers
Java Spring/Hibernate PostgreSQL JDBC cassandra-driver-core-yb
Go Gorm pq gocql
NodeJS Sequelize pg cassandra-driver
Python SQLAlchemy psycopg2 yb-cassandra-driver
Ruby ActiveRecord pg yugabyte-ycql-driver
C# EntityFramework npgsql CassandraCSharpDriver
C++ Not tested libpqxx cassandra-cpp-driver
C Not tested libpq Not tested

What's being worked on?

This section was last updated in June, 2021.

Current roadmap

Here is a list of some of the key features being worked on for the upcoming releases (the YugabyteDB v2.7 latest release has been released in May, 2021, and the v2.4 stable release was released in Jan 2021).

Feature Status Release Target Progress Comments
Automatic tablet splitting enabled by default PROGRESS v2.6, v2.7 Track
Point in time Recovery PROGRESS v2.6, v2.7 Track
YSQL: table statistics and Cost based optimizer(CBO) PROGRESS v2.7 Track
[YSQL] Support GIN indexes PROGRESS v2.7 Track
[YSQL] Feature support - ALTER TABLE PROGRESS v2.6, v2.7 Track
[YSQL] Online schema migration PROGRESS v2.7 Track
[YSQL] Collation Support - support COLLATE option for text-based columns PROGRESS v2.7 Track
[YSQL] Support SAVEPOINT - Transaction Control Commands PROGRESS v2.7 Track
[YCQL] LDAP Support for Yugabyte's YCQL API PROGRESS v2.7 Track
Incorporate PostgreSQL 12 features PLANNING v2.7 Track
Improving day 2 operations of Yugabyte Platform PROGRESS v2.5 Track
Row-level geo-partitioning PROGRESS v2.7 Track Enhance YSQL language support
Improve TPC-C benchmarking PROGRESS v2.7 Track
Transparently restart transactions PROGRESS v2.5 Track Decrease the incidence of transaction restart errors seen in various scenarios
Pessimistic locking Design PROGRESS v2.7 Track
Support Liquibase, Flyway, ORM schema migrations PROGRESS v2.7

Planned additions to the roadmap

The following items are being planned as additions to the roadmap.

Feature Status Release Target Progress Comments
Support pgloader to migrate from MySQL PLANNING Track
Make COLOCATED tables default for YSQL PLANNING Track
Support Kafka as source and sink PLANNING Support source and sink for both YSQL and YCQL
Support for transactions in async xCluster replication PLANNING Track Apply transactions atomically on consumer cluster.

Recently released features

Feature Status Release Target Docs / Enhancements Comments
Support Spark 3.x on YCQL API DONE v2.5, v2.6 Track
Support ALTER TABLE add primary key PROGRESS v2.6, v2.7 Track
Identity and access management in YSQL DONE v2.5 Track LDAP and Active Directory support
Follower reads in YSQL BETA v2.5 Issue Ability to perform follower reads for YSQL and transactional tables in YCQL.
YSQL cluster administration features - Node-Level statistics DONE v2.5 Issue Per-node view of currently active queries, find which queries are slow, what active connections are doing, etc.
Support loading large data sets into YSQL using COPY DONE v2.5 Issue Improving transactions which have a very large number of operations, as well as provide various options to batch load data more efficiently
Database runtime activity monitoring DONE v2.5 Issue Activity monitoring, audit logging, inactivity monitoring
Online rebuild of indexes DONE v2.2 Docs coming soon. See pending enhancements
DEFERRED constraints in YSQL DONE v2.2 Docs coming soon. See pending enhancements.
COLOCATED tables GA DONE v2.2 Docs coming soon
Online schema migration framework DONE v2.2 Note that this is just the framework implementation. See planned enhancements in this area.
Distributed backups for transactional tables DONE v2.2 Docs coming soon. See pending enhancements.
IPV6 support for YugabyteDB DONE v2.2 Docs coming soon
Automatic tablet splitting BETA v2.2 Docs See further enhancements
Change data capture BETA This feature is currently available but in beta.
xCluster replication (async cross-cluster replication) DONE v2.1 Docs
Encryption of data at rest DONE v2.1 Docs

Architecture

YugabyteDB Architecture

Review detailed architecture in our Docs.

Need Help?

Contribute

As an an open-source project with a strong focus on the user community, we welcome contributions as GitHub pull requests. See our Contributor Guides to get going. Discussions and RFCs for features happen on the design discussions section of our Forum.

License

Source code in this repository is variously licensed under the Apache License 2.0 and the Polyform Free Trial License 1.0.0. A copy of each license can be found in the licenses directory.

The build produces two sets of binaries:

  • The entire database with all its features (including the enterprise ones) are licensed under the Apache License 2.0
  • The binaries that contain -managed in the artifact and help run a managed service are licensed under the Polyform Free Trial License 1.0.0.

By default, the build options generate only the Apache License 2.0 binaries.

Read More