MySQL fork maintained and used at Twitter
Pull request Compare This branch is 2272 commits behind twitter:master.
Permalink
Failed to load latest commit information.
.bzr-mysql post-merge fix Sep 21, 2011
BUILD Updated/added copyright headers Jun 30, 2011
Docs WL#5665: Removal of the autotools-based build system Nov 20, 2010
client Merge commit 'tags/clone-5.5.21-build' into twitter-dev Feb 6, 2012
cmake MYSQL-28: Produce installable dmg+pkg builds for Mac OS X Jan 11, 2012
cmd-line-utils weave merge of mysql-5.1->mysql-5.5 Jan 12, 2012
dbug Bug#12779790 COMPILATION FAILS OS X 10.7 - IMPLICIT DECLARATION OF Jul 21, 2011
extra Backport patch that removes the any references to the the global Jan 10, 2012
include MYSQL-15: Implement server-side statement timeout support Mar 6, 2012
libmysql Bug#13604121 - LIBMYSQLCIENT.SO MISSING SOME FUNCTIONS PRESENT IN LIB… Jan 30, 2012
libmysqld MYSQL-15: Implement server-side statement timeout support Mar 6, 2012
libservices Updated/added copyright headers Jun 30, 2011
man Exclude NDB man pages from a source tarball, Oct 4, 2011
mysql-test Merge branch 'MYSQL-15' into twitter-dev Mar 6, 2012
mysys MYSQL-15: Implement server-side statement timeout support Feb 29, 2012
packaging/WiX Updated/added copyright headers Jun 30, 2011
plugin Fix false positive uninitialized variable warnings. Oct 24, 2011
regex Merge from mysql-5.1 for bug#58026. Feb 4, 2011
scripts MYSQL-15: Implement server-side statement timeout support Mar 6, 2012
sql-bench Updated/added copyright headers Jun 30, 2011
sql-common Merging BUG#13458237 from 5.1. Jan 24, 2012
sql Merge branch 'MYSQL-15' into twitter-dev Mar 6, 2012
storage Bug#64556: Interrupting a query inside InnoDB causes an unrelated war… Mar 6, 2012
strings Bug#13359121 LARGE NUMBERS, /STRINGS/DTOA.C:662 Jan 25, 2012
support-files Use the appropriate team name in the packager field. Feb 23, 2012
tests Auto-merge from mysql-5.1-security. Oct 20, 2011
unittest MYSQL-15: Implement server-side statement timeout support Feb 29, 2012
vio Updated/added copyright headers Jul 3, 2011
win Updated/added copyright headers Jun 30, 2011
zlib WL#5665: Removal of the autotools-based build system Nov 20, 2010
.bzrfileids Merge with main for BUG#12969301 Jan 30, 2012
.bzrignore Specify intentionally untracked files to ignore. Oct 24, 2011
.reviewboardrc Specify the repository name for Review Board. Nov 16, 2011
BUILD-CMAKE Updated/added copyright headers Jun 30, 2011
CMakeLists.txt Bug#13551687 CMAKE GENERATED PROJECT FILES ALWAYS HAVE THE SAME NAME … Jan 16, 2012
COPYING Use a new version of "COPYING", the GPL text. Mar 4, 2010
INSTALL-SOURCE ReadMe.txt, INSTALL-WIN-SOURCE, INSTALL-BINARY, INSTALL-SOURCE: Nov 23, 2007
INSTALL-WIN-SOURCE ReadMe.txt, INSTALL-WIN-SOURCE, INSTALL-BINARY, INSTALL-SOURCE: Nov 23, 2007
README-MySQL Add Twitter README.md; Rename README to README-MySQL Apr 9, 2012
README.md Complete a thought and correct for consistency. Apr 9, 2012
VERSION Merge commit 'tags/clone-5.5.21-build' into twitter-dev Feb 6, 2012
config.h.cmake MYSQL-15: Implement server-side statement timeout support Feb 29, 2012
configure.cmake MYSQL-15: Implement server-side statement timeout support Feb 29, 2012

README.md

Twitter MySQL 5.5

This is Twitter's MySQL development branch, which is based on MySQL 5.5 as published by Oracle on MySQL on Launchpad.

This repository is published in order to share code and information and is not intended to be used directly outside of Twitter. We provide no guarantees of bug fixes, ongoing maintenance, compatibility, or suitability for any user outside of Twitter.

The original README file provided with the upstream MySQL release can be found at README-MySQL.

Features in Twitter MySQL

Additional status variables

Additional status variables have been added, particularly from the internals of InnoDB. This allows us to monitor our systems more effectively and understand their behavior better when handling production workloads. The variables added are:

  • The number of InnoDB files and tablespace files opened, closed, and currently open. This information was previously not exposed by InnoDB.
  • The number of deadlocks encountered. This information was previously not exposed by InnoDB.
  • The current log sequence number (LSN) as well as the LSN flushed up to and checkpointed up to. This information has previously been available in SHOW ENGINE INNODB STATUS.

Optimization of memory allocation under NUMA

On most recent multi-processor systems, a non-uniform memory access NUMA (NUMA) architecture is in use, which divides the total system memory across multiple NUMA "nodes". When allocating large amounts of memory to InnoDB's buffer pool, as is typical, some inefficiencies as well as serious problems can be encountered. More details about the problems typical in NUMA systems running MySQL can be found in a blog post by Jeremy Cole. The following changes have been made to optimize and improve this:

  • An option has been added to forcibly pre-allocate the entire buffer pool during startup. This is primarily intended to force the system to decide which pages to allocate, and on which NUMA node to allocate them. If the buffer pool can't be fully allocated for any reason, InnoDB will abort during startup.
  • An option has been added to mysqld_safe to wrap the start of mysqld with numactl --interleave=all to interleave memory allocation between all NUMA nodes available. This ensures that no NUMA node is favored for any allocation, so that memory usage will remain even over time between multiple NUMA nodes.
  • An option has been added to mysqld_safe to flush the operating system buffer caches before startup (on Linux only). Linux will normally not evict optional caches to make new allocations unless the system is under memory pressure, which can cause supposedly evenly interleaved memory allocations to still be done unevenly, favoring the node with less data cached before startup. Flushing the caches before startup ensures that no large cache allocations are present in the system before InnoDB allocates its buffer pool.

Server-side query timeout

  • Reduced unnecessary work through improved server-side statement timeout support. This allows the server to proactively cancel queries that run longer than a millisecond-granularity timeout.

Buffer pool export and restore by prefetch

  • Export and restore InnoDB buffer pool in using a safe and lightweight method. This enables us to build tools to support rolling restarts of our services with minimal pain.

Optimization for solid-state drives (SSDs)

  • Optimize MySQL for SSD-based machines, including page-flushing behavior and reduction in writes to disk to improve lifespan.