NoSQL data store using the seastar framework, compatible with Apache Cassandra
C++ Python GAP Thrift Hack Assembly Other
Switch branches/tags
Clone or download
duarten and pdziepak tests/cql_test_env: Increase eventually() attempts
The current value has proved to be insufficient for our CI
infrastructure.

Signed-off-by: Duarte Nunes <duarte@scylladb.com>
Message-Id: <20180814112201.8595-1-duarte@scylladb.com>
Latest commit 25a0a0f Aug 14, 2018
Permalink
Failed to load latest commit information.
.github github: direct users asking questions to our mailing list. Jun 21, 2018
api storage_service: Get rid of moving operation Aug 1, 2018
auth auth: Don't use unsupported hashing algorithms Aug 5, 2018
conf config: default murmur3_ignore_msb_bits to 12 even if not specified i… Aug 8, 2018
cql3 cql3: throw proper request exception for INSERT JSON Aug 13, 2018
data data::cell: expose size overhead of external chunks Jun 28, 2018
db config: default murmur3_ignore_msb_bits to 12 even if not specified i… Aug 8, 2018
debug debug: scylla_row_cache_report: Remove duplicated phrase from printout Mar 7, 2018
dht range_streamer: Remove unordered_multimap usage Aug 1, 2018
dist dist/common/scripts/scylla_setup: don't proceed RAID setup until user… Aug 11, 2018
docs docs: fix system.large_partitions doc entry Aug 1, 2018
exceptions cql: add read/write failure exceptions Dec 5, 2017
gms gossip: Log seeds seen Aug 1, 2018
idl gossip: Reduce continuous memory usage Jul 17, 2018
imr imr: detect lsa migrator mismatch Aug 1, 2018
index index: add target_column getter to index Jul 11, 2018
interface thrift: change generated code namespace May 5, 2017
licenses Add xxhash (fast non-cryptographic hash) as submodule Feb 1, 2018
locator range_streamer: Remove unordered_multimap usage Aug 1, 2018
message messaging: categorize more streaming/repair verbs as streaming Jul 15, 2018
repair repair: fix small error-handling logic mistake Jul 10, 2018
scripts scripts: add a script to compute optimal number of compile jobs Jul 30, 2018
seastar @ 8ad870f Merge seastar upstream Aug 12, 2018
service cql3: pass query options to restrictions filter Aug 9, 2018
sstables compaction: demote compaction start/end messages to DEBUG level Jul 25, 2018
streaming streaming: Use streaming_write_priority for the sstable writer Aug 8, 2018
swagger-ui @ 1b212bb Update swagger-ui for local fix (change URL to not to point to pet st… Jun 25, 2015
tests tests/cql_test_env: Increase eventually() attempts Aug 14, 2018
thrift storage_proxy: implement write timeouts Jul 8, 2018
tools/scyllatop Scyllatop to use prometheus by default May 31, 2018
tracing tracing: Pass string_view instead of string to add_query Aug 13, 2018
transport tracing: Pass string_view instead of string to add_query Aug 13, 2018
utils logalloc: reindent Jul 31, 2018
xxHash @ 744892b Add xxhash (fast non-cryptographic hash) as submodule Feb 1, 2018
.gitattributes Add .gitattributes file to classify C++ source Oct 5, 2015
.gitignore .gitignore: add resources directory Jun 19, 2018
.gitmodules Add xxhash (fast non-cryptographic hash) as submodule Feb 1, 2018
.gitorderfile gitorderfile: make changes into *.py files appear first May 12, 2015
CMakeLists.txt CMakeLists: Add xxhash directory Feb 1, 2018
CONTRIBUTING.md CONTRIBUTING.md: add sections for help and issues Nov 18, 2016
Doxyfile docs: exclude dpdk Jun 24, 2015
HACKING.md HACKING.md: update ./install-dependencies.sh filename Aug 1, 2018
IDL.md Add an IDL definition file Jan 24, 2016
LICENSE.AGPL Add the AGPL license Sep 20, 2015
MAINTAINERS scripts/find-maintainer: Find subsystem maintainer Jan 30, 2018
NOTICE.txt utils::crc32: add power64 crc32 HW accelerated implementation Dec 8, 2017
ORIGIN Update ORIGIN for gossip and storage_service Dec 1, 2015
README-DPDK.md README: fix typos and paramter syntax Jun 28, 2015
README.md README.md: Add quick-start section and defer to `HACKING.md` Jul 25, 2017
SCYLLA-VERSION-GEN build: improve support for custom builds Jan 22, 2017
atomic_cell.cc atomic_cell: accept fragmented_temporary_buffer::view values Jul 18, 2018
atomic_cell.hh atomic_cell: accept fragmented_temporary_buffer::view values Jul 18, 2018
atomic_cell_hash.hh atomic_cell: switch to new IMR-based cell reperesentation May 31, 2018
atomic_cell_or_collection.hh atomic_cell: switch to new IMR-based cell reperesentation May 31, 2018
backlog_controller.hh controller: adjust constants for compaction controller Jun 18, 2018
bytes.cc Fix pre-ScyllaDB copyright statements Apr 8, 2016
bytes.hh types: move compare_unsigned() to bytes.hh May 9, 2018
bytes_ostream.hh bytes_ostream: increase max chunk size to 128 kB Jul 9, 2018
cache_flat_mutation_reader.hh row_cache: Fix violation of continuity on concurrent eviction and pop… Jul 17, 2018
caching_options.hh Merge seatar upstream (seastar namespace) May 21, 2017
canonical_mutation.cc Swap arguments order of mutation constructor Jan 21, 2018
canonical_mutation.hh remove db/serializer.hh includes Mar 2, 2016
cartesian_product.hh Fix pre-ScyllaDB copyright statements Apr 8, 2016
cell_locking.hh utils: Extract small_vector.hh May 30, 2018
checked-file-impl.hh checked_file_impl: remove unneeded variant of open_checked_file_dma Nov 13, 2017
clocks-impl.cc Move common clock implementation helpers Jun 23, 2017
clocks-impl.hh Move common clock implementation helpers Jun 23, 2017
clustering_bounds_comparator.hh Use std::reference_wrapper instead of a plain reference in bound_view. Jun 28, 2018
clustering_key_filter.hh clustering_key_filter_ranges: Fix move assignment to avoid undefined … Aug 8, 2018
clustering_ranges_walker.hh sstables: Minor clean-up and renaming to clustering_ranges_walker. Jul 20, 2018
coding-style.md Add `coding-style.md` referencing Seastar Oct 30, 2017
combine.hh Fix pre-ScyllaDB copyright statements Apr 8, 2016
compaction_strategy.hh database: rename column_family to table Jun 24, 2018
compatible_ring_position_view.hh sstables_set::incremental_selector: use ring_position instead of token Jul 4, 2018
compound.hh Attach backtrace to marshal_exception-s thrown from generic functions. Dec 5, 2017
compound_compat.hh Pass sstable version to describe_type Apr 24, 2018
compress.cc compress: adjust HAVE_LZ4_COMPRESS_DEFAULT macro for new name Apr 29, 2018
compress.hh sstables::compress/compress: Make compression a virtual object Feb 7, 2018
configure.py build: compress debug sections Aug 11, 2018
converting_mutation_partition_applier.hh mutation_partition_view: pass cell by value to visitor Jun 28, 2018
counters.cc atomic_cell: switch to new IMR-based cell reperesentation May 31, 2018
counters.hh atomic_cell: switch to new IMR-based cell reperesentation May 31, 2018
cql_serialization_format.hh Replace iostream include with iosfwd in headers Jan 17, 2017
database.cc Add a counter to count large partition warning in compaction Aug 7, 2018
database.hh Merge "database: support multiple data directories" from Glauber Jul 15, 2018
database_fwd.hh database: rename column_family to table Jun 24, 2018
db_clock.hh Seal clock definitions Jun 23, 2017
debug.hh Fix pre-ScyllaDB copyright statements Apr 8, 2016
digest_algorithm.hh service/storage_service: Add and use xxhash feature Feb 1, 2018
digester.hh row: Use cached hash for hash calculation Feb 1, 2018
dirty_memory_manager.hh database: tag dirty memory managers with scheduling groups Jul 31, 2018
disk-error-handler.cc Move `thread_local` declarations out of `main.cc` Nov 27, 2017
disk-error-handler.hh Merge seatar upstream (seastar namespace) May 21, 2017
duration.cc duration.cc: Remove nop `const` qualifier on return type Aug 10, 2017
duration.hh Change `const` qualifier declaration order for `duration` Aug 10, 2017
encoding_stats.hh Fix timestamp_epoch value which was truncated on exceeding int32_t ty… May 4, 2018
enum_set.hh enum_set: Add iterator Feb 14, 2018
fix_system_distributed_tables.py tracing: introduce a span ID and parent span ID Apr 26, 2017
flat_mutation_reader.cc flat_mutation_reader: Move field initialization to initializer list Jun 25, 2018
flat_mutation_reader.hh treewide: require type to compute cell memory usage May 31, 2018
fnv1a_hasher.hh add fnv1a hasher Feb 2, 2017
frozen_mutation.cc treewide: require type to compute cell memory usage May 31, 2018
frozen_mutation.hh frozen_mutation: introduce frozen_mutation_fragment May 25, 2018
frozen_schema.cc schema_tables: Require context object in schema load path Feb 7, 2018
frozen_schema.hh schema_tables: Require context object in schema load path Feb 7, 2018
gc_clock.hh gc_clock: introduce operator<<(ostream&, gc_clock::time_point) Dec 6, 2017
gen_segmented_compress_params.py Revert "Revert "Revert "Revert "Merge "Compress in-memory compression… Aug 28, 2017
hashing.hh Merge seatar upstream (seastar namespace) May 21, 2017
hashing_partition_visitor.hh range_tombstone: Replace feed_hash() member function with appending_hash Feb 1, 2018
idl-compiler.py idl: remove for_each from fragmented serialization Jun 13, 2018
init.cc messaging: tag RPC services with scheduling groups Jul 13, 2018
init.hh messaging: tag RPC services with scheduling groups Jul 13, 2018
install-dependencies.sh install_dependencies.sh: centos: add systemd-devel Apr 26, 2018
install.sh dist: drop collectd support from package Jun 14, 2018
intrusive_set_external_comparator.hh intrusive_set_external_comparator: Introduce container_of_only_member() Mar 6, 2018
json.cc json: add value_to_quoted_string helper function Jul 25, 2018
json.hh json: add value_to_quoted_string helper function Jul 25, 2018
keys.cc keys: schema-aware printing of a partition_key Jul 17, 2018
keys.hh keys: schema-aware printing of a partition_key Jul 17, 2018
lister.cc Merge seatar upstream (seastar namespace) May 21, 2017
lister.hh Merge seatar upstream (seastar namespace) May 21, 2017
log.hh Merge seastar upstream Aug 27, 2017
main.cc main: run prometheus and API servers under streaming group Jul 30, 2018
map_difference.hh map_difference: Allow on unordered_map Apr 20, 2016
marshal_exception.hh Use meaningful error messages when throwing a marshal_exception Nov 21, 2017
md5_hasher.hh Fix Scylla compilation with Crypto++ v6. Mar 4, 2018
memtable-sstable.hh database, sstables, tests: add large_partition_handler May 4, 2018
memtable.cc memtable, cache: Fix exception safety of partition entry insertions Jul 17, 2018
memtable.hh memtable, cache: Run mutation_cleaner worker in its own scheduling group Jun 27, 2018
multishard_writer.cc multishard_writer: Introduce multishard_writer Jun 28, 2018
multishard_writer.hh multishard_writer: Introduce multishard_writer Jun 28, 2018
mutation.cc treewide: require type info for copying atomic_cell_or_collection May 31, 2018
mutation.hh keys: Replace feed_hash() member function with appending_hash Feb 1, 2018
mutation_cleaner.hh memtable, cache: Run mutation_cleaner worker in its own scheduling group Jun 27, 2018
mutation_compactor.hh treewide: require type info for copying atomic_cell_or_collection May 31, 2018
mutation_fragment.cc range_tombstone_stream: Remove an unused boolean flag. Jun 18, 2018
mutation_fragment.hh mutation_fragment: Fix clustering_row::equal() using incorrect column… Jul 9, 2018
mutation_partition.cc mutation_partition: Fix exception safety of row::apply_monotonically() Aug 9, 2018
mutation_partition.hh mutation_partition: Fix exception safety of row::apply_monotonically() Aug 9, 2018
mutation_partition_applier.hh treewide: require type for creating collection_mutation_view May 31, 2018
mutation_partition_serializer.cc atomic_cell: introduce fragmented buffer value interface May 31, 2018
mutation_partition_serializer.hh Rename streamed_mutation* files to mutation_fragment* Jan 24, 2018
mutation_partition_view.cc mutation_partition_view: use column_mapping_entry::is_atomic() Jun 28, 2018
mutation_partition_view.hh mutation_partition_view: pass cell by value to visitor Jun 28, 2018
mutation_partition_visitor.hh mutation_partition: Add support for specifying continuity Jun 24, 2017
mutation_query.cc query-result: Introduce class result_options Feb 1, 2018
mutation_query.hh database: group statements in their own scheduling group Mar 20, 2018
mutation_reader.cc mutation_reader: reader_selector: use ring_position instead of token Jul 4, 2018
mutation_reader.hh mutation_reader: reader_selector: use ring_position instead of token Jul 4, 2018
mutation_rebuilder.hh Swap arguments order of mutation constructor Jan 21, 2018
noexcept_traits.hh Introduce noexcept_traits Dec 7, 2015
partition_builder.hh mutation_partition_view: pass cell by value to visitor Jun 28, 2018
partition_range_compat.hh Convert to use dht::partition_range_vector and dht::token_range_vector Dec 19, 2016
partition_slice_builder.cc partition_slice_builder: Use proper column's type instead of regular_… Jul 17, 2017
partition_slice_builder.hh partition_slice_builder: Add with_ranges() Feb 23, 2017
partition_snapshot_reader.hh mvcc: Use RAII to ensure that partition versions are merged Jun 27, 2018
partition_snapshot_row_cursor.hh partition_snapshot_row_cursor: initialize _dummy and _continuous Jun 2, 2018
partition_version.cc row_cache: Fix memtable reads concurrent with cache update missing wr… Jul 1, 2018
partition_version.hh mutation_cleaner: Make merge() redirect old instance to the new one Jun 27, 2018
partition_version_list.hh mvcc: Introduce partition_version_list May 30, 2018
position_in_partition.hh position_in_partition: Add a constructor from range_tag_t{}, bound_ki… Jul 20, 2018
querier.cc querier: find_querier(): return end() when no querier matches the range Jun 19, 2018
querier.hh querier_cache: restructure entries storage Jun 19, 2018
query-request.hh Add query UUID to read command Mar 13, 2018
query-result-reader.hh query::result_view: add get_last_partition_and_clustering_key() Jul 26, 2018
query-result-set.cc query::result: avoid copying and linearising cell value Jun 25, 2018
query-result-set.hh Remove exception specifications May 5, 2017
query-result-writer.hh query-result: Use digester instead of md5_hasher Feb 1, 2018
query-result.hh Configure query result memory limiter size limit during object creation Jun 11, 2018
query.cc Replace query::full_slice with schema::full_slice() Oct 17, 2017
query_result_merger.hh query_result_merger: Limit rows Dec 15, 2016
range.hh range: clean the deduced transformed type May 10, 2018
range_tombstone.cc Use std::reference_wrapper instead of a plain reference in bound_view. Jun 28, 2018
range_tombstone.hh Use std::reference_wrapper instead of a plain reference in bound_view. Jun 28, 2018
range_tombstone_list.cc Merge "Make in-memory partition version merging preemptable" from Tomasz Jul 1, 2018
range_tombstone_list.hh Merge "Make in-memory partition version merging preemptable" from Tomasz Jul 1, 2018
read_context.hh cache_flat_mutation_reader: Pre-calculate cell hash Feb 1, 2018
reader_concurrency_semaphore.hh Resource-based cache eviction Mar 13, 2018
real_dirty_memory_accounter.hh cache: real_dirty_memory_accounter: Move unpinning out of the hot path May 30, 2018
release.cc Fix pre-ScyllaDB copyright statements Apr 8, 2016
release.hh Fix pre-ScyllaDB copyright statements Apr 8, 2016
reversibly_mergeable.hh Fix pre-ScyllaDB copyright statements Apr 8, 2016
row_cache.cc memtable, cache: Fix exception safety of partition entry insertions Jul 17, 2018
row_cache.hh memtable, cache: Run mutation_cleaner worker in its own scheduling group Jun 27, 2018
schema.cc schema: column_mapping_entry: reduce logic duplication Jun 28, 2018
schema.hh schema: column_mapping_entry: cache abstract_type::is_atomic() Jun 28, 2018
schema_builder.hh schema: add clearing indexes to schema builder May 22, 2018
schema_mutations.cc schema_mutations: Use mutation_opt instead of stdx::optional<mutation> Sep 14, 2017
schema_mutations.hh schema_tables: Don't alter tables which differ only in version Sep 14, 2017
schema_registry.cc schema_tables: Require context object in schema load path Feb 7, 2018
schema_registry.hh schema_tables: Require context object in schema load path Feb 7, 2018
schema_upgrader.hh atomic_cell: require column_definition for creating atomic_cell views May 31, 2018
scylla-gdb.py gdb: Fix scylla heapprof command Jul 12, 2018
scylla-housekeeping scylla-housekeeping: support new 2018.1 path variation May 9, 2018
seastarx.hh seastarx: add missing make_shared forward declaration Jun 22, 2017
serialization_visitors.hh idl: add start_frame() overload for seastar::simple_output_stream Feb 27, 2017
serializer.hh serializer: Add chunked_vector support Jul 17, 2018
serializer_impl.hh serializer: Add chunked_vector support Jul 17, 2018
setup.py Add overall python code QA and guidelines with flake8 Aug 8, 2017
stdx.hh seastarx: don't make seastar namespace inline Jun 22, 2017
supervisor.cc supervisor: remove dependency on init.hh Aug 27, 2017
supervisor.hh Merge seatar upstream (seastar namespace) May 21, 2017
table_helper.cc cql: Add schema extensions processing to properties Feb 7, 2018
table_helper.hh cql3: change cql_statement methods to accept a local storage_proxy Apr 16, 2018
test.py tests: add test for fragmented_temporary_buffer Jul 18, 2018
timeout_config.hh timeout_config: introduce timeout configuration Apr 29, 2018
timestamp.hh Seal clock definitions Jun 23, 2017
to_string.hh to_string: Add std::list and utils::chunked_vector support Jul 17, 2018
tombstone.hh mutation_partition: Define + operator on tombstones Feb 6, 2018
tox.ini Add overall python code QA and guidelines with flake8 Aug 8, 2017
types.cc types: enable parsing numeric JSON values from string Aug 13, 2018
types.hh cql3: provide to_json_string for optional bytes argument Aug 9, 2018
unimplemented.cc metadata_type: add Serialization type Apr 24, 2018
unimplemented.hh metadata_type: add Serialization type Apr 24, 2018
validation.cc validation: Add KS validation + convinence methods Apr 19, 2016
validation.hh validation: Add KS validation + convinence methods Apr 19, 2016
version.hh Merge seatar upstream (seastar namespace) May 21, 2017
view_info.hh view: add is_index method Jun 5, 2018
vint-serialization.cc Add signed_vint::serialized_size_from_first_byte May 9, 2018
vint-serialization.hh Add signed_vint::serialized_size_from_first_byte May 9, 2018
xx_hasher.hh digest: Introduce xxHash hash algorithm Feb 1, 2018

README.md

Scylla

Quick-start

$ git submodule update --init --recursive
$ sudo ./install-dependencies.sh
$ ./configure.py --mode=release
$ ninja-build -j4 # Assuming 4 system threads.
$ ./build/release/scylla
$ # Rejoice!

Please see HACKING.md for detailed information on building and developing Scylla.

Running Scylla

  • Run Scylla
./build/release/scylla

  • run Scylla with one CPU and ./tmp as data directory
./build/release/scylla --datadir tmp --commitlog-directory tmp --smp 1
  • For more run options:
./build/release/scylla --help

Building Fedora RPM

As a pre-requisite, you need to install Mock on your machine:

# Install mock:
sudo yum install mock

# Add user to the "mock" group:
usermod -a -G mock $USER && newgrp mock

Then, to build an RPM, run:

./dist/redhat/build_rpm.sh

The built RPM is stored in /var/lib/mock/<configuration>/result directory. For example, on Fedora 21 mock reports the following:

INFO: Done(scylla-server-0.00-1.fc21.src.rpm) Config(default) 20 minutes 7 seconds
INFO: Results and/or logs in: /var/lib/mock/fedora-21-x86_64/result

Building Fedora-based Docker image

Build a Docker image with:

cd dist/docker
docker build -t <image-name> .

Run the image with:

docker run -p $(hostname -i):9042:9042 -i -t <image name>

Contributing to Scylla

Guidelines for contributing