Testing Use Cases

[spencerkimball]

Create use cases for acceptance and performance / load tests. These should follow the general format of the banking example in examples/sql_bank and can/should run the gamut of creativity. A key feature of the test should be a mix of continual verification. Each test should allow its major features to be tunable with command line flags, including # of iterations (including and option for infinite looping).

Please add your test case below with your name and a description which highlights what's being tested as well as how it's being verified. Before adding your test case, read the existing entries and try to avoid duplicative efforts. Follow the format below.

[DO NOT DELETE THIS PLACEHOLDER]

[Test Name] - [Github username]
[Description]
[Functionality tested]

[Verification mechanism]

[petermattis]

Filesystem - petermattis

Create a fuse mountable filesystem that stores its data in a CockroachDB instance. The filesystem metadata (directories and files) would be an adaptation of the namespace table and TableDescriptor used by SQL. File data would stored in 4KB chunks. I'd ignore permissions and all that jazz, this being only an example.

Not sure what to do for verification, but there are many possibilities. We could mount the filesystem, copy the cockroach source code into and compile it. I'm open to suggestions here.

[tbg]

Real Bank - tschottdorf

We already have the bank example, but our friends at Mondo have a real bank. Currently they're using Postgres for their main ledger, but are obviously concerned about scalability.

Correctness, of course, is straightforward. What I'm hoping to get out of this is a little tool with various knobs to simulate high loads using their real-world queries.

[bdarnell]

Slackroach - bdarnell

Chat rooms with fake real-time updates using the same model as friendfeed. Tests mixed read/write contention on the tail of a log-style table.

Verification: Measure time from a writer posting a message to the last reader seeing it.

[vivekmenezes]

inverted-index - vivek (https://en.wikipedia.org/wiki/Inverted_index)

Develop an inverted index as a foundation for full text search. Take a document stored in a text column, split it into words, and add the document into posting lists for each word contained within the document.

Verification: Measure time it takes to index a document in the presence of posting-list contention.

[BramGruneir]

Artificial Neural Network - BramGruneir

Artificial Neural Networks (ANNs) seem to be coming back in vogue but the catch is that to be effective, they've just become much larger and contain more hidden layers (see Deep Learning). Running a neural network requires a large number parallel operations. Training a neural network is even worse. I propose we build a simulation of a basic but scalable neural network on Cockroach, trying to maximize the parallelization of the operations.

Verification: This is all about parallelism, to so see how large we can scale out the problem and what amount of load it can handle.

[mrtracy]

Work Lease Service - mrtracy

The simulation will involve incoming "work tickets" which require N steps to complete in a definite order; each step is performed by a specialized worker. There will be multiple available workers of each specialization, concurrently competing for the tickets. To be successfully completed, a ticket must be exclusively modified by a worker of each type in the correct order. Workers can fail before completing their task, at which point a different worker should pick up the ticket.

Verification: All tickets are completed correctly, no work is duplicated, all workers agree on who completed which tickets.

[vivekmenezes]

The inverted index is something that we will be putting on our roadmap so I'm not going to do it for Tuesday.

I just got bombarded with a number of deployment tools that I'd like to play around with on Tuesday:

packer.io/
https://hashicorp.com/
https://terraform.io/
https://clusterhq.com/flocker/introduction/

[tbg]

between wedding-related OOO and catching up on work I didn't manage to talk to the mondo guys at all. I'll try to catch them today but I expect that I'll just be working on my active issues instead.

[spencerkimball]

Create an asset price database with schema (timestamp, cusip, price). Each asset has a random number of post-hoc adjustments made to its price. Queries are done to scan all prices at both market close time and current time to verify the original and adjusted values.