Skip to content

Latest commit

 

History

History
105 lines (72 loc) · 9.07 KB

MastercoinDistributedExchangeTestPlan.md

File metadata and controls

105 lines (72 loc) · 9.07 KB

Mastercoin Distributed Exchange Test Plan

Introduction

The Distributed Exchange (Dist-Ex) is a primary feature of Master Protocol / Mastercoin (MSC) functionality. MSC system integrity and user confidence in the system depend on the Dist-Ex performing correctly under all circumstances.

Testing Philosophy

Our main goal is to ensure that all Dist-Ex components adhere to the MSC Protocol spec - primarily creation of all appropriate transactions when presented with valid, well-formed inputs and appropriate initial conditions. This requires:

  • presentation of a correct, meaningful message to the user that the transaction has been created
  • all input and output data structures are left in a demonstrably correct state that reflects successful creation of the transaction

Dist-Ex components also have to detect errors in all phases of processing and react gracefully. Graceful reaction requires:

  • presentation of a correct, meaningful message to the user, with options if that’s appropriate
  • the transaction is fully undone/rolled back if it cannot be completed successfully
  • all input and output data structures are left as they were before the transaction was attempted - in a demonstrably known state with no corruption
  • presentation of a correct, meaningful message to the user that the transaction has either succeeded or failed

Since MSC Protocol transaction data is embedded in the Bitcoin blockchain, testing must confirm that the blockchain is not corrupted or incorrectly modified by the Dist-Ex components. Please refer to the MSC Protocol spec appendix Storing Mastercoin Data in the Blockchain for details about how MSC Protocol transaction data is to be embedded in the Bitcoin blockchain.

Test Plan Highlights

We’ll need test harnesses, object inspectors (e.g. Masterchain.info and other utilities from https://github.com/grazcoin/mastercoin-tools, http://wiki.mastercoin.org/index.php/Block_Explorers), scripts, known inputs, etc. to set up and run comprehensive and repeatable functionality testing in clean environments. We’ll have to verify that these components themselves are correct. For now, the bulk of testing can be done using Test Mastercoins (TMSC) on the Bitcoin mainnet. After testing with TMSC, it’s best to test with MSC on the Bitcoin mainnet if possible, to confirm that everything works in the production environment. In the future, we will test using TMSC on Bitcoin testnet, MSC on testnet, TMSC on mainnet, and finally with MSC on mainnet.

Major Items & Scenarios To Be Tested

Each candidate application will be tested for the following minimum functionality:

  1. user is able to enter valid values for all appropriate fields
  2. valid input data is parsed and interpreted correctly, including all valid MSC Protocol transaction types in the blockchain that are supported by the application
  3. erroneous input data, including a corrupted blockchain, is detected as not valid and handled correctly
  4. correct transactions are generated for all MSC Protocol transaction types supported by the application
  5. multi-participant sequences (e.g. Sell Offer by user A then Purchase Offer by user B)
  6. multi-user tests (e.g. multiple users attempt to accept the same Sell Offer)
  7. runtime error conditions are detected and handled correctly (e.g. insufficient funds, application/network/OS failure)
  8. edge cases are handled correctly
  9. correct, meaningful messages reflecting transaction success/status or failure are presented to the user

The MSC Protocol spec appendix Webservice verification API has information about basic transaction verification services that should be implemented by web-based Mastercoin services.

MSC Protocol Transaction Types

For convenience, here’s a list of the MSC Protocol transaction types from the MSC Protocol spec, version 0.4.5.1 at the time of this update. The spec always has the authoritative list and descriptions of MSC Protocol transactions.

Basic Tests

A minimum level of functionality is required for all transactions.

Positive Tests

These tests will determine if the application under test (AUT) can produce correct results when provided with valid inputs.

Transactions for each supported transaction type will be generated. For transaction data items that accept a range of values, tests should include values at both ends of the range as well as at least three values in between. For transaction data items that accept values from a list, all values should be tested if possible.

For each transaction, the resulting blockchain will be read by at least:

  • the AUT instance that generated the transaction, if possible
  • a second instance of the AUT
  • an independent blockchain explorer

Independently developed dist-ex applications, if available, should also attempt to read the resulting blockchain.

The applications that read the resulting blockchain will verify that the AUT:

  • did not corrupt the blockchain
  • correctly added the transaction to the blockchain
  • made no other changes to the blockchain

Negative Tests

These tests will determine if the application can detect erroneous inputs and runtime environmental errors and react properly.

Erroneous inputs include items such as missing data, malformed/corrupted data (including a bad blockchain), numeric values outside of acceptable ranges, values not in the list of valid inputs, non-numeric data where numeric values are required, unsupported transaction messages and transaction message versions. Each application is expected to have additional erroneous inputs. Runtime environmental errors can be introduced by doing things such as manually killing the application process, disconnecting the network, removing or changing permissions of data files, powering down the machine.

Transaction-specific Tests

Baseline test requirements for each transaction type are identified in TestRequirements. These test requirements serve as guidelines for implementing specific test procedures for each application under test.