Skip to content

Commit

Permalink
docs: adding acceptance tests description (#10534)
Browse files Browse the repository at this point in the history
## Description

We start to adopt acceptance tests when designing new module. We had few internal sessions on how to do and started to practice it. 
We should better document the process and start requiring it with new documents. 

---

### Author Checklist

*All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.*

I have...

- [x] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [x] added `!` to the type prefix if API or client breaking change
- [x] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting))
- [x] provided a link to the relevant issue or specification
- [x] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules)
- [x] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)
- [x] added a changelog entry to `CHANGELOG.md`
- [x] included comments for [documenting Go code](https://blog.golang.org/godoc)
- [x] updated the relevant documentation or specification
- [x] reviewed "Files changed" and left comments if necessary
- [x] confirmed all CI checks have passed

### Reviewers Checklist

*All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.*

I have...

- [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] confirmed `!` in the type prefix if API or client breaking change
- [ ] confirmed all author checklist items have been addressed 
- [ ] reviewed state machine logic
- [ ] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [ ] manually tested (if applicable)
  • Loading branch information
robert-zaremba committed Dec 10, 2021
1 parent 94def69 commit f7f1b86
Show file tree
Hide file tree
Showing 7 changed files with 74 additions and 10 deletions.
65 changes: 64 additions & 1 deletion CODING_GUIDELINES.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,62 @@ Security:
+ code must be always deterministic
+ Thread safety. If some functionality is not thread-safe, or uses something that is not thread-safe, then clearly indicate the risk on each level.

## Testing
## Acceptance tests

Start the design by defining Acceptance Tests. The purpose of Acceptance Testing is to
validate that the product being developed corresponds to the needs of the real users
and is ready for launch. Hence we often talk about **User Acceptance Test** (UAT).
It also gives a better understanding of the product and helps designing a right interface
and API.

UAT should be revisited at each stage of the product development:

![acceptance-tests.png](./docs/assets/acceptance-tests.png)

### Why Acceptance Testing?

- Automated acceptance tests catch serious problems that unit or component test suites could never catch.
- Automated acceptance tests deliver business value the users are expecting as they test user scenarios.
- Automated acceptance tests executed and passed on every build help improve the software delivery process.
- Testers, developers, and customers need to work closely to create suitable automated acceptance test suites.

### How to define Acceptance Test?

The best way to define AT is by starting from the user stories and think about all positive and negative scenarios a user can perform.

Product Developers should collaborate with stakeholders to define AT. Functional experts and business users are both needed for defining AT.

A good pattern for defining AT is listing scenarios with [GIVEN-WHEN-THEN](https://martinfowler.com/bliki/GivenWhenThen.html) format where:

- **GIVEN**: A set of initial circumstances (e.g. bank balance)
- **WHEN**: Some event happens (e.g. customer attempts a transfer)
- **THEN**: The expected result as per the defined behavior of the system

In other words: we define a use case input, current state and the expected outcome. Example:

> Feature: User trades stocks.
> Scenario: User requests a sell before close of trading
>
> Given I have 100 shares of MSFT stock
> And I have 150 shares of APPL stock
> And the time is before close of trading
>
> When I ask to sell 20 shares of MSFT stock
>
> Then I should have 80 shares of MSFT stock
> And I should have 150 shares of APPL stock
> And a sell order for 20 shares of MSFT stock should have been executed
*Reference: [writing acceptance tests](https://openclassrooms.com/en/courses/4544611-write-agile-documentation-user-stories-acceptance-tests/4810081-writing-acceptance-tests)*.

### How and where to add acceptance tests?

Acceptance tests are written in the Markdown format, using the scenario template described above, and be part of the specification (`xx_test.md` file in _spec_ directory). Example: (`eco-credits/spec/06.test.md`)[https://github.com/regen-network/regen-ledger/blob/7297783577e6cd102c5093365b573163680f36a1/x/ecocredit/spec/06_tests.md]

Acceptance tests should be defined during the design phase or at an early stage of development. Moreover, they should be defined before writing a module architecture - it will clarify the purpose and usage of the software.
Automated tests should cover all acceptance tests scenarios.

## Automated Tests

Make sure your code is well tested:

Expand Down Expand Up @@ -87,3 +142,11 @@ Desired outcomes:
- QA reports. Goal is to guide with new tasks and be one of the QA measures.
As a developer, you must help the QA team by providing instructions for User Experience (UX) and functional testing.
### QA Team to cross check Acceptance Tests
Once the AT are defined, the QA team will have an overview of the behavior a user can expect and:
- validate the user experience will be good
- validate the implementation conforms the acceptance tests
- by having a broader overview of the use cases, QA team should be able to define **test suites** and test data to efficiently automate Acceptance Tests and reuse the work.
Binary file added docs/assets/acceptance-tests.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/spec/SPEC-SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ following list is nonbinding and all files are optional.
- `XX_client.md` - list and describe CLI commands and gRPC and REST endpoints
- `XX_params.md` - list all module parameters, their types (in JSON) and examples
- `XX_future_improvements.md` - describe future improvements of this module
- `XX_tests.md` - acceptance tests
- `XX_appendix.md` - supplementary details referenced elsewhere within the spec

### Notation for key-value mapping
Expand Down
2 changes: 1 addition & 1 deletion types/simulation/rand_util.go
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ func RandTimestamp(r *rand.Rand) time.Time {
unixTime := r.Int63n(60*60*24*365*200) * 1000 // convert to milliseconds

// Get milliseconds for a time between Jan 1, 2062 and Jan 1, 2262
rtime := time.UnixMilli(start + unixTime).UnixMilli() / 1000
rtime := time.UnixMilli(start+unixTime).UnixMilli() / 1000
return time.Unix(rtime, 0)
}

Expand Down
9 changes: 5 additions & 4 deletions x/group/internal/orm/spec/01_table.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,12 @@ The model provided for creating a `PrimaryKeyTable` should implement the `Primar
+++ https://github.com/cosmos/cosmos-sdk/blob/9f78f16ae75cc42fc5fe636bde18a453ba74831f/x/group/internal/orm/primary_key.go#L28-L41

`PrimaryKeyFields()` method returns the list of key parts for a given object.
The primary key parts can be []byte, string, and `uint64` types.
The primary key parts can be []byte, string, and `uint64` types.

### Key codec

Key parts, except the last part, follow these rules:
- []byte is encoded with a single byte length prefix
- strings are null-terminated
- `uint64` are encoded using 8 byte big endian.

- []byte is encoded with a single byte length prefix
- strings are null-terminated
- `uint64` are encoded using 8 byte big endian.
6 changes: 3 additions & 3 deletions x/group/internal/orm/spec/02_secondary_index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Secondary Index

Secondary indexes can be used on `Indexable` [tables](01_table.md). Indeed, those tables implement the `Indexable` interface that provides a set of functions that can be called by indexes to register and interact with the tables, like callback functions that are called on entries creation, update or deletion to create, update or remove corresponding entries in the table secondary indexes.
Secondary indexes can be used on `Indexable` [tables](01_table.md). Indeed, those tables implement the `Indexable` interface that provides a set of functions that can be called by indexes to register and interact with the tables, like callback functions that are called on entries creation, update or deletion to create, update or remove corresponding entries in the table secondary indexes.

+++ https://github.com/cosmos/cosmos-sdk/blob/430163ed4eefcc0d67b706411ffc0b7c5414cd90/x/group/internal/orm/types.go#L88-L92

Expand All @@ -16,9 +16,9 @@ Internally, it uses an `Indexer` that manages the persistence of the index based

+++ https://github.com/cosmos/cosmos-sdk/blob/430163ed4eefcc0d67b706411ffc0b7c5414cd90/x/group/internal/orm/indexer.go#L15-L18

The currently used implementation of an `indexer`, `Indexer`, relies on an `IndexerFunc` that should be provided when instantiating the index. Based on the source object, this function returns one or multiple index keys as `[]interface{}`. Such secondary index keys should be bytes, string or `uint64` in order to be handled properly by the [key codec](01_table.md#key-codec) which defines specific encoding for those types.
The currently used implementation of an `indexer`, `Indexer`, relies on an `IndexerFunc` that should be provided when instantiating the index. Based on the source object, this function returns one or multiple index keys as `[]interface{}`. Such secondary index keys should be bytes, string or `uint64` in order to be handled properly by the [key codec](01_table.md#key-codec) which defines specific encoding for those types.
In the index prefix store, the keys are built based on the source object's `RowID` and its secondary index key(s) using the key codec and the values are set as empty bytes.

## UniqueIndex

As opposed to `MultiKeyIndex`, `UniqueIndex` is an index where duplicate keys are prohibited.
As opposed to `MultiKeyIndex`, `UniqueIndex` is an index where duplicate keys are prohibited.
1 change: 0 additions & 1 deletion x/group/internal/orm/spec/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,3 @@ The orm package provides a framework for creating relational database tables wit
3. **[Iterator and Pagination](03_iterator_pagination.md)**
- [Iterator](03_iterator_pagination.md#iterator)
- [Pagination](03_iterator_pagination.md#pagination)

0 comments on commit f7f1b86

Please sign in to comment.