Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically generate golden tests for ToJSON instances of API types #91

Closed
2 tasks done
Anviking opened this issue Mar 20, 2019 · 2 comments
Closed
2 tasks done
Assignees

Comments

@Anviking
Copy link
Collaborator

Anviking commented Mar 20, 2019

Context

Golden tests can help us verify that our API json format doesn't change accidentally.

There were some talk about golden-tests on the legacy-wallet (and there was one ticket created input-output-hk/cardano-wallet-legacy#124)

API breaking changes are bad and also tricky to identify somehow. Right now, there's nothing really helping us not breaking the API apart from our own discipline. Despite being disciplined, something can also slip through our attention and get merged, unnoticed. A simple step towards this would be to add a few golden tests to make sure that API types aren't actually changing in a non backward-compatible ways.

If a given API type has one or a few associated golden test, hopefully, upon making a change, the underlying change would fail. Then, we can assess whether this change is a breaking change or just a soft extension.

We have / will have API-level types with Aeson ToJSON/FromJSON - instances. The API types can be composed of smaller types as product (record) or sum types. It seems like we could generate golden tests automatically.

Decision

Find a way to generate golden-tests automatically, to ensure that the JSON-format stays consistent. We want to test all interesting cases of our types (e.g make sure we test the format of all the error-cases in an error sum-type)

Acceptance Criteria

  1. There must exist a way to trigger re-generation of golden-tests
  2. Generated golden-tests should test the format of all API types, in all interesting cases. (May be weakened if infeasible)
  3. We may utilise https://hackage.haskell.org/package/hspec-golden-aeson

Development Plan

  • Play around with it, make pr
  • Check how it handles large types (whether or not it adapts the number of samples to cover all cases) for second acceptance criteria

PR

Number Base
#102 master
#116 (closed) master

QA

@KtorZ
Copy link
Member

KtorZ commented Mar 22, 2019

(ready for QA?)

@piotr-iohk
Copy link
Contributor

lgtm 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants