-
Notifications
You must be signed in to change notification settings - Fork 1
Testing
... it does not exist 🥺
The way we envisioned testing, however, was to have it mirror the format of our technical architecture, which is a 'narrow waist' waist architecture that is elaborated more on here. Essentially, our app enforces different layers of abstractions, each using a set of common functions at lower levels of abstractions, and so we can build our testing suite in that same structure as well (this is what we initially attempted, elaborated below in the Old Testing Suite section).
Our old testing suite lived in a __tests__ folder in src/, with the folder structure there mirroring the structure of our src/ folder. We had unit tests that mirrored our technical architecture:
-- peoplepower-web
-- src
-- __tests__
-- lib
- airtable.spec.js
- request.spec.js
airtable.spec.js had unit tests for the lowest level of function calls that we made directly to Airtable (or Airlock), while request.spec.jshad unit tests for the request.js functions.
All this was set up in a testing pipeline (see Testing Pipeline section below) that ran every time we pushed, so we had an immediate, first-cut alarm for pushes that broke anything.
These tests were basically testing calls to Airtable APIs, which read, updated and wrote some data to Airtable to ensure that this functionality exist. In order to support these tests, a dummy table called Test (Development) was made in our Airtable base, and dummy records (beginning with Test ...) was also present in various tables.
The following section is more discussion-based and is meant to be a collection of thoughts and trade-offs we considered when it came to testing, for future developers to consider when they consider implementing their own testing suite.
What we also had planned but never got around to was having unit tests for our utility functions as well, and finally having integration tests for our react components using Jenkins. This is what we meant by having a testing suite that mirrored our technical architecture. We still think this is the optimal approach; so in the ideal case, there would be a unit test for every function in lib, and a test for every react component, as well as integration tests on top of that. In practice, however, that might be difficult to maintain.
The current approach of testing relies on specific dummy data being in Airtalbe, which probably wouldn't be the most ideal solution in a production environment. Some alternatives we considered were:
- have the tests make a request to a different Airtable base. The point of this reset, then, would just be to check that the API is still functional.
- The cons of such an approach would be the need to have an entire base supported just for testing. This was initially simple with naked Airtable called because this entailed simply swapping out a BASE_ID, but with Airlock integrated, this might become more challenging.
- have dummy data that we seed so that tests hitting that endpoint would always return that data. This would have the added benefit of not requiring an internet connection to run tests
- This is a good approach for tests at high levels of abstraction. For instance, one the
airtable.jscalls are tested and ensured to be robust, subsequent calls using airtable.js could just return dummy data. There should still be a network call at the lowest level of the technical architecture though (airtable.js, in this case) in @dfangshuo's opinion, because otherwise a test with seeded data would not be testing anything at all, since all the tests forairtable.jstests for would be the validity of the API.
- This is a good approach for tests at high levels of abstraction. For instance, one the
- @aivantg thoughts: I think the most important candidates for testing are the utility functions that build on top of
airtable.jsandrequest.js.airtable.jsandrequest.jsare effectively part of a separate package all together, so testing them does not really represent our app functioning. I think testing utility functions would be essential, since they are the ones that handle the business logic of our app and the ones that are likely the most fragile. If there is a problem lower down in the hierarchy, it will be surfaced by these higher level tests.
We've had a testing pipeline for most our development in Spring 2020, but we've removed it with #164 because the unit tests depended on some fictitious records in Airtable.
The way to set it up, however, would be to create a workflows folder in the .github folder at root. Within the workflows folder, create a file called core.yml that looks like:
name: Core workflow # TODO: replace with name for this workflow
on: [push]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v1
- uses: borales/actions-yarn@v2.0.0
with:
cmd: install # will run `yarn install` command
- uses: borales/actions-yarn@v2.0.0
env:
REACT_APP_AIRTABLE_API_KEY: ${{ secrets.REACT_APP_AIRTABLE_API_KEY }}
with:
cmd: test # will run `yarn test` command
The tests should live in src/__tests__. How you want to structure tests within that folder is completely up to your discretion, but we recommend mirroring the structure of your src folder. The test files should be named {some name}.spec.js, and they can be run locally with `yarn test.
To put all of what was said above in a more graphical form, what we had was:
-- peoplepower-web
-- .github
-- workflows
- core.yml
-- src
-- __tests__
-- lib
- airtable.spec.js
- request.spec.js
We have never attempted a testing pipeline on peoplepower-node, but it is a great candidate for one. The backend is made up of a number of different endpoints that all have defined tasks. Some are simple and testing involves just making sure that a properly formatted requests triggers an expected action.
If we were able to simulate the data the endpoints received from external APIs (like Enphase or UtilityAPI) we could set up expectations for them and test bill generation as well. This is aided by the fact that bill generation is broken up into multiple helper functions. Though it is difficult to test for valid content since the output is a PDF, which can only be assessed visually. Instead, we could make tests to inspect the data on airtable.