Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup command line based regression tests #4

Open
happybeing opened this issue Jan 31, 2019 · 0 comments
Open

Setup command line based regression tests #4

happybeing opened this issue Jan 31, 2019 · 0 comments
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed

Comments

@happybeing
Copy link
Owner

This is something that anyone could help with, and where even very small amounts of your time can be useful (eg writing a single test), and where several people could collaborate.

We have a crude regression test script (test-safedrive.sh) which can be used as a template for what's needed. So the first step would be to implement equivalent functionality to that, and then extend with more tests.


Setup scripts and tests using BATS (Bash Automated Testing System) to automate a set of regression tests that can run against mock and live network.

This should include a series of small commands to exercise SAFE Drive feature (creating files in different ways, listing directories, checking file content, creating public names, mounting containers in different ways etc).

The tests would check both the interaction with the console (eg copy file command shows success, not error), and by linking multiple commands together, can test that the commands also had the desired effect (eg checking that the file was actually copied).

In general good outcomes and bad outcomes should be tested for. So for example both a copy file command that should succeed, and one that should fail.

Tests should be able to run either individually, or in small self contained groups rather than as one very long script, so that re-running a test while checking the behaviour after a code change can be done quickly. So npm run tests copy-file etc, as well as...

It should also be possible to run all tests from start to finish with a single command (eg npm run tests all), and for test output to be visible on the console and logged to a file for reference.

Any files required as inputs for the tests should be kept in a tests/input sub-directory or directory tree, and any results files stored in their own tests/results directory.

Provide instructions on how to set up and run the tests, either as a scripted command (eg npm run tests without parameters) or as as a document that is referred to when the test script is run. This can explain, for example, how to start the browser, whether to login/create account etc before starting the test script.

The tests might of course need to cover behaviour with/without browser started, with/without account logged in, so tests with different starting conditions should ideally prompt for confirmation that the conditions are as required before beginning to run the test itself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

1 participant