-
Notifications
You must be signed in to change notification settings - Fork 1
Testing Strategy
RapidFTR has been built following Behaviour-Driven Development practices using Cucumber and RSpec, and contributors should continue to follow these practices. To briefly sum up our development process and testing strategy:
Stories (all viewable in Mingle) marked ready to be picked up by developers should already have some acceptance criteria attached to them. These can be used the developer as guidance for what needs to be done for the story to be complete. These acceptance criteria will make a good basis for defining a number of acceptance tests, using Cucumber. (If you don’t have Mingle access yet, then see the bottom of this page for an example of a story’s acceptance criteria.)
A Cucumber test should be created before code for a new feature is added to the system. (If new to Cucumber, have a look at our existing Cucumber tests in RapidFTR/capybara_features for examples, and study the Cucumber documentation.) A failing Cucumber test will help guide you in deciding what code needs to be written. Start with a single complete Cucumber test, run it to see it fail (and the reason why) then make it pass one step at a time by writing code. (Remember to rerun the whole suite of Cucumber tests before committing to make sure your changes haven’t broken other functionality!) Once a cucumber test is passing successfully, check in, then write the next cucumber test required to make sure all acceptance criteria on the story are covered.
When writing code to make each step of a Cucumber test pass, we use RSpec tests to drive out those changes. (Rspec tests are unit tests- they specify the behaviour of individual objects, e.g. models, controllers, and views.) Sometimes the implementation of some of that code can be so trivial that it isn’t worth writing an RSpec test around it, and the fact that the Cucumber test has been made to pass is confirmation enough that this code is correct. So in this case we wouldn’t expect a developer to necessarily have RSpec tests for everything in their views and/or controllers. This is something we leave to a developer’s discretion. (Untested model code is not something we would expect!)
As this is an open source project with many different contributors, we consider good test coverage very important to maintain a fully working system at all times. Please help us by having your code submissions accompanied by Cucumber and/or RSpec tests that verifies that code does what it should. Having well named tests will really help us too.
- Any defects that creep up can be captured on the “Default Wall” in Mingle or the GitHub bug tracker to be picked up by developers and fixed.
- Automation tests should be written as part of the fix, unless there’s a valid reason not to. A highly recommended approach is:
- Write an automated test which exposes the defect
- Make sure the test fails
- Fix the defect
- Make sure the test passes
- A story should be closed only after all the major defects against the story have been fixed.
- As the system evolves over time, we need to ensure that the user experience in terms of page load response time (Performance) is up to the expectations. At the same time we should ensure that the system does not crash when the number of hits exceed a certain limit (Load). It is ideal to have a separate performance environment for these kind of testing.
- Security also needs to be covered here. We need to look out for things like session time outs, multiple logins through multi browsers / locations.
- The user interface and user experience should be verified against the design specifications. Testing the application across well known browsers is highly recommended to ensure compatibility.
Story #41 Edit a user’s details
- Requirements
- An adminstrator can edit a user’s details from the list of users
- Any field can be edited
- Admin can save changes
- Updated information is shown in list of users
Acceptance Tests
- As an admin,
- Given that I am on the user listing view page,
- When I click on the edit link against any record,
- Then I should be taken to a page where I can edit the user details.
- As an admin,
- Given that I am on the edit user details page,
- Then I should not be able to edit the “username” field
- As an admin,
- Given that I edit user details,
- When I click on update button
- Then the changed user details should be updated and the others should remain the same.
- As a user,
- Given that I am on the edit user record page,
- Then all the validations of the fields should still hold true
For developers submitting pull requests for review, please do keep in mind the following:
1. For Web(RoR) submissions, one must write good unit tests (in rspec) and appropriate functional test (in cucumber/capybara).
2. For Android submissions, Robolectric tests for unit test coverage must be written. RapidFTR uses Robotium (http://code.google.com/p/robotium/) for integration tests — and hence appropriate coverage must also be present in robotium tests.
We would like to have unit test that cover all the scenarios (positive & negative) and functional tests for the happy path scenario.
Please do see some of the older commits done by other developers for better understanding, or send an email for clarifications to the RapidFTR mailing list.
- To run unit tests:
bundle exec rake spec jasmine:ci - To run cucumber tests:
bundle exec rake cucumber:all