Best practices

Testing Clients best practices

This document provides an unordered list of best practices for extending the framework and writing tests. Some of these advices might seem like common sense, some might not.

Try to not reinvent the wheel

The testing client sprovide a lot of rules and clients for using one or multiple AEM instances in a generic way. It's also layered, from generic actions in sling to more specific actions in cq and granite (to be merge with cq).

If it looks like there might be a client action in the testing clients, use that.

Junit rules

• Use and compose the provided Junit rules. They're great! Like the CQAuthorPublishClassRule
• Try to avoid the TestBase parent class pattern. Anything you can do with parent classes and more, you can do with Junit Rules. Chaining rules looks like this:

public class BaseTestClassRule implements TestRule {

    public BaseTestClassRule() {
        super();
        cqClassRule = new CQClassRule();
        author1Rule = ClassRuleUtils.newInstanceRule(true).withRunMode("author");
        author2Rule = ClassRuleUtils.newInstanceRule(true).withRunMode("author");
        publishRule = ClassRuleUtils.newInstanceRule(true).withRunMode("publish");
        defaultReplicationAgentsRule = new DefaultReplicationAgents(author1Rule, publishRule);
        ruleChain = RuleChain.outerRule(cqClassRule)
                .around(author1Rule)
                .around(author2Rule)
                .around(publishRule)
                .around(defaultReplicationAgentsRule);
    }
	
	[...]    

    @Override
    public Statement apply(Statement base, Description description) {
        return ruleChain.apply(base, description);
    }
}

• Then, instead of extending a BaseTest class, apply a BaseTestClassRule

Don't copy/paste code

If you find yourself performing a certain action more than once, put that in a utility class or a custom client - they provide a lot of goodies.

Try avoiding raw HTTP requests

Try to use the most top-level clients and utilities provided by the framework - they provide a lot of stuff out-of-the-box (e.g. pointing to the topology under test, authentication, sharing the cookie store, timeouts, etc.)

Avoid using things like HttpGet directly and use AbstractSlingClient#doGet.

Waiting for resources (polling)

As a rule of thumb, you should never use Thread.sleep(). Always use AbstractPoller or better yet, the newer Polling class for trying the action until it returns the expected result.

Content paths and scoping

Always use a specific name-spaced paths. E.g. add tst- prefix to all the resources created. You will later be able to easily identify and clean the generated content

Always clean up in the tests

Make sure tests don't leave side-effects behind. For example:

• Use OsgiInstanceConfig to set an osgi property and restore it at the end to the original value before the test
• Use the Page resource rule that deletes the page at the end
• Clean up/ restore in an @After or @AfterClass anything that was created or changed in the test. Better yet, use a rule (it's like a reusable class that provides before and after )

Permissive setup/ cleanup and strict asserts

When writing an end-to-end test, one often times needs to do things like prepare some content or create a user, in order to actually test a certain functionality.

In those cases, obviously separate the setup and cleanup (!) in before/ after methods or their class equivalents. Additionally, try and follow this rule:

• Setup and cleanup should be permissive - e.g. retry a page creation or deletion. Afterall, it's not what the test actually needs to test; there's a different test for page creation.
• On the other hand, make asserts strict - e.g. adding a component to that prepared page shoudl work the first time! (unless specified otherwise in documentation or the expectations of the feature)

Make your tests fast, but don't assume they run on fast machines

• Use sensible values for Polling, based on the expectations.
• Max one order of magnitude higher timeouts - e.g. expected to create the user async in 100ms, you can poll every 250ms, for max 2.5 seconds.
• It's not enough if "it works on my machine". The tests are reused in multiple scenarios, on different types of deployments, with varying resources.

Follow the testing pyramid

The http testing clients are meant for functional/ integrated http tests. They are relatively high on the testing pyramid, under UI tests, but clearly above unit tests. That means the complexity, the execution time and needed resources are quite high. Try covering as much as possible with unit tests and minimal integration tests. Try the OSGi mocks or pax exam, they're great!

Never catch InterruptedException

InterruptedException is thrown by blocking methods, e.g. Thread.sleep() and it's a mechanism to gracefully terminate a program. If this exception is inhibited (catch without rethrow), the program/test will not respect the SIGINT signal (e.g. will not stop at CTRL+C). In corner cases, you might need to catch the exception to allow the method to finish the task, but make sure you rethrow it at the end.

Besides this, throws InterruptedException is a way to notify the user that the method is blocking (performing a wait).