Timeouts for unit tests

[thekid]

Scope of Change

A timeout facility will be added to unit tests. When a test runs longer
than the time span defined by the timeout facility, the test will be
marked as timeouted.

Rationale

Integration tests that test certain system parts such as an LDAP server
might want to test not only that accessing the server works but also
that a query succeeds in a certain time.

Functionality

A test can be annotated with the @limit annotation as follows:

<?php
  class LdapIntegrationTest extends TestCase {

    #[@test, @limit(time= 1.0)]
    function lookup() {
      // ...
    }
  }
?>

Test failure message:

<?php
  util.profiling.unittest.AssertionFailedError (Timeout) { 
    expected: ["0.100"] but was: ["0.201"] 
  }
?>

Security considerations

n/a

Speed impact

Minimal overhead for checking the existance of the @limit annotation.

Dependencies

n/a

Related documents

• xp-framework/rfc Add annotations for Unittest API #32 - This is where the annotation was originally suggested,
  but later on decided not to be implemented (see comments). It had
  a different meaning there, though.
• http://xp-framework.net/downloads/rfc0059.diff
  Patch implementing this RFC.
• http://xp-framework.net/downloads/rfc0059-unittests.diff
  Patch for XP framework unittest. The author is fully
  aware of the existance of the usleep() function but uses Windows
  at home:)
• http://unittests.xp-framework.net/?net.xp_framework.unittest.tests.LimitTest
  Verifies the limit functionality works

[thekid]

Maybe for timeout we should not have TestFailure as result but
TestTimeout to be able to distinguish?

friebe, Tue, 20 Jun 2006 16:27:39 +0200

[thekid]

Based on discussion with the PHP-Dev team and Frank Kleine of
Schlund+Partner, we decided to name the annotation @limit which
is a more generic name for this and may be enhanced in the future
to support @limit(memory= ...) etc.

friebe, Mon, 26 Jun 2006 16:23:00 +0200