Permalink
Commits on Feb 27, 2016
  1. Merge pull request #80 from dylan-lang/indent-log-report

    [log report] Indent based on suite/test hierarchy.
    cgay committed Feb 27, 2016
  2. [log report] Indent based on suite/test hierarchy.

    * reports.dylan
      (log-report-function): Wrap the stream in an indenting stream.
       When recursing on subresults, indent the stream by 2 spaces.
    waywardmonkeys committed Feb 27, 2016
Commits on Feb 23, 2016
  1. Merge pull request #79 from dylan-lang/cleanup-do-check-functions

    Cleanup do-check-* functions
    cgay committed Feb 23, 2016
Commits on Feb 17, 2016
  1. Update assert-instance? messages.

    The other checks use the same terminology as each other, but
    the `do-check-instance?` was still using the old terminology.
    
    * assertions.dylan
      (do-check-instance?): Update message text.
    waywardmonkeys committed Feb 17, 2016
  2. Make all do-check-* functions have same result type.

    One of the do-check-* functions was returning the <result> object
    while the others were all just returning the <result-status> object.
    Making them all return the same type of result will make it easier
    to combine the bulk of the logic into a single helper method or
    macro. It also makes it easier to extend the testing in the future
    to cover the reasons that are generated for test failures.
    
    * assertions.dylan
      (do-check-instance?): Update return values.
      (do-check-true): Likewise.
      (do-check-false): Likewise.
      (do-check-condition): Likewise.
    
    * tests/testworks-test-suite.dylan
      (test testworks-check-true-test,
       test testworks-assert-true-test,
       test testworks-check-false-test,
       test testworks-assert-false-test,
       test testworks-assert-equal-test,
       test testworks-assert-not-equal-test,
       test testworks-check-instance?-test,
       test testworks-assert-instance?-test,
       test testworks-not-assert-instance?-test,
       test testworks-check-condition-test,
       test testworks-assert-signals-test,
       test testworks-check-no-errors-test,
       test testworks-assert-no-errors-test): Update for new return values
        and to get the actual result status from the result object.
    waywardmonkeys committed Feb 17, 2016
Commits on Feb 5, 2016
  1. Merge pull request #78 from dylan-lang/execute-component-values

    Let execute-component return <component-result>.
    cgay committed Feb 5, 2016
  2. Let execute-component return <component-result>.

    This increases duplication of calling make to create the result,
    but it allows for execute-component to do something different to
    create results unlike the current arrangement.
    
    This change lays the ground work for an improvement in how
    benchmarks are implemented.
    
    * run.dylan
      (maybe-execute-component): Update to create result in skipped
       case, otherwise, use result returned from executing the component.
      (execute-component): Change return value to be a result and
       create and return the result.
    waywardmonkeys committed Feb 5, 2016
Commits on Feb 4, 2016
  1. Update comment.

    The slot being described here is for all results, not just benchmarks.
    waywardmonkeys committed Feb 4, 2016
Commits on Sep 20, 2015
  1. [docs] Improve example of assertion description.

    * documentation/users-guide/source/reference.rst
      (Assertions): Turn example into a code block and change the 2nd
        expression into a more complicated expression rather than just
        using 2 equivalent values.
    waywardmonkeys committed Sep 20, 2015
  2. [docs] Explain about argument order for assertions.

    * documentation/users-guide/source/reference.rst
      (Assertions): Discuss argument order.
      (assert-instance?, assert-not-instance?): Provide a warning that
        the argument order here differs from the arguments for the
        instance? function.
    waywardmonkeys committed Sep 20, 2015
Commits on Sep 19, 2015
  1. Improve generated text for instance assertions.

    Instead of making it look like a call to instance?, just generate
    nice looking text. Also make it so that the instance? case is different
    from the not-instance? case.
    
    * assertions.dylan
      (macro assert-instance?, macro assert-not-instance?): Improve generated
        description text.
    waywardmonkeys committed Sep 19, 2015
  2. Correct generated description for instance? assertions.

    * assertions.dylan
      (macro assert-instance?, assert-not-instance?): Arguments shown in the
        generated description text were in the wrong order.
    waywardmonkeys committed Sep 19, 2015
Commits on Sep 17, 2015
  1. Merge pull request #74 from dylan-lang/assert-instance-or-not

    assert-instance?, assert-not-instance?
    cgay committed Sep 17, 2015
  2. assert-instance?, assert-not-instance?

    This brings assert-instance? and assert-not-instance? macros to the newer
    assert-based macros. These can be used to get better test failure output
    when an assertion fails (rather than assert-true(instance?(...)) or
    assert-false(instance?(...))).
    
    * assertions.dylan
      (do-check-instance?): Add negate? flag to handle ~instance? as well.
      (check-instance?): Pass #f for negate? to do-check-instance?.
      (assert-instance?, assert-not-instance?): New macros.
    
    * library.dylan
      (module testworks): Export assert-instance?, assert-not-instance?.
    
    * documentation/users-guide/source/reference.rst
      (assert-instance?, assert-not-instance?): Document.
      (check-instance?): Correct terminology: type, not class.
    
    * documentation/users-guide/source/usage.rst
      (Assertions): Link to new assertions.
    
    * tests/specification.dylan
      (module-spec testworks): Mention new assertions and stub tests.
    
    * tests/testworks-test-suite.dylan
      (testworks-assert-instance?-test,
       testworks-assert-not-instance-test): Test new assertions. We test these
         here to be consistent with the other assertion tests. In the future,
         this could move to the testworks specifications tests.
      (testworks-assertion-macros-suite): Run the new tests.
    waywardmonkeys committed Sep 16, 2015
Commits on Sep 16, 2015
  1. [documentation] Note return value meaning for expected-failure?.

    Carl pointed out that this should say that it is any true value
    and doesn't require #t being returned specifically.
    
    * documentation/users-guide/source/reference.rst
    waywardmonkeys committed Sep 16, 2015
  2. Merge pull request #73 from dylan-lang/expected-failure

    Add expected-failure support.
    cgay committed Sep 16, 2015
  3. Merge pull request #71 from dylan-lang/remove-assignment

    Remove assignment, just use value of expression.
    cgay committed Sep 16, 2015
Commits on Sep 15, 2015
  1. Add expected-failure support.

    It is useful to be able to note that a test is expected to fail
    and to note when it unexpectedly succeeds.  This will be useful
    for the Dylan test suites where we have some long-standing failures
    that can be flagged as expecting to fail so that we will notice
    new failures or other changes in the status of tests.
    
    Fixes bug #72.
    
    * components.dylan
      (%expected-failure?): New slot on <runnable>, parent class for
        both <test> and <benchmark>.
      (expected-failure?): New method to see if a test is expected
        to fail. This inspects %expected-failure? and evaluates the
        method if there is one.
    
    * results.dylan
      ($expected-failure, $unexpected-success): New status values.
      (<result-status>): Update to include new status values.
      (status-name): Update to handle new status values.
    
    * reports.dylan
      (count-results): Count $expected-failure as a pass, $unexpected-success
        as a failure.
    
    * coloring.dylan
      ($expected-failure-attributes, $unexpected-success-attributes): New
        text attributes.
      (result-status-to-attributes): Map new status values to new text attributes.
    
    * run.dylan
      (execute-component on <suite>): Count an $expected-failure result as
        a passing result.
      (execute-component on <test>): Check expected-failure? to handle
        $passed and $failed correctly.
    
    * library.dylan
      (module %testworks): Export $expected-failure and $unexpected-success.
    
    * documentation/users-guide/source/reference.rst
      (test-definer, benchmark-definer): Update to discuss expected-failure?.
    
    * documentation/users-guide/source/usage.rst
      (Tests): Update to discuss expected failures.
    
    * tests/testworks-test-suite.dylan: Update to test expected failure and
      unexpected success scenarios.
    waywardmonkeys committed Sep 15, 2015
  2. [documentation] Correctly note keyword parameters.

    * documentation/users-guide/source/reference.rst
      (suite-definer, test-definer, benchmark-definer): Note the keyword
        parameters correctly.
    waywardmonkeys committed Sep 15, 2015
  3. [documentation] Fix typo: identity.

    * documentation/users-guide/source/usage.rst: Fix typo for identity.
    waywardmonkeys committed Sep 15, 2015
  4. [documentation] Fix typo.

    * documentation/users-guide/source/usage.rst: Fix typo.
    waywardmonkeys committed Sep 15, 2015
Commits on Sep 14, 2015
  1. Remove assignment, just use value of expression.

    * run.dylan
      (execute-component): Just assign the value of the profiled expression
        to cond rather than first setting cond to #f and then assigning
        a new value to it.
    waywardmonkeys committed Sep 14, 2015
Commits on Sep 9, 2015
  1. Merge pull request #70 from housel/test-options

    testworks: Support passing options to tests
    waywardmonkeys committed Sep 9, 2015
  2. testworks: Support passing options to tests

    Sometimes test functions need to have information about external
    entities, such as the directory path of test or reference data files, or
    the hostname of a test database server. This commit allows this sort of
    information to be supplied on the command line of the test application.
    
    * run.dylan
      (runner-options): New constant slot in <test-runner>.
      (test-option): New function to return an option value if one is set.
    
    * command-line.dylan
      (make-runner-from-command-line): Add arguments of the form name=value
       to the runner-options table for retrieval by test-option.
    
    * library.dylan (module testworks): Export test-option.
    
    * tests/specification.dylan
      (module-spec testworks): Add test-option.
      (function-test test-option): New function-test.
    
    * documentation/users-guide/source/reference.rst: Document test-option.
    housel committed Sep 8, 2015
Commits on Aug 25, 2015
  1. [documentation] Update for new sphinx-extensions.

    This brings the opendylan-docs theme as well.
    waywardmonkeys committed Aug 25, 2015
Commits on Aug 12, 2015
  1. Merge pull request #65 from dylan-lang/colorize-output

    Colorizing output.
    cgay committed Aug 12, 2015
Commits on Aug 11, 2015
  1. Color output.

    * coloring.dylan: New file providing some color utility constants
        and a converter from a result status to text attributes.
    
    * library.dylan
      (library testworks): Use coloring-stream library.
      (module %testworks): Use coloring-stream module.
    
    * reports.dylan
      (print-result-summary): Update for color.
      (summary-report-function): Likewise.
    
    * run.dylan
      (<test-runner>): Make the runner-output-stream default to a coloring
        wrapper around *standard-output*.
      (various show-progress methods): Update for color.
    
    * testworks.lid: Build new coloring.dylan file.
    waywardmonkeys committed Aug 10, 2015
Commits on Aug 6, 2015
  1. Move *runner* definition and export it.

    * library.dylan
      (module testworks): Export *runner*.
    
    * utils.dylan
      (*runner*): Move from here ...
    
    * run.dylan
      (*runner*): ... to here.
    waywardmonkeys committed Aug 6, 2015
Commits on Jul 26, 2015
  1. Merge pull request #68 from dylan-lang/use-choice-options

    Use <choice-option> in command line parser.
    cgay committed Jul 26, 2015
  2. Use <choice-option> in command line parser.

    Using <choice-option> results in better error reporting.
    
    * command-line.dylan
      (general): Move some constants to the top of the file before
        they are used.
      (parse-args): Switch from <optional-parameter-option> and
        <parameter-option> to <choice-option>.
      (make-runner-from-command-line): Remove extraneous error handling
        now that <choice-option> is handling it.
    
    * tests/test-command-line.dylan
      (test-make-runner-from-command-line): Remove test of bare --debug
        for now as it doesn't work any longer.
    waywardmonkeys committed Jul 26, 2015
Commits on Jul 21, 2015
  1. Merge pull request #67 from housel/fix-benchmark-results

    testworks: Fix reporting support for benchmarks
    waywardmonkeys committed Jul 21, 2015
  2. testworks: Fix reporting support for benchmarks

    * results.dylan
      (result-type-name on <benchmark-result>): New method to provide
       a reporting label for benchmark results.
    housel committed Jul 21, 2015
Commits on Apr 26, 2015
  1. Merge pull request #64 from dylan-lang/more-frequent-output

    Output progress text more frequently.
    cgay committed Apr 26, 2015