Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

please keep cl-test-grid informed about testsuite changes #10

Open
avodonosov opened this issue Nov 9, 2013 · 0 comments
Open

please keep cl-test-grid informed about testsuite changes #10

avodonosov opened this issue Nov 9, 2013 · 0 comments

Comments

@avodonosov
Copy link

Hello.

As cl-fad is one of the most often downloaded Quicklisp systems, cl-fad testsuite is being constantly tested by cl-test-grid on various lisps and OSes:
http://common-lisp.net/project/cl-test-grid/library/cl-fad.html

When cl-fad test-suite changes it breaks cl-test-grid adapter which needs to be updated (there were already two such cases: once when the test code has moved from test.lisp to fad.test.lisp, next when the test code was separated into several files and organized into an ASDF system).

It is definitely good that the testsuite is being extended.

I would appreciate if cl-fad developers drop a note to cl-test-grid@googlegroups.com when they start planning or implementing the testsuite changes. Otherwise some time passes before I notice the breakage and also some time is needed to implement the adjustment; during this time cl-fad is not covered by cl-test-grid.

Let me also make some notes about the current testsuite state.

The part of the tests defined in the fad.test.lisp is implemented using cl:assert.
Another part, defined in temporary-files.test.lisp is implemented using the unit-test framework and not integrated into the documented function cl-fad-test:test.

A disadvantage of the unit-test framework is that it's main function
unit-test:run-all-tests return value does not provide list of failed test cases,
it only returns true/false meaning "all passed" or "some failed".

It would be better if all tests were implemented using the same test framework,
preferably one allowing to determine what test cases has failed (clunit, eos, fiveam, lift, nst, rt, stefil, ...).

These problems are not critical, and of course I do not expect anyone to immediately jump in and start spending efforts on it. I just hope these notes may be useful if someone if the future will work with the tests.

Best regards,

  • Anton
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant