Some tests are not referenced in a test suite #300

wants to merge 1 commit into


None yet
3 participants

divdavem commented Dec 19, 2012

This pull request adds the following tests in the corresponding suite and fixes them (so that the build is successful):

  • test.aria.templates.issue142.HtmlStyleTemplateTestCase
  • test.aria.templates.issue279.ButtonSpacingTestCase

divdavem closed this in 9956682 Jan 9, 2013


piuccio commented Jan 9, 2013

I've integrated the changes with commit 9956682 but we should have a task that detects orphan tests.
I'm thinking about a grunt task...

piuccio reopened this Jan 9, 2013


jakub-g commented Feb 20, 2013

I'm wondering what would be the best solution for this.

We can't rely on file names because the majority of the test cases doesn't have any Test part in their name.

We could list all .js files in test dir, and filter only those which contain the magic words to be the test case (because some 30% are tpl scripts and other util JS files used by tests etc. so we need to exclude them).

The best way to check for this would be to look at $extends because each test case/suite extends, directly or not, the base test case/suite file.

The problems are with indirect inheritance.

I've quickly analyzed the contents of the folder and we have test cases/suites that extend from:

$extends : 'aria.jsunit.TestCase',
$extends : 'aria.jsunit.RobotTestCase',
$extends : 'aria.jsunit.TemplateTestCase',
$extends : "aria.jsunit.ModuleCtrlTestCase",
$extends : "aria.jsunit.WidgetTestCase",

$extends : "test.aria.widgets.container.issue80.shared.BindableSizeTestCase",
$extends : "test.aria.widgets.form.multiselect.issue312.Common",

and all test suites extend from

$extends : "aria.jsunit.TestSuite",

Regarding test cases, the last two are families of very specialized per-issue test cases inheriting from the base test cases and differing only in widgets under tests.

So if we renamed the last file say to Issue312TestCase then we could analyze the contents of the files looking for match for $extends : *TestCase match (either with regex or AST).

Then we need to obtain all the tests added by this.addTests( ) in the suites and compare the two arrays.


jakub-g commented Feb 20, 2013

I'm in favor of wildcard match, because we have e.g. aria.jsunit.MultiSelectTemplateTestCase in CC, and we may potentially add new in the future, so a bulletproof solution will be preferred.

BTW we should add some marker to BindableSizeTestCase and issue312.Common saying that those classes are abstract test cases ($abstract : true ?) and exclude them.


jakub-g commented Feb 20, 2013

We can also go the other way and rename all test files to end with Test / TestCase - this will at least not need to read 300 files from disk to do the check...


piuccio commented Feb 20, 2013

The best would be to remove the TestSuite altogether, they are just a duplication of information.
The test runner should be able to understand which files are test cases and run them all.


jakub-g commented Jan 14, 2014

I'm closing this as this is an old specific pull request, I opened a new issue #924 for tracking the general issue (I don't like pull request to be visible as open while they were merged).

jakub-g closed this Jan 14, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment