What is the purpose of #include section in #testing?

[peteruhnak]

What is the purpose of the #include section?

    #categories : [ 'MyProject-*' ],                          // Define categories to test explicitly
    #classes : [ #MyProjectTestCase ],                        // Define classes to test explicitly
    #packages : [ 'MyProject.*' ]                             // Define Pharo packages to test
    #include : {
        #categories : [ 'AnotherProject-Tests' ],               // Include categories to test
        #classes : [ #AnotherProjectTestCase ],                 // Include classes to test
        #packages : [ 'AnotherProject.*' ]                      // Include Pharo packages to test
    },

Isn't it equal to having it directly in the array?

    #categories : [ 'MyProject-*', 'AnotherProject-Tests' ],                          // Define categories to test explicitly
    #classes : [ #MyProjectTestCase, #AnotherProjectTestCase ],                        // Define classes to test explicitly
    #packages : [ 'MyProject.*', 'AnotherProject.*' ]                             // Define Pharo packages to test

[peteruhnak]

Nevermind, apparently the second variant doesn't work (even though the array syntax would suggest otherwise.)

[fniephaus]

The #testing section currently works as follows:

1. If not set, determine tests to run automatically (just like builderCI)
2. If #classes or #categories are set, use these for testing and ignore everything else
3. If #include or #exclude are set, determine tests to run automatically and add or remove tests accordingly. Note that 2. should not be the case (#classes and #categories should not be set).

I hope the above makes sense.

[fniephaus]

If you have an idea how to improve the doc, please send a PR 😃

[peteruhnak]

My point was different:

if I have

#packages : [ 'Foo.*' ]

then Foo is tested as expected ( https://travis-ci.org/peteruhnak/ci-foo/builds/109091321 )

however if I have

#packages : [ 'Foo.*', 'Bar.*' ]

then only Foo is tested, and not Bar ( https://travis-ci.org/peteruhnak/ci-foo/builds/109091408 )

So I have to do

#packages : [ 'Foo.*' ]
#include: {
  #packages : [ 'Bar.*' ]
}

to have them tested both ( https://travis-ci.org/peteruhnak/ci-foo/builds/109091536 )

So my question is: why does the second variant not work?

My impression is that the first #packages is based on LoadSpecs and ignores what I actually write there (because it will test also tests from different packages that are part of the project but do not match the regex).

[peteruhnak]

Just to confirm, are my observations about the default package: not really having effect correct? (i.e. that they will use LoadSpecs regardless of what I put there.)

[fniephaus]

Sorry for leaving this unanswered for so long.
I understand that

#packages : [ 'Foo.*', 'Bar.*' ]

only causes Foo to be tested and not Bar? Is this the current behavior? If so, it's a bug.

[peteruhnak]

That was the case as you can see in the linked travis builds.

I seem to have deleted the github repos, but it should be easy to setup again…

[fniephaus]

That's my bad I guess. But yes, shouldn't be to hard :) I'll look into it in the next few days.

[fniephaus]

@peteruhnak you were absolutely right, explicit #packages have been disregarded completely. A fix should be very simple...