ceylon.test: run Callables as tests

[lucaswerkmeister]

This allows to run Anything()s as tests. If you pack them into a tuple, you can also name them.

With this change, you can run a test function several times with different parameters, like this:

"Runs [[testFile]] for each file in [[testFileDirectory]]."
TestRunner runner = createTestRunner([
    for (file in testFileDirectory.files())
        [() => testFile(file), file.name]
        // or, if you don’t care about the name:
        // () => testFile(file)
]);

The code for the CallableTestExecutor is mostly copied from the MethodTestExecutor, with a lot of stuff that’s not possible on Callables (callbacks from annotations etc.) removed.

What do you think?

[thradec]

Hi Lucas

I'm not sure if this approach is correct. Interface TestRunner respectively its factory method createTestRunner weren't planned to be used directly by developer, its role is similar like JUnitCore, which is only facade used by various tools.

I think that developers will most often run tests from ide (eg. selecting module/package/... and use Run as > Ceylon Test) or command line via ceylon test tool. Both ways use test annotation for finding tests. How will this work in your case?

In the near future, I would like to add some support "data providers", something like junit parametrized tests or theory, but in more ceylonish way. Maybe this will solve your use case.

[lucaswerkmeister]

Both ways use test annotation for finding tests. How will this work in your case?

That’s the problem – both don’t support my use case (writing a tool that processes files, test it using input + expected output file pairs). Currently, I can either write a method for each file pair (which I’m doing here), or I can write one method which loops over the the files (with the disadvantage that I can’t easily see which test failed, since it’s only one test).

In the near future, I would like to add some support "data providers", something like junit parametrized tests or theory, but in more ceylonish way. Maybe this will solve your use case.

Probably, yes. That would be nice. But is there any particular reason why you shouldn’t be able to run arbitrary Callables as tests? I think that it could still be useful in some cases if the developer wants to use TestRunner directly.

[thradec]

The only reason is that when developer run ceylon test foo, he expect, that all tests in module will be executed, and this behavior would not be possible, if we couldn't identify them.

[thradec]

I looked at your code and with "data providers" should be possible write something like this.

{File*} testFiles() => testFileDirectory.files("*.ceylon");

test
void shouldFormatCode(dataProvider(`testFiles`) File f) {
    ...
}

[lucaswerkmeister]

The only reason is that when developer run ceylon test foo, he expect, that all tests in module will be executed, and this behavior would not be possible, if we couldn't identify them.

I was about to argue that this would be a totally unreasonable assumption that no one would ever make because it doesn’t make sense and bla bla – until I remembered that I did try this out, and I did kind of hope that it would work :D So I guess you’re right there.

The data providers look nice, are they coming soon?

[gavinking]

in more ceylonish way

It's "ceylonic", dammit ;-)

[ncorai]

It's "ceylonic", dammit ;-)

Not in Czech, it isn't

[gavinking]

haha

[thradec]

It's "ceylonic", dammit ;-)

Say it to Tako, he introduced this word into my dictionary :-), see [1] and [2]. In Czech it would be "ceyloní" ;-).

[2] https://groups.google.com/d/msg/ceylon-dev/BcDNMVpGa80/bFftKrw6DBIJ
[1] http://stackoverflow.com/questions/20184367/infinite-iterable-generator-in-ceylon/20240118#20240118

[thradec]

The data providers look nice, are they coming soon?

Hopefully into 1.1. I'm waiting to this issue ceylon/ceylon-spec#847.

[thradec]

@lucaswerkmeister Can I close this pull request and open new issue in sdk for support "data providers"?

[lucaswerkmeister]

Can I close this pull request and open new issue in sdk for support "data providers"?

Yeah, alright. I think I’ll keep the branch in my fork, but I’ll be happy with data providers.

It's "ceylonic", dammit ;-)

Compromise: ceylonicish code.

class Klazz() {
    variable shared String name = "hi";

    shared String getName() => name;
    shared void setName(String newName) => name = newName;
}

Kinda ceylonic, kinda not :-D