Tests under xt/ should not execute per default #278
Comments
|
If xt tests don't run during release, when would they run? You are the author and releaser, so surely you'd want to know about failures at release time. |
This one is surprising to me -- 'minil release' should be setting AUTHOR_TESTING=1 so this test can run. |
|
@karenetheridge According to which spec should tests under xt run? Or, opposite question, where should I place tests which need external huge data (here GoldStandard), compare each iteration with the result of the original version, do extensive benchmarks--both speed and f-score--etc., generate very huge permutions of combinations? Really, should they be performed during build and release of the package? They are module-specific development tools, to be used by developers. They are not targeting the average user, thus not scripts/ or examples/. |
|
I think it is reasonable that Minilla runs test in xt. If you don't want Minilla to run some tests, |
|
@wollmers: this is covered in the Lancaster Consensus document: https://github.com/Perl-Toolchain-Gang/toolchain-site/blob/master/lancaster-consensus.md#environment-variables-for-testing-contexts (the placement of files into xt/ is not explicitly covered, but convention is that tests there are in subdirs named similarly for the environment variables that would protect them, e.g. xt/author/, xt/release/ etc.) |
|
milla/dzil doesn't run every xt/*.t, so something must be wrong: the spec, milla/dzil or minil. Of course I could (and should) skip them. Just to be sure, that milla/dzil do not run them and minil does: |
|
@karenetheridge Thanks for the link. I knew about it (participated Berlin 2015), but it's hard to find. From Lancaster Consensus:
and
For me "author's personal development process" implies running at authors will. With such convenience tools like Milla and Minilla the author delegates the control more or less to the tool. That's fine, if the tools work as expected, which they do in most of the cases. The Lancaster Consensus is not clear about the exact handling of author tests, nor does it provide rules for including or excluding the subdirectories Roughly Milla executes A better solution would be, that the related tools Zilla, Milla, Minilla and others agree to the same default behaviour and semantics of testing options. |
I would not expect that tests under
xt/would be executed during build or whatever per default.Migrated from Dist::Milla and tried to release:
The text was updated successfully, but these errors were encountered: