dontrun or donttest or what? #1798
Comments
|
Based on section 2.1.1 of https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Writing-R-documentation-files, I am starting to think we should change every I am a bit worried that Hadley Wickham (r-lib/devtools#2216 (comment)) seems to suggest that doing so might invite CRAN rejection. Still, I think this is worth the risk ... at least then we'll know more about the system. Relevant portion of docsThe following is from section 2.1.1 of https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Writing-R-documentation-files; note that it will be formatted poorly, so your best plan is to go to the stated URL. \examples{…} Examples are not only useful for documentation purposes, but also provide test code used for diagnostic checking of R code. By default, text inside \examples{} will be displayed in the output of the help page and run by example() and by R CMD check. You can use \dontrun{} for text that should only be shown, but not run, and \dontshow{} for extra commands for testing that should not be shown to users, but will be run by example(). (Previously this was called \testonly, and that is still accepted.) Text inside \dontrun{} is ‘verbatim’, but the other parts of the \examples section are R-like text. For example, x <- runif(10) # Shown and run. Example code must be capable of being run by example, which uses source. This means that it should not access stdin, e.g. to scan() data from the example file. Data needed for making the examples executable can be obtained by random number generation (for example, x <- rnorm(100)), or by using standard data sets listed by data() (see ?data for more info). Finally, there is \donttest, used (at the beginning of a separate line) to mark code that should be run by example() but not by R CMD check (by default: the option --run-donttest can be used). This should be needed only occasionally but can be used for code which might fail in circumstances that are hard to test for, for example in some locales. (Use e.g. capabilities() or nzchar(Sys.which("someprogram")) to test for features needed in the examples wherever possible, and you can also use try() or tryCatch(). Use interactive() to condition examples which need someone to interact with.) Note that code included in \donttest must be correct R code, and any packages used should be declared in the DESCRIPTION file. It is good practice to include a comment in the \donttest section explaining why it is needed. Output from code between comments IGNORE_RDIFF_BEGINIGNORE_RDIFF_ENDis ignored when comparing check output to reference output (a -Ex.Rout.save file). This markup can also be used for scripts under tests. |
|
FYI, this is the number of places where we use |
|
I will be resubmitting oce to CRAN, likely this Friday, to get around a problem with the present version of "sf" on CRAN (#1795). We found out about this "sf" problem because Ripley's machine (and maybe now other machines) are running \donttest{} blocks. We use \donttest{} blocks to illustrate the use of non-provided datasets (or to avoid very slow tests) on 35 occasions within oce. I think I ought to convert each of these to \dontrun{} blocks. However, I am a bit worried, because Wickham has said (in another context ... see above) that using \dontrun{} might invite package rejection. QUESTION: Do you think I ought to switch the \donttest{} to \dontrun{}? |
|
The amazingly helpful @hadley has clarified that \dontrun{} causing rejection is mainly a worry for new submissions. So I'm going to go ahead and change to \dontrun{} all instances where we use \donttest{} because of non-provided data, whilst retaining those that are about slow operations. I'll need to look at the instances one by one. Below is a checklist, created with The following codes show what I've done:
Checklist
|
|
Sounds like you have it under control! Another strategy you can use is wrapping examples conditionally like: if (some_test) {
# do the example
}Mostly this is useful because then pkgdown will run your examples if |
|
From my reading of the docs (section 2.1.1 of https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Writing-R-documentation-files) and my understanding of CRAN policies (which give a NOTE on slow examples), it seems to me that I ought to retain \donttest for slow-running examples. Do you think that's right, @paleolimbot? |
|
I think so. In addition to CRAN, it's useful in real life because |
See #1798, which was inspired by a problem that was found within a donttest block (#1795) and thus indicated that CRAN is now running code within donttest blocks. So, we ought to use those blocks only for slow code that we know works. For code that won't work, e.g. because of the assumption of data not provided with the package, we should use dontrun.
|
I think this is fine now in "develop" commit fb18a76, as evinced by
|
We have quite a few spots in documentation where
\donttestis used to prevent testing, because the test involves data not provided with oce.I think all of these might fail now, because CRAN is testing things inside
\donttestblocks. I say that because the note I got from CRAN about oce being on the verge of deletion from CRAN had its problem in a new link called "donttest". (See snapshot below. Those NOTEs were not why oce is in trouble ... it's the new link called 'donttest'.)The question is how we are now meant to "hide" examples that will not work on a test machine, e.g. because they need data not provided with oce, or because they are too slow (since CRAN has a limit on execution time of examples).
I need to look into this as a high priority, because I think we have a lot of things within
dontestblocks. Starting points might be as follows.Snapshot of oce check page
The text was updated successfully, but these errors were encountered: