Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Order of magnitude longer run time when using recent version of covr #274

Closed
pralitp opened this issue Aug 1, 2017 · 3 comments
Closed
Labels
bug an unexpected problem or unintended behavior

Comments

@pralitp
Copy link

pralitp commented Aug 1, 2017

We have been using covr + testthat + travis for continuous integration checks on our gcamdata project. This set up had been working great although recently the time taken to run covr in particular had gotten unbearable enough that we decided to look into it. Long story short if we revert back to some older version of covr (2.2.2 / adffd69) we see runtimes more inline with what we would expect; using the latest version (3.0.0.9000 / 4e6da57) takes one order of magnitude longer:

> library(covr)
> system.time(x <- package_coverage())
   user  system elapsed 
104.357  12.284 117.530 
> sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: macOS Sierra 10.12.6

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] covr_2.2.2

loaded via a namespace (and not attached):
[1] lazyeval_0.2.0 magrittr_1.5   rex_1.1.1      tools_3.3.2    withr_2.0.0   
> devtools::install_github("jimhester/covr")
Downloading GitHub repo jimhester/covr@master
from URL https://api.github.com/repos/jimhester/covr/zipball/master
Installing covr
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ --no-save --no-restore --quiet CMD INSTALL  \
  '/private/var/folders/db/txjmp0ms653f_z_6yj2hk8880000gn/T/RtmpNz9Qus/devtools58ad7a5fa036/jimhester-covr-4e6da57'  \
  --library='/Library/Frameworks/R.framework/Versions/3.3/Resources/library' --install-tests 

* installing *source* packagecovr...
** libs
clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG  -I/usr/local/include -I/usr/local/include/freetype2 -I/opt/X11/include    -fPIC  -Wall -mtune=core2 -g -O2  -c reassign.c -o reassign.o
clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o covr.so reassign.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
installing to /Library/Frameworks/R.framework/Versions/3.3/Resources/library/covr/libs
** R
** inst
** tests
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded
* DONE (covr)
Reloading installed covr
> system.time(x <- package_coverage())
    user   system  elapsed 
2098.471   79.524 2181.623 
> sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: macOS Sierra 10.12.6

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] covr_3.0.0.9000

loaded via a namespace (and not attached):
 [1] httr_1.2.1      R6_2.2.0        lazyeval_0.2.0  magrittr_1.5    rex_1.1.1       tools_3.3.2     withr_2.0.0    
 [8] curl_2.3        memoise_1.0.0   knitr_1.15.1    git2r_0.18.0    digest_0.6.10   devtools_1.12.0

Unfortunately I don't have much more insight than this as to what is going on. I have tried deleting all but the most minimal of tests and / or adding exclusions for the bulk of the code but in all cases the length of time to run the coverage tests are an order of magnitude longer than when running the full test using v2.2.2.

For the time being we will be reverting to using v2.2.2 but think it would still be a good idea to get to the bottom of this. Let me know if there is any more info I can provide to help diagnose what is going on.

@jimhester
Copy link
Member

I can confirm the performance regression. It has to due with how R stores the parseData for packages, which covr uses for creating source references for conditionals withthout curly braces.

I will work on a fix in the near term.

@jimhester jimhester added the bug an unexpected problem or unintended behavior label Aug 10, 2017
@jimhester
Copy link
Member

This should now be fixed, thank you for reporting the issue!

@pralitp
Copy link
Author

pralitp commented Aug 15, 2017

Thanks for the quick fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug an unexpected problem or unintended behavior
Projects
None yet
Development

No branches or pull requests

2 participants