Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

make memory profiling optional and fail-safe. #63

Closed
wants to merge 2 commits into from
Closed

make memory profiling optional and fail-safe. #63

wants to merge 2 commits into from

Conversation

fabian-s
Copy link
Contributor

closes #62

Also modifies memory allocation reporting:
allocated bytes are NA instead of 0 if memory profiling was not done or failed.

library(bench)


keep_busy <- function(n = 1e3) {
  r <- rnorm(n)
  p <- pnorm(r)
  q <- qnorm(p)
  o <- order(q)
}
mark(keep_busy())
#> # A tibble: 1 x 6
#>   expression       min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr>  <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 keep_busy()    307µs    322µs     3079.    38.3KB     7.67

mark(parallel::mclapply(seq_len(1e3), keep_busy, mc.cores = 8))
#> Warning in value[[3L]](cond): memory profiling failed.
#> # A tibble: 1 x 6
#>   expression                                                    min median
#>   <bch:expr>                                                 <bch:> <bch:>
#> 1 parallel::mclapply(seq_len(1000), keep_busy, mc.cores = 8) 45.4ms   49ms
#> # … with 3 more variables: `itr/sec` <dbl>, mem_alloc <bch:byt>,
#> #   `gc/sec` <dbl>

# switch off memory profiling globally:
mark(1, 2, check = NULL, memory = FALSE, iterations = 1)
#> # A tibble: 2 x 6
#>   expression      min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 1            70.1ns   70.1ns 14268994.        NA        0
#> 2 2                 0        0      Inf         NA      NaN

# switch off memory profiling per expresssion:
mark(1, 2, check = NULL, memory = c(FALSE, TRUE), iterations = 1)
#> # A tibble: 2 x 6
#>   expression      min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 1              71ns     71ns 14081860.        NA        0
#> 2 2              70ns     70ns 14292736.        0B        0

Created on 2019-10-21 by the reprex package (v0.3.0)

Session info 
devtools::session_info()
#> ─ Session info ──────────────────────────────────────────────────────────
#>  setting  value                       
#>  version  R version 3.6.1 (2019-07-05)
#>  os       Linux Mint 19.1             
#>  system   x86_64, linux-gnu           
#>  ui       X11                         
#>  language en_GB                       
#>  collate  en_GB.UTF-8                 
#>  ctype    en_GB.UTF-8                 
#>  tz       Europe/Berlin               
#>  date     2019-10-21                  
#> 
#> ─ Packages ──────────────────────────────────────────────────────────────
#>  package     * version    date       lib source        
#>  assertthat    0.2.1      2019-03-21 [1] CRAN (R 3.6.1)
#>  backports     1.1.5      2019-10-02 [1] CRAN (R 3.6.1)
#>  bench       * 1.0.4.9001 2019-10-21 [1] local         
#>  callr         3.3.2      2019-09-22 [1] CRAN (R 3.6.1)
#>  cli           1.1.0      2019-03-19 [1] CRAN (R 3.6.1)
#>  crayon        1.3.4      2017-09-16 [1] CRAN (R 3.6.1)
#>  desc          1.2.0      2018-05-01 [1] CRAN (R 3.6.1)
#>  devtools      2.2.1      2019-09-24 [1] CRAN (R 3.6.1)
#>  digest        0.6.21     2019-09-20 [1] CRAN (R 3.6.1)
#>  ellipsis      0.3.0      2019-09-20 [1] CRAN (R 3.6.1)
#>  evaluate      0.14       2019-05-28 [1] CRAN (R 3.6.1)
#>  fansi         0.4.0      2018-10-05 [1] CRAN (R 3.6.1)
#>  fs            1.3.1      2019-05-06 [1] CRAN (R 3.6.1)
#>  glue          1.3.1      2019-03-12 [1] CRAN (R 3.6.1)
#>  highr         0.8        2019-03-20 [1] CRAN (R 3.6.1)
#>  htmltools     0.4.0      2019-10-04 [1] CRAN (R 3.6.1)
#>  knitr         1.25       2019-09-18 [1] CRAN (R 3.6.1)
#>  magrittr      1.5        2014-11-22 [1] CRAN (R 3.6.1)
#>  memoise       1.1.0      2017-04-21 [1] CRAN (R 3.6.1)
#>  pillar        1.4.2      2019-06-29 [1] CRAN (R 3.6.1)
#>  pkgbuild      1.0.5      2019-08-26 [1] CRAN (R 3.6.1)
#>  pkgconfig     2.0.3      2019-09-22 [1] CRAN (R 3.6.1)
#>  pkgload       1.0.2      2018-10-29 [1] CRAN (R 3.6.1)
#>  prettyunits   1.0.2      2015-07-13 [1] CRAN (R 3.6.1)
#>  processx      3.4.1      2019-07-18 [1] CRAN (R 3.6.1)
#>  profmem       0.5.0      2018-01-30 [1] CRAN (R 3.6.1)
#>  ps            1.3.0      2018-12-21 [1] CRAN (R 3.6.1)
#>  R6            2.4.0      2019-02-14 [1] CRAN (R 3.6.1)
#>  Rcpp          1.0.2      2019-07-25 [1] CRAN (R 3.6.1)
#>  remotes       2.1.0      2019-06-24 [1] CRAN (R 3.6.1)
#>  rlang         0.4.0      2019-06-25 [1] CRAN (R 3.6.1)
#>  rmarkdown     1.15       2019-08-21 [1] CRAN (R 3.6.1)
#>  rprojroot     1.3-2      2018-01-03 [1] CRAN (R 3.6.1)
#>  sessioninfo   1.1.1      2018-11-05 [1] CRAN (R 3.6.1)
#>  stringi       1.4.3      2019-03-12 [1] CRAN (R 3.6.1)
#>  stringr       1.4.0      2019-02-10 [1] CRAN (R 3.6.1)
#>  testthat      2.2.1      2019-07-25 [1] CRAN (R 3.6.1)
#>  tibble        2.1.3      2019-06-06 [1] CRAN (R 3.6.1)
#>  usethis       1.5.1      2019-07-04 [1] CRAN (R 3.6.1)
#>  utf8          1.1.4      2018-05-24 [1] CRAN (R 3.6.1)
#>  vctrs         0.2.0      2019-07-05 [1] CRAN (R 3.6.1)
#>  withr         2.1.2      2018-03-15 [1] CRAN (R 3.6.1)
#>  xfun          0.10       2019-10-01 [1] CRAN (R 3.6.1)
#>  yaml          2.2.0      2018-07-25 [1] CRAN (R 3.6.1)
#>  zeallot       0.1.0      2018-01-28 [1] CRAN (R 3.6.1)
#> 
#> [1] /home/lmmista-wap218/R/x86_64-pc-linux-gnu-library/3.6
#> [2] /usr/local/lib/R/site-library
#> [3] /usr/lib/R/site-library
#> [4] /usr/lib/R/library

@fabian-s
make memory profiling optional and fail-safe.
6e123a9 
close #62.
modify memory allocation reporting:
NA instead of 0 if not done or failed.
@fabian-s
bump version + add contributor FS
ffa59b7
jimhester
jimhester reviewed Oct 21, 2019
View reviewed changes
R/mark.R
@@ -22,6 +22,8 @@ NULL
#' with [all.equal()], if `FALSE` checking is disabled. If `check` is a
#' function that function will be called with each pair of results to
#' determine consistency.
#' @param memory Profile memory allocation? Defaults to `TRUE`. Can also be a
#' vector to switch memory profiling on or off for each element in `exprs`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to support a vector of values, just a global on or off.

R/mark.R
results$result[[i]] <- res$result
}
results$memory[[i]] <- res$memory
if (profile_memory[i] | isTRUE(check)) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (profile_memory[i] | isTRUE(check)) {
if (profile_memory[[i]] || isTRUE(check)) {

R/mark.R
@@ -231,7 +245,14 @@ summary.bench_mark <- function(object, filter_gc = TRUE, relative = FALSE, time_

object$mem_alloc <-
bench_bytes(
vdapply(object$memory, function(x) if (is.null(x)) NA else sum(x$bytes, na.rm = TRUE)))
vdapply(object$memory, function(x) {
if (is.null(x) || x$trace[1] == "*FAILED*") {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When would trace == *FAILED*?

@jimhester jimhester closed this in fce1e23 Jan 8, 2020
@jimhester
Copy link
Member

Thanks for working on this!

I ended up implementing this somewhat differently, but I adapted your test so made you a co-author on the commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jimhester jimhester jimhester left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

feature request: option to switch off memory profiling
2 participants
@fabian-s @jimhester