Skip to content

Commit

Permalink
Merge branch 'release/v1.0.2'
Browse files Browse the repository at this point in the history
  • Loading branch information
Saito, Takaya authored and Saito, Takaya committed Feb 10, 2022
2 parents d7e0a0b + 6bb4e5f commit 40cf25a
Show file tree
Hide file tree
Showing 78 changed files with 266 additions and 9,356 deletions.
2 changes: 2 additions & 0 deletions .Rbuildignore
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,5 @@
^_pkgdown\.yml$
^scripts$
^CRAN-RELEASE$
^\.github$
^CRAN-SUBMISSION$
1 change: 1 addition & 0 deletions .github/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
*.html
47 changes: 47 additions & 0 deletions .github/workflows/R-CMD-check.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Workflow derived from https://github.com/r-lib/actions/tree/v2/examples
# Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help
on:
push:
branches: [main, _develop]
pull_request:
branches: [main, _develop]

name: R-CMD-check

jobs:
R-CMD-check:
runs-on: ${{ matrix.config.os }}

name: ${{ matrix.config.os }} (${{ matrix.config.r }})

strategy:
fail-fast: false
matrix:
config:
- {os: macOS-latest, r: 'release'}
- {os: windows-latest, r: 'release'}
- {os: ubuntu-latest, r: 'devel', http-user-agent: 'release'}
- {os: ubuntu-latest, r: 'release'}
- {os: ubuntu-latest, r: 'oldrel-1'}

env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
R_KEEP_PKG_SOURCE: yes

steps:
- uses: actions/checkout@v2

- uses: r-lib/actions/setup-pandoc@v2

- uses: r-lib/actions/setup-r@v2
with:
r-version: ${{ matrix.config.r }}
http-user-agent: ${{ matrix.config.http-user-agent }}
use-public-rspm: true

- uses: r-lib/actions/setup-r-dependencies@v2
with:
extra-packages: any::rcmdcheck
needs: check

- uses: r-lib/actions/check-r-package@v2
35 changes: 35 additions & 0 deletions .github/workflows/pkgdown.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Workflow derived from https://github.com/r-lib/actions/tree/master/examples
# Need help debugging build failures? Start at https://github.com/r-lib/actions#where-to-find-help
on:
push:
branches: [main, _develop]
release:
types: [published]
workflow_dispatch:

name: pkgdown

jobs:
pkgdown:
runs-on: ubuntu-latest
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v2

- uses: r-lib/actions/setup-pandoc@v1

- uses: r-lib/actions/setup-r@v1
with:
use-public-rspm: true

- uses: r-lib/actions/setup-r-dependencies@v1
with:
extra-packages: pkgdown
needs: website

- name: Deploy package
run: |
git config --local user.name "$GITHUB_ACTOR"
git config --local user.email "$GITHUB_ACTOR@users.noreply.github.com"
Rscript -e 'pkgdown::deploy_to_branch(new_process = FALSE)'
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ java/auc2/bin
java/auc2/.idea
java/auc2/*.iml
/.project
docs
4 changes: 2 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

branches:
only:
- main
# - develop
- _main
- _develop

os:
- linux
Expand Down
15 changes: 9 additions & 6 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
Package: prcbench
Type: Package
Title: Testing Workbench for Precision-Recall Curves
Version: 1.0.1
Date: 2021-04-06
Version: 1.0.2
Date: 2022-02-03
Authors@R: c(
person("Takaya", "Saito", email = "takaya.saito@outlook.com", role = c("aut", "cre")),
person("Marc", "Rehmsmeier", email = "marc.rehmsmeier@ii.uib.no", role = c("aut")))
Description: A testing workbench for evaluating precision-recall curves under various conditions.
person("Takaya", "Saito", email = "takaya.saito@outlook.com", role = c("aut", "cre"),
comment = c(ORCID = "0000-0002-0154-8452")),
person("Marc", "Rehmsmeier", email = "marc.rehmsmeier@ii.uib.no", role = c("aut"),
comment = c(ORCID = "0000-0002-5021-7721")))
Description: A testing workbench to evaluate tools that calculate precision-recall curves.
Saito and Rehmsmeier (2015) <doi:10.1371/journal.pone.0118432>.
URL: https://evalclass.github.io/prcbench/, https://github.com/evalclass/prcbench
BugReports: https://github.com/evalclass/prcbench/issues
Depends:
Expand All @@ -18,7 +21,6 @@ Imports:
ROCR (>= 1.0-7),
PRROC (>= 1.1),
precrec (>= 0.1),
rJava (>= 0.9-7),
R6 (>= 2.1.1),
assertthat (>= 0.1),
grid,
Expand All @@ -31,6 +33,7 @@ RoxygenNote: 7.1.1
Encoding: UTF-8
Suggests: microbenchmark (>= 1.4-2.1),
PerfMeas (>= 1.2.1),
rJava (>= 0.9-7),
testthat (>= 0.11.0),
knitr (>= 1.11),
rmarkdown (>= 0.8.1)
Expand Down
26 changes: 23 additions & 3 deletions R/tool_zzz.R
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,10 @@ ToolAUCCalculator <- R6::R6Class(
private$jarpath <- arglist[["jarpath"]]
}
}
if (private$available && !requireNamespace("rJava", quietly = TRUE)) {
print("rJava is not available.")
private$available = FALSE
}
private$f_setjar()
},

Expand Down Expand Up @@ -278,6 +282,7 @@ ToolAUCCalculator <- R6::R6Class(
),
private = list(
toolname = "AUCCalculator",
available = TRUE,
curvetype = "SPR",
auctype = "java",
print_methods = function() {
Expand All @@ -293,12 +298,27 @@ ToolAUCCalculator <- R6::R6Class(
} else {
jarpath <- private$jarpath
}
private$auc2 <- .get_java_obj("auc2", jarpath, private$curvetype)
if (private$available) {
private$auc2 <- .get_java_obj("auc2", jarpath, private$curvetype)
} else {
private$auc2 <- NA
}
},
f_wrapper = function(testset, calc_auc, store_res) {
.auccalc_wrapper(testset, private$auc2, calc_auc, store_res,
private$auctype)
if (private$available) {
.auccalc_wrapper(testset, private$auc2, calc_auc, store_res,
private$auctype)
} else {
if (store_res) {
x <- seq(0.0, 1.0, 0.1)
y <- rep(0.5, length(x))
list(x = x, y = y, auc = 0.5)
} else {
NULL
}
}
}

)
)

Expand Down
2 changes: 1 addition & 1 deletion README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ output:

# prcbench <img src="man/figures/logo.png" align="right" alt="" width="100" />

[![Travis](https://travis-ci.org/evalclass/prcbench.svg?branch=main)](https://travis-ci.org/evalclass/prcbench/)
[![R-CMD-check](https://github.com/evalclass/prcbench/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/evalclass/prcbench/actions/workflows/R-CMD-check.yaml)
[![AppVeyor Build Status](https://ci.appveyor.com/api/projects/status/github/evalclass/prcbench?branch=main&svg=true)](https://ci.appveyor.com/project/takayasaito/prcbench/)
[![codecov.io](https://codecov.io/github/evalclass/prcbench/coverage.svg?branch=main)](https://codecov.io/github/evalclass/prcbench?branch=main)
[![CodeFactor](https://www.codefactor.io/repository/github/evalclass/prcbench/badge)](https://www.codefactor.io/repository/github/evalclass/prcbench/)
Expand Down
32 changes: 11 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@

# prcbench <img src="man/figures/logo.png" align="right" alt="" width="100" />

[![Travis](https://travis-ci.org/evalclass/prcbench.svg?branch=main)](https://travis-ci.org/evalclass/prcbench/)
[![R-CMD-check](https://github.com/evalclass/prcbench/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/evalclass/prcbench/actions/workflows/R-CMD-check.yaml)
[![AppVeyor Build
Status](https://ci.appveyor.com/api/projects/status/github/evalclass/prcbench?branch=main&svg=true)](https://ci.appveyor.com/project/takayasaito/prcbench/)
[![codecov.io](https://codecov.io/github/evalclass/prcbench/coverage.svg?branch=main)](https://codecov.io/github/evalclass/prcbench?branch=main)
[![CodeFactor](https://www.codefactor.io/repository/github/evalclass/prcbench/badge)](https://www.codefactor.io/repository/github/evalclass/prcbench/)
[![CRAN\_Status\_Badge](https://www.r-pkg.org/badges/version-ago/prcbench)](https://cran.r-project.org/package=prcbench)
[![CRAN\_Logs\_Badge](https://cranlogs.r-pkg.org/badges/grand-total/prcbench)](https://cran.r-project.org/package=prcbench)
[![CRAN_Status_Badge](https://www.r-pkg.org/badges/version-ago/prcbench)](https://cran.r-project.org/package=prcbench)
[![CRAN_Logs_Badge](https://cranlogs.r-pkg.org/badges/grand-total/prcbench)](https://cran.r-project.org/package=prcbench)

The aim of the `prcbench` package is to provide a testing workbench for
evaluating precision-recall curves under various conditions. It contains
Expand Down Expand Up @@ -43,7 +43,7 @@ predefined test data sets.

### Java

`AUCCalculator` requires a Java runtime (&gt;= 6).
`AUCCalculator` requires a Java runtime (>= 6).

### Bioconductor libraries

Expand Down Expand Up @@ -137,23 +137,18 @@ library(prcbench)
testset <- create_testset("bench", "b10")
toolset <- create_toolset(set_names = "auc5")
res <- run_benchmark(testset, toolset)
```

## [1] "microbenchmark is not available. system.time will be used instead."
## [1] "PerfMeas is not available."

``` r
## Use knitr::kable to show the result in a table format
knitr::kable(res$tab, digits = 2)
```

| testset | toolset | toolname | min | lq | mean | median | uq | max | neval |
|:--------|:--------|:--------------|----:|----:|-----:|-------:|----:|----:|------:|
| b10 | auc5 | AUCCalculator | 4 | 5 | 7 | 11.2 | 9 | 31 | 5 |
| b10 | auc5 | PerfMeas | 0 | 0 | 0 | 0.2 | 0 | 1 | 5 |
| b10 | auc5 | precrec | 7 | 8 | 9 | 42.4 | 11 | 177 | 5 |
| b10 | auc5 | PRROC | 0 | 1 | 1 | 2.6 | 1 | 10 | 5 |
| b10 | auc5 | ROCR | 3 | 3 | 4 | 16.6 | 24 | 49 | 5 |
| testset | toolset | toolname | min | lq | mean | median | uq | max | neval |
|:--------|:--------|:--------------|-----:|-----:|------:|-------:|------:|-------:|------:|
| b10 | auc5 | AUCCalculator | 3.00 | 3.98 | 8.07 | 4.90 | 5.45 | 23.04 | 5 |
| b10 | auc5 | PerfMeas | 0.06 | 0.07 | 85.07 | 0.07 | 0.10 | 425.04 | 5 |
| b10 | auc5 | precrec | 5.56 | 6.13 | 42.58 | 7.43 | 12.04 | 181.75 | 5 |
| b10 | auc5 | PRROC | 0.18 | 0.19 | 1.82 | 0.19 | 0.23 | 8.32 | 5 |
| b10 | auc5 | ROCR | 1.89 | 1.99 | 12.33 | 2.13 | 17.08 | 38.56 | 5 |

### Evaluation of precision-recall curves

Expand All @@ -178,11 +173,6 @@ autoplot(scores1)
## Plot the results of PerfMeas and PRROC on c1, c2, and c3 test sets
toolset <- create_toolset(c("PerfMeas", "PRROC"))
scores2 <- run_evalcurve(testset, toolset)
```

## [1] "PerfMeas is not available."

``` r
autoplot(scores2, base_plot = FALSE)
```

Expand Down
Binary file modified README_files/figure-gfm/unnamed-chunk-5-1.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion appveyor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ install:
branches:
only:
- main
# - develop
- _develop

skip_tags: true

Expand Down
32 changes: 22 additions & 10 deletions cran-comments.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,33 @@
## Version 1.0.1
## Version 1.0.2
This is a submission for updating the already published package - prcbench.

In this version I have:

* Transferred the GitHub repository to an organization GitHub account,
* Improved several documents,

* Updated citation, and
* Skipped rJava tests when rJava is not installed,

* Updated the version.
* 0.9.1 -> 1.0.1
* Replaced Travis CI with GitHub actions,

* and Updated the version.
* 1.0.1 -> 1.0.2

## Test environments
* local Ubuntu 20.04 High Sierra, R 4.0.5
* local macOS High Sierra, R 4.0.4
* win-builder, R Under development (unstable) (2021-04-02 r80141)
* Ubuntu 16.04.6 LTS (on travis-ci), R 4.0.2 & R Under development (2021-04-02 r80141)
* Windows Server 2012 R2 x64 (on AppVeyor), R 4.0.5 Patched (2021-03-31 r80143)
- local Ubuntu 20.04, R 4.1.2

- local macOS Big Sur, R 4.1.2

- win-builder, R Under development (unstable) (2022-02-01 r81619 ucrt)

- Windows Server 2012 R2 x64 (on AppVeyor), R 4.1.0 Patched (2022-02-01 r81619 ucrt)

- GitHub Actions

- macOS-latest (release)
- windows-latest (release)
- ubuntu-latest (devel)
- ubuntu-latest (release)
- ubuntu-latest (oldrel-1)

## R CMD check results
* There were no ERRORs or WARNINGs.

0 comments on commit 40cf25a

Please sign in to comment.