-
Notifications
You must be signed in to change notification settings - Fork 1
/
benchmarks.Rmd
64 lines (51 loc) · 1.55 KB
/
benchmarks.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
---
title: "Benchmarks"
output:
bookdown::html_document2:
fig.width: 18
fig.height: 8
---
```{r benchmarks_setup, message = FALSE}
library("targets")
library("mlr3viz")
library("viridis")
library("ggplot2")
tar_load(bmrk_results)
tar_load(bmrk_aggr)
p.value <- 0.2
```
# Overview
```{r benchmarks_plot_c_harrell, fig.width = 18}
autoplot(bmrk_results) +
geom_boxplot(aes(fill = learner_id)) +
geom_jitter(position = position_jitter(0.2)) +
scale_fill_viridis(discrete = TRUE) +
theme(axis.text.x = element_text(angle = 90, hjust = 1, vjust = 0.5))
```
# Aggregated performance
Aggregated performance across all `nrow(bmrk_results$tasks)` datasets.
```{r benchmarks_aggr_plot_c_harrell}
autoplot(bmrk_aggr, type = "box", meas = "harrell_c") +
geom_boxplot(aes(fill = learner_id)) +
geom_jitter(position = position_jitter(0.2)) +
scale_fill_viridis(discrete = TRUE) +
theme(axis.text.x = element_text(angle = 90, hjust = 1, vjust = 0.5))
```
# Pairwise comparison
*p*-value is set to `r p.value`
(otherwise an error is thrown because of insignificant results).
```{r benchmarks_aggr_pairwise_harrell_c}
autoplot(
bmrk_aggr, type = "fn", meas = "harrell_c", p.value = p.value,
col = palette.colors(2L)[2L]
)
```
# Critical difference
*p*-value is set to `r p.value`
(otherwise an error is thrown because of insignificant results).
```{r benchmarks_aggr_critdiff_c_harrell}
autoplot(
bmrk_aggr, type = "cd", meas = "harrell_c", style = 2, p.value = p.value,
minimize = FALSE
) + scale_color_viridis(discrete = TRUE)
```