/
irmi.Rmd
147 lines (110 loc) · 5.42 KB
/
irmi.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
---
title: "Imputation Method IRMI"
author: Wolfgang Rannetbauer
output: rmarkdown::html_vignette
vignette: >
%\VignetteIndexEntry{Imputation Method IRMI}
%\VignetteEngine{knitr::rmarkdown}
%\VignetteEncoding{UTF-8}
---
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.width = 6,
fig.align = "center"
)
```
## Overview
In addition to Model based Imputation Methods (see `vignette("modelImp")`) the `VIM` package also presents an iterative imputation method.
This vignette showcases the function `irmi()`. **IRMI** is short for **I**terative **R**obust **M**odel-based **I**mputation. This method can be used to generate imputations for several variables in a dataset.
Basically `irmi()` mimics the functionality of IVEWARE [(Raghunathan et al., 2001)](https://www150.statcan.gc.ca/n1/pub/12-001-x/2001001/article/5857-eng.pdf), but there are several improvements with respect to the stability of the initialized values, or the robustness of the imputed values. In contrast to other imputation methods, the IRMI algorithm does not require at least one fully observed variable. In each step of the iteration, one variable is used as a response variable and the remaining variables serve as the regressors. Thus the "whole" multivariate information will be used for imputation in the response variable. For more details, please see [IRMI Imputation](http://file.statistik.tuwien.ac.at/filz/papers/CSDA11TKF.pdf).
## Data
The following example demonstrates the functionality of `irmi()` using a subset of `sleep`. The columns have been selected deliberately to include some interactions between the missing values.
```{r setup, message=F}
library(VIM)
dataset <- sleep[, c("Dream", "NonD", "BodyWgt", "Span")]
dataset$BodyWgt <- log(dataset$BodyWgt)
dataset$Span <- log(dataset$Span)
aggr(dataset)
```
The plot indicates several missing values in `Dream`, `NonD`, and `Span. `
```{r}
sapply(dataset, function(x) sum(is.na(x)))
```
## Imputing multiple variables
The call of the function is straightforward and the algorithm usually converges in a few iterations.
```{r}
imp_irmi <- irmi(dataset)
aggr(imp_irmi, delimiter = "_imp")
```
We can see that `irmi()` imputed all missing values for all variables in our dataset.
### Diagnosing the results
As we can see in the next plot, for imputing missing values in `NonD` `Bodygt` plays an important role. The original data structure of `NonD` and `BodyWgt` is preserved by the `irmi()` imputation method.
```{r, fig.height=5}
imp_irmi[, c("NonD", "BodyWgt", "NonD_imp")] |>
marginplot(delimiter = "_imp")
```
The same is true for the data structure of `Span` and `BodyWgt`.
```{r, fig.height=5}
imp_irmi[, c("Span", "BodyWgt", "Span_imp")] |>
marginplot(delimiter = "_imp")
```
## Performance of method
In order to validate the performance of `irmi()` and to highlight the ability to impute different datatypes the `iris` dataset is used. Firstly, some values are randomly set to `NA`.
```{r}
data(iris)
df <- iris
colnames(df) <- c("S.Length", "S.Width", "P.Length", "P.Width", "Species")
# randomly produce some missing values in the data
set.seed(1)
nbr_missing <- 50
y <- data.frame(row = sample(nrow(iris), size = nbr_missing, replace = TRUE),
col = sample(ncol(iris), size = nbr_missing, replace = TRUE))
y <- y[!duplicated(y), ]
df[as.matrix(y)] <- NA
aggr(df)
sapply(df, function(x) sum(is.na(x)))
```
We can see that there are missings in all variables and some observations reveal missing values on several points.
```{r}
imp_irmi <- irmi(df)
aggr(imp_irmi, delimiter = "imp")
```
The plot indicates that all missing values have been imputed by the IRMI algorithm. The following table displays the rounded first five results of the imputation for all variables.
```{r echo=F,warning=F}
library(reactable)
results <- cbind("TRUE1" = as.numeric(iris[as.matrix(y[which(y$col==1),])]),
"IMPUTED1" = round(as.numeric(imp_irmi[as.matrix(y[which(y$col==1),])]),2),
"TRUE2" = as.numeric(iris[as.matrix(y[which(y$col==2),])]),
"IMPUTED2" = round(as.numeric(imp_irmi[as.matrix(y[which(y$col==2),])]),2),
"TRUE3" = as.numeric(iris[as.matrix(y[which(y$col==3),])]),
"IMPUTED3" = round(as.numeric(imp_irmi[as.matrix(y[which(y$col==3),])]),2),
"TRUE4" = as.numeric(iris[as.matrix(y[which(y$col==4),])]),
"IMPUTED4" = round(as.numeric(imp_irmi[as.matrix(y[which(y$col==4),])]),2),
"TRUE5" = (iris[as.matrix(y[which(y$col==5),])]),
"IMPUTED5" = (imp_irmi[as.matrix(y[which(y$col==5),])]))[1:5,]
reactable(results, columns = list(
TRUE1 = colDef(name = "True"),
IMPUTED1 = colDef(name = "Imputed"),
TRUE2 = colDef(name = "True"),
IMPUTED2 = colDef(name = "Imputed"),
TRUE3 = colDef(name = "True"),
IMPUTED3 = colDef(name = "Imputed"),
TRUE4 = colDef(name = "True"),
IMPUTED4 = colDef(name = "Imputed"),
TRUE5 = colDef(name = "True"),
IMPUTED5 = colDef(name = "Imputed")
),
columnGroups = list(
colGroup(name = "S.Length", columns = c("TRUE1", "IMPUTED1")),
colGroup(name = "S.Width", columns = c("TRUE2", "IMPUTED2")),
colGroup(name = "P.Length", columns = c("TRUE3", "IMPUTED3")),
colGroup(name = "P.Width", columns = c("TRUE4", "IMPUTED4")),
colGroup(name = "Species", columns = c("TRUE5", "IMPUTED5"))
),
striped = TRUE,
highlight = TRUE,
bordered = TRUE
)
```