-
Notifications
You must be signed in to change notification settings - Fork 2
/
README.Rmd
151 lines (108 loc) · 7.06 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r, echo = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "README-"
)
```
# facerec
[![Travis-CI Build Status](https://travis-ci.org/cschwem2er/facerec.svg?branch=master)](https://travis-ci.org/cschwem2er/facerec)
[![AppVeyor Build Status](https://ci.appveyor.com/api/projects/status/github/cschwem2er/facerec?branch=master&svg=true)](https://ci.appveyor.com/project/cschwem2er/facerec)
[![CRAN status](https://www.r-pkg.org/badges/version/facerec)](https://cran.r-project.org/package=facerec)
[![CRAN downloads](https://cranlogs.r-pkg.org/badges/grand-total/facerec)](https://cran.rstudio.com/web/packages/facerec/index.html)
```{r, include = FALSE}
#[![CRAN status](https://www.r-pkg.org/badges/version/facerec)](https://cran.r-project.org/package=facerec)
#[![CRAN downloads](https://cranlogs.r-pkg.org/badges/grand-total/facerec)](https://cran.rstudio.com/web/packages/facerec/index.html)
```
## An Interface for Face Recognition in R
This package provides an interface to the [Kairos Face Recognition API](https://www.kairos.com/docs/api/). The API detects faces in images and returns estimates for demographics. It is also capable of recognizing and verifying humans across several images.
Important note: Kairos decided to shut down its online demo for diversity recognition features (see [here](http://www.kairos.com/blog/we-ve-retired-our-diversity-recognition-app-here-s-why)). As of July 2018, the API still returns ethnicity annotations, but it also seems as if they will not provide free access anymore. Until the release of an official statement from Kairos, I will stop the development of this package.
## How to Install
To install the CRAN version of facerec use ``install.packages('facerec')``.
You can also download and install the latest development version of the app by running ``devtools::install_github('cschwem2er/facerec')``. For Windows users installing from github requires proper setup of [Rtools](https://cran.r-project.org/bin/windows/Rtools/), for which a tutorial is available [here](https://github.com/stan-dev/rstan/wiki/Install-Rtools-for-Windows).
## How to Use
### Authentification
After loading facerec you first need to initiate your authentification credentials. Kairos offers a [free plan](https://www.kairos.com/pricing) for API access. After signing up, you will receive an application id and an application key. Both credentials need to be set as environment variables before using the initialization function `facerec_init()`:
```{r, eval = FALSE}
Sys.setenv(kairos_id = "Your Kairos API id")
Sys.setenv(kairos_key = "Your Kairos API key")
```
```{r, eval = FALSE}
library(facerec)
facerec_init()
```
You only need to call `facerec_init()` once after loading the package. In order to avoid entering your credentials for each session, you can permanently store them in your `.Renviron`. I recommend `usethis::edit_r_environ()` to find and edit your environment file.
### Face Recognition
Kairos accepts image of file type JPG, PNG, or BMP. Images can be passed to several facerec functions, either as an url string or a local image prepared with `prep_image()`. In the following example, `detect()` is used to recognize the face of the Star Wars character [Finn](https://en.wikipedia.org/wiki/Finn_(Star_Wars)):
<img src="https://upload.wikimedia.org/wikipedia/en/2/2a/Finn-Force_Awakens_%282015%29.png" width="250">
```{r, eval = FALSE}
finn_image <- 'https://upload.wikimedia.org/wikipedia/en/2/2a/Finn-Force_Awakens_%282015%29.png'
finn_face <- detect(image = finn_image)
```
The function returns a dataframe with annotations for the recognized face in the input image. Variables include positional features of recognized faces, such as x and y coordinates for eyes. Moreover, demographic attributes like gender, ethnicity and age are available.
Features can be visualized with the packages [magick](https://cran.r-project.org/web/packages/magick/index.html) and [ggplot2](https://cran.r-project.org/web/packages/ggplot2/index.html):
```{r fig.height=4, fig.width=3, eval = FALSE}
library(magick)
library(ggplot2)
finn_image %>% image_read() %>% image_ggplot() +
geom_rect(data = finn_face,
aes(xmin = top_left_x, xmax = top_left_x + width,
ymin = top_left_y, ymax = top_left_y + height),
fill = NA, linetype = 'dashed', size = 2, color = '#377eb8') +
geom_label(data = finn_face,
aes(x = chin_tip_x, y = chin_tip_y + 20,
label = paste('Gender:',
scales::percent(face_gender_male_confidence),
'Male')), size = 6, color = '#377eb8') +
geom_label(data = finn_face,
aes(x = chin_tip_x, y = chin_tip_y + 60,
label = paste('Ethnicity:', scales::percent(face_black),
'Black')), size = 6, color = '#377eb8') +
theme(legend.position="none")
```
<img src="man/figures/finn_facerec.png" width="250">
Kairos has [some recommendations](https://www.kairos.com/docs/api/best-practices) to improve the quality of its recognition service, but in general, the API also works with multiple faces inside an image:
```{r fig.height=3, fig.width=4.8, eval = FALSE}
sw_img <- "https://upload.wikimedia.org/wikipedia/en/8/82/Leiadeathstar.jpg"
sw_faces <- detect(sw_img)
sw_img %>% image_read() %>% image_ggplot() +
geom_rect(data = sw_faces,
aes(xmin = top_left_x , xmax = top_left_x + width,
ymin = top_left_y, ymax = top_left_y + height,
color = factor(face_id)),
fill = NA, linetype = 'dashed', size = 2) +
geom_label(data = sw_faces,
aes(x = chin_tip_x, y = chin_tip_y + 15,
label = face_gender_type,
color = factor(face_id)), size = 8) +
theme(legend.position="none")
```
<img src="man/figures/sw_facerec.png" width="400">
Besides annotating faces in single images, face recognition data can be stored permantly with the Kairos. This allows to assign multiple images to subject ids and to provide estimates about whether faces from different images belong to the same subjects.
```{r, eval = FALSE}
finn_face <- enroll(image = finn_image, subject_id = 'finn', gallery = 'starwars')
finn_new <- 'https://upload.wikimedia.org/wikipedia/commons/b/b6/John_Boyega_by_Gage_Skidmore.jpg'
finn_rec <- recognize(image = finn_new, gallery = 'starwars',
show_candidate_images = FALSE)
```
The function `recognize()` returns a dataframe including the probability of a match in the column `confidence`.
## Citation
If you use facerec for your publications please consider citing it:
```
Carsten Schwemmer (2018). facerec: An interface for face recognition in R. R package version 0.1.0.
https://github.com/cschwem2er/facerec
```
A BibTeX entry for LaTeX users is:
```
@Manual{,
title = {facerec: An interface for face recognition in R},
author = {Carsten Schwemmer},
year = {2018},
note = {R package version 0.1.0},
url = {https://github.com/cschwem2er/facerec},
}
```