The goal of {readelan} is to access the data (i.e., annotations) and
metadata (e.g., tier types/structure, linked media and controlled
vocabularies) in ELAN annotation
files (.eaf), template files (.etf) and external controlled
vocabulary files (.ecv) directly in R.
The {readelan} package is a fast yet lightweight package with only one
dependency outside of core packages, namely
{xml2}. The {xml2} package is central to
access the inner structure of ELAN files, which are all fundamentally
XML files.
You can install the development version of {readelan} like so:
pak::pak("borstell/readelan")You can also get the official release version from CRAN:
install.packages("readelan")The most obvious use of {readelan} is to be able to read annotations
directly from an .eaf file:
library(readelan)
eaf_file <- system.file("extdata",
"example.eaf",
package = "readelan")
read_eaf(eaf_file)
#> filename a time_1 time_2 annotation tier tier_type participant
#> 1 example.eaf a1 ts1 ts2 I words default-lt s001
#> 2 example.eaf a2 ts3 ts4 like words default-lt s001
#> 3 example.eaf a3 ts5 ts6 cats words default-lt s001
#> 4 example.eaf a4 ts1 ts2 pronoun pos pos s001
#> 5 example.eaf a7 ts1 ts2 subject syntax syntax s001
#> 6 example.eaf a5 ts3 ts4 verb pos pos s001
#> 7 example.eaf a8 ts3 ts4 predicate syntax syntax s001
#> 8 example.eaf a6 ts5 ts6 noun pos pos s001
#> 9 example.eaf a9 ts5 ts6 object syntax syntax s001
#> annotator parent_ref a_ref language_ref cv_id
#> 1 ABC <NA> <NA> <NA> <NA>
#> 2 ABC <NA> <NA> <NA> <NA>
#> 3 ABC <NA> <NA> <NA> <NA>
#> 4 DEF words a1 eng pos
#> 5 GHI words a1 eng syntax
#> 6 DEF words a2 eng pos
#> 7 GHI words a2 eng syntax
#> 8 DEF words a3 eng pos
#> 9 GHI words a3 eng syntax
#> cve_ref start end duration time_unit
#> 1 <NA> 490 1590 1100 milliseconds
#> 2 <NA> 2010 3670 1660 milliseconds
#> 3 <NA> 4370 5980 1610 milliseconds
#> 4 cveid_89b2ac38-7313-4737-aa5f-19e1231ccb18 490 1590 1100 milliseconds
#> 5 cveid_4990ed36-c1d1-40c6-800e-2bd264d9a89b 490 1590 1100 milliseconds
#> 6 cveid_d5558ab7-11c3-47d5-9a0f-403724b0e0b7 2010 3670 1660 milliseconds
#> 7 cveid_de8c8f23-6b49-42da-9bd4-5b8b59d8b1da 2010 3670 1660 milliseconds
#> 8 cveid_f2eec815-c427-4b61-84eb-cd1e1601c9b4 4370 5980 1610 milliseconds
#> 9 cveid_f37323ed-9b7e-48d9-bbd6-b9105186ed02 4370 5980 1610 millisecondsThe tiers and xpath arguments in read_eaf() allow the user to
specify tiers of interest already on reading the files, which simplifies
and speeds up the process in cases where there are many tiers but only
some of which are necessary to import:
library(readelan)
eaf_file <- system.file("extdata",
"example.eaf",
package = "readelan")
read_eaf(eaf_file, tiers = list(tier = "words"))
#> filename a time_1 time_2 annotation tier tier_type participant
#> 1 example.eaf a1 ts1 ts2 I words default-lt s001
#> 2 example.eaf a2 ts3 ts4 like words default-lt s001
#> 3 example.eaf a3 ts5 ts6 cats words default-lt s001
#> annotator parent_ref a_ref language_ref cv_id cve_ref start end duration
#> 1 ABC <NA> <NA> <NA> <NA> <NA> 490 1590 1100
#> 2 ABC <NA> <NA> <NA> <NA> <NA> 2010 3670 1660
#> 3 ABC <NA> <NA> <NA> <NA> <NA> 4370 5980 1610
#> time_unit
#> 1 milliseconds
#> 2 milliseconds
#> 3 millisecondsSome ELAN files may contain controlled
vocabularies,
which are pre-defined sets of labels that can be used as annotation
values. The function read_cv() can extract all CVs from a file –
whether a regular annotation file (.eaf), a template file (.etf) or
an external CV file (.ecv):
library(readelan)
eaf_file <- system.file("extdata",
"example.eaf",
package = "readelan")
dplyr::glimpse(read_cv(eaf_file))
#> Rows: 12
#> Columns: 7
#> $ filename <chr> "example.eaf", "example.eaf", "example.eaf", "example.eaf", "…
#> $ url <chr> NA, NA, NA, NA, NA, NA, "./inst/extdata/syntax.ecv", "./inst/…
#> $ cv_id <chr> "pos", "pos", "pos", "pos", "pos", "pos", "syntax", "syntax",…
#> $ cve_id <chr> "cveid_89b2ac38-7313-4737-aa5f-19e1231ccb18", "cveid_89b2ac38…
#> $ lang_ref <chr> "eng", "fin", "eng", "fin", "eng", "fin", "eng", "fin", "eng"…
#> $ language <chr> "English (eng)", "Finnish (fin)", "English (eng)", "Finnish (…
#> $ value <chr> "pronoun", "pronomini", "verb", "verbi", "noun", "substantiiv…
etf_file <- system.file("extdata",
"CLP_annotation_v2-2.etf",
package = "readelan")
dplyr::glimpse(read_cv(etf_file))
#> Rows: 92
#> Columns: 7
#> $ filename <chr> "CLP_annotation_v2-2.etf", "CLP_annotation_v2-2.etf", "CLP_an…
#> $ url <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, N…
#> $ cv_id <chr> "CLP scene", "CLP scene", "CLP scene", "CLP scene", "CLP scen…
#> $ cve_id <chr> "cveid_be1d709d-9158-4278-a220-620d2df5ceef", "cveid_321f1558…
#> $ lang_ref <chr> "und", "und", "und", "und", "und", "und", "und", "und", "und"…
#> $ language <chr> "undetermined (und)", "undetermined (und)", "undetermined (un…
#> $ value <chr> "1a", "1b", "1c", "2a", "2b", "2c", "2d", "3a", "3b", "3c", "…
ecv_file <- system.file("extdata",
"syntax.ecv",
package = "readelan")
dplyr::glimpse(read_cv(ecv_file))
#> Rows: 6
#> Columns: 7
#> $ filename <chr> "syntax.ecv", "syntax.ecv", "syntax.ecv", "syntax.ecv", "synt…
#> $ url <chr> NA, NA, NA, NA, NA, NA
#> $ cv_id <chr> "syntax", "syntax", "syntax", "syntax", "syntax", "syntax"
#> $ cve_id <chr> "cveid_4990ed36-c1d1-40c6-800e-2bd264d9a89b", "cveid_4990ed36…
#> $ lang_ref <chr> "eng", "fin", "eng", "fin", "eng", "fin"
#> $ language <chr> "English (eng)", "Finnish (fin)", "English (eng)", "Finnish (…
#> $ value <chr> "subject", "subjekti", "predicate", "predikaatti", "object", …To access information about tiers in an annotation file (.eaf) or
template file (.etf), the read_tiers() function can be used:
library(readelan)
eaf_file <- system.file("extdata",
"example.eaf",
package = "readelan")
dplyr::glimpse(read_tiers(eaf_file))
#> Rows: 3
#> Columns: 9
#> $ filename <chr> "example.eaf", "example.eaf", "example.eaf"
#> $ tier_type <chr> "default-lt", "pos", "syntax"
#> $ tier <chr> "words", "pos", "syntax"
#> $ participant <chr> "s001", "s001", "s001"
#> $ annotator <chr> "ABC", "DEF", "GHI"
#> $ parent_ref <chr> NA, "words", "words"
#> $ language_ref <chr> NA, "eng", "eng"
#> $ constraint <chr> NA, "Symbolic_Association", "Symbolic_Association"
#> $ cv_ref <chr> NA, "pos", "syntax"
etf_file <- system.file("extdata",
"CLP_annotation_v2-2.etf",
package = "readelan")
dplyr::glimpse(read_tiers(etf_file))
#> Rows: 14
#> Columns: 9
#> $ filename <chr> "CLP_annotation_v2-2.etf", "CLP_annotation_v2-2.etf", "CL…
#> $ tier_type <chr> "CLP main", "CLP congruence", "CLP event", "CLP interacti…
#> $ tier <chr> "CLP handshape", "CLP congruence", "CLP event", "CLP inte…
#> $ participant <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA
#> $ annotator <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA
#> $ parent_ref <chr> NA, "CLP handshape", "CLP handshape", "CLP handshape", "C…
#> $ language_ref <chr> NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA
#> $ constraint <chr> NA, "Symbolic_Association", "Symbolic_Association", "Symb…
#> $ cv_ref <chr> "CLP handshape", "CLP congruence", "CLP event", "CLP inte…To access information about linked media file in an annotation file
(.eaf), the read_media() function can be used:
library(readelan)
eaf_file <- system.file("extdata",
"example.eaf",
package = "readelan")
read_media(eaf_file)
#> filename item attribute value
#> 1 example.eaf 1 MEDIA_URL file:///Users/user01/video_01.mp4
#> 2 example.eaf 1 MIME_TYPE video/mp4
#> 3 example.eaf 1 RELATIVE_MEDIA_URL ./video_01.mp4
#> 4 example.eaf 2 MEDIA_URL file:///Users/user01/video_02.mp4
#> 5 example.eaf 2 MIME_TYPE video/mp4
#> 6 example.eaf 2 RELATIVE_MEDIA_URL ./video_02.mp4
#> 7 example.eaf 3 MEDIA_URL file:///Users/user01/audio_02.mp3
#> 8 example.eaf 3 MIME_TYPE audio/*
#> 9 example.eaf 3 RELATIVE_MEDIA_URL ./audio_02.mp3The read_ functions all allow for either a single file or a vector of
files to be used as file input. With multiple files, the function will
iterate over them, and return a single data frame:
library(readelan)
eaf_file2 <- system.file("extdata",
"example_no_annotations.eaf",
package = "readelan")
dplyr::glimpse(read_eaf(c(eaf_file, eaf_file2)))
#> Rows: 9
#> Columns: 18
#> $ filename <chr> "example.eaf", "example.eaf", "example.eaf", "example.eaf…
#> $ a <chr> "a1", "a2", "a3", "a4", "a7", "a5", "a8", "a6", "a9"
#> $ time_1 <chr> "ts1", "ts3", "ts5", "ts1", "ts1", "ts3", "ts3", "ts5", "…
#> $ time_2 <chr> "ts2", "ts4", "ts6", "ts2", "ts2", "ts4", "ts4", "ts6", "…
#> $ annotation <chr> "I", "like", "cats", "pronoun", "subject", "verb", "predi…
#> $ tier <chr> "words", "words", "words", "pos", "syntax", "pos", "synta…
#> $ tier_type <chr> "default-lt", "default-lt", "default-lt", "pos", "syntax"…
#> $ participant <chr> "s001", "s001", "s001", "s001", "s001", "s001", "s001", "…
#> $ annotator <chr> "ABC", "ABC", "ABC", "DEF", "GHI", "DEF", "GHI", "DEF", "…
#> $ parent_ref <chr> NA, NA, NA, "words", "words", "words", "words", "words", …
#> $ a_ref <chr> NA, NA, NA, "a1", "a1", "a2", "a2", "a3", "a3"
#> $ language_ref <chr> NA, NA, NA, "eng", "eng", "eng", "eng", "eng", "eng"
#> $ cv_id <chr> NA, NA, NA, "pos", "syntax", "pos", "syntax", "pos", "syn…
#> $ cve_ref <chr> NA, NA, NA, "cveid_89b2ac38-7313-4737-aa5f-19e1231ccb18",…
#> $ start <dbl> 490, 2010, 4370, 490, 490, 2010, 2010, 4370, 4370
#> $ end <dbl> 1590, 3670, 5980, 1590, 1590, 3670, 3670, 5980, 5980
#> $ duration <dbl> 1100, 1660, 1610, 1100, 1100, 1660, 1660, 1610, 1610
#> $ time_unit <chr> "milliseconds", "milliseconds", "milliseconds", "millisec…Note: In this case, the resulting data frame is the same as in the
first example, since the second .eaf file does not contain any
annotations.
When iterating over a large number of files, setting the progress
argument to TRUE can be advisable, in order to get visual information
of the progress.
There are other packages that also contain functions to read ELAN
annotation files, e.g.,
{act} and
{phonfieldwork}.
However, the goal of {readelan} is to be a lightweight package
dedicated to ELAN files (including .etf and .ecv files), and which
is also fast. Benchmarking against these other packages indicate that
{readelan} is fast option.
eaf_file <- system.file("extdata",
"example.eaf",
package = "readelan")
microbenchmark::microbenchmark(
readelan::read_eaf(eaf_file),
phonfieldwork::eaf_to_df(eaf_file),
act::import_eaf(eaf_file),
times = 100
)
#> Unit: milliseconds
#> expr min lq mean median
#> readelan::read_eaf(eaf_file) 3.54554 3.752337 4.467007 3.916893
#> phonfieldwork::eaf_to_df(eaf_file) 12.21567 13.002686 17.925177 13.628694
#> act::import_eaf(eaf_file) 59.55592 63.103674 66.648167 63.936351
#> uq max neval cld
#> 4.522892 11.86703 100 a
#> 15.973282 376.29846 100 b
#> 65.180649 142.63887 100 cAdditionally, since {readelan}’s read_eaf() function can inherently
iterate over many files, and specify target tiers or tier types using
the tiers or xpath arguments, it becomes much faster to read larger
sets of annotation files for which only specific tiers are of interest
to import.