forked from nlplab/brat
-
Notifications
You must be signed in to change notification settings - Fork 1
/
case-studies.html
420 lines (382 loc) · 17.5 KB
/
case-studies.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
<html lang="en-US" xml:lang="en-US" xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>brat rapid annotation tool</title>
<link rel="stylesheet" type="text/css" href="style.css"/>
<link rel="stylesheet" type="text/css" href="jquery-theme/jquery-ui-redmond.css"/>
<link rel="shortcut icon" href="../favicon.ico"/>
</head>
<style type="text/css">
<!--
-->
</style>
<body>
<div id="manual-main" class="center">
<div id="header" class="ui-widget-header rounded">
<span><a href="index.html">home</a></span>
<span style="color:lightblue" class="unselectable">|</span>
<span><a href="introduction.html">introduction</a></span>
<span style="color:lightblue" class="unselectable">|</span>
<span><a href="case-studies.html">examples</a></span>
<span style="color:lightblue" class="unselectable">|</span>
<span><a href="features.html">features</a></span>
<span style="color:lightblue" class="unselectable">|</span>
<span><a href="installation.html">installation</a></span>
<span style="color:lightblue" class="unselectable">|</span>
<span><a href="manual.html">manual</a></span>
<div id="menulogo" class="logo unselectable">brat</div>
</div>
<h1>case studies and compatible annotation tasks</h1>
<p>brat has been applied in a number real-world annotation
projects and is applicable to a great variety of annotation
tasks. This page showcases a number of tasks of both of
these categories.
</p>
<div>
<div id="notebox">Identifying information on this page
regarding projects associated with brat development has been
redacted for anonymous review. These details will be
restored prior to public release.
</div>
</div>
<br style="clear:both"/>
<!-- ############################################################ -->
<h1>projects using brat</h1>
<p>brat has been used in annotation efforts throughout its
development, and has by version 1.0 already been used to mark
tens of thousands of individual annotations in thousands of
documents comprising hundreds of thousands of words.
</p>
<p>Some annotation projects that have used (or are currently
using) brat are introduced in the following.
</p>
<br style="clear:both"/>
<!-- ############################################################ -->
<a href="case-studies/11419822-full.png">
<img class="right" src="case-studies/11419822-small.png"/>
</a>
<h2 class="nomargin">Event annotation for cancer biology</h2>
<p><span class="redacted">[REDACTED]</span> are using brat in
<span class="redacted">[REDACTED]</span> for structured event
annotation for scientific manuscript abstracts in cancer
domains.
</p>
<p>The task involves a large number of types of both physical
entities (genes/proteins, chemicals, cell, tissues,
pathological formations etc.), molecular level events, and
biological processes (gene expression, binding, development,
regulation etc.). To assist in the annotation,
brat <!-- <a href="rapid-annotation.html"> -->
rapid annotation<!-- </a> --> mode
is used to suggest likely types to annotators.
</p>
<p>
<div class="inforow">
<span class="infolabel">Examples in brat:</span>
<span class="infovalue"><a target="brat-example" href="http://li305-194.members.linode.com/~eacl/brat/#/static-examples/Event-extraction/Anonymized-project-2/">see it live on brat</a> (read-only data)</span>
</div>
<div class="inforow">
<span class="infolabel">Project website:</span>
<span class="infovalue"><span class="redacted">[REDACTED]</span></span>
</div>
<div class="inforow">
<span class="infolabel">Publications:</span>
<span class="infovalue"><span class="redacted">[REDACTED]</span></span>
</div>
</p>
<br style="clear:both"/>
<!-- ############################################################ -->
<a href="case-studies/susumu-full.png">
<img class="right" src="case-studies/susumu-small.png"/>
</a>
<h2 class="nomargin">Japanese verb frame annotation</h2>
<p>brat is used by the <span class="redacted">[REDACTED]</span>
project for verb frame annotation in
<a href="http://framenet.icsi.berkeley.edu/">FrameNet</a>-like
representation.
</p>
<p>The annotation involves identifying the core and peripheral
arguments of specific verbs in sentence scope and marking up
their roles.
</p>
<p>To assist in the visualization and annotation of Japanese
text, brat integrates the <a href="http://mecab.sourceforge.net/">MeCab</a> Japanese word
segmentation tool to split the text into tokens and sentences in order to increase
the level of read-ability even for languages that lack explicit word segmentation markers.
</a>
<p>
<div class="inforow">
<span class="infolabel">Examples in brat:</span>
<span class="infovalue"><a target="brat-example" href="http://li305-194.members.linode.com/~eacl/brat/#/static-examples/Verb-frame/Japanese/">see it live on brat</a> (read-only data)</span>
</div>
<div class="inforow">
<span class="infolabel">Project website:</span>
<span class="infovalue"><span class="redacted">[REDACTED]</span></span>
</div>
<div class="inforow">
<span class="infolabel">Publications:</span>
<span class="infovalue"><span class="redacted">[REDACTED]</span></span>
</div>
</p>
<br style="clear:both"/>
<!-- ############################################################ -->
<a href="case-studies/1334229-04-Results-p02-full.png">
<img class="right" src="case-studies/1334229-04-Results-p02-small.png"/>
</a>
<h2 class="nomargin">Gene-mutation-phenotype relations</h2>
<p><span class="redacted">[REDACTED]</span> are using brat for
the annotation of entities and their relations in full-text
scientific publications to capture the associations of
specific genes, mutations, and phenotype characteristics.
</p>
<p>The annotation involves 12 entity mention types (gene,
disease, mutation, characteristic, etc.) and binary relations
between them.
<p>
<div class="inforow">
<span class="infolabel">Examples in brat:</span>
<span class="infovalue"><a target="brat-example" href="http://li305-194.members.linode.com/~eacl/brat/#/static-examples/Relation-extraction/Anonymized-project-3/">see it live on brat</a> (read-only data)</span>
</div>
<div class="inforow">
<span class="infolabel">Project website:</span>
<span class="infovalue"><span class="redacted">[REDACTED]</span></span>
</div>
<div class="inforow">
<span class="infolabel">Publications:</span>
<span class="infovalue"><span class="redacted">[REDACTED]</span></span>
</div>
</p>
<br style="clear:both"/>
<!-- ############################################################ -->
<img class="right tinymargin" src="case-studies/biocreative-2-GN.png"/>
<img class="right tinymargin" src="case-studies/AImed.png"/>
<img class="right tinymargin" src="case-studies/GREC.png"/>
<h2 class="nomargin">Consolidation of biomedical information extraction resources</h2>
<p><span class="redacted">[REDACTED]</span> are using brat for
the visualization and consolidation of publicly available
resources for a large variety of biomedical information
extraction tasks.
</p>
<p>The effort has so far converted over 20 corpora with
annotation for a wide range of entity recognition, relation
detection, and event extraction targets.
</p>
<p>The various corpora include annotation for entities such as
genes/proteins, chemicals, drugs, diseases, treatments, and
anatomical locations; relations such as treatment-disease,
protein-protein binding and drug-drug interaction; and events
such as gene expression, phosphorylation and localization.
</p>
<p>
<div class="inforow">
<span class="infolabel">Examples in brat:</span>
<span class="infovalue"><a target="brat-example" href="http://li305-194.members.linode.com/~eacl/brat/#/static-examples/Relation-extraction/">see it live on brat</a> (read-only data)</span>
</div>
<div class="inforow">
<span class="infolabel">Project website:</span>
<span class="infovalue"><span class="redacted">[REDACTED]</span></span>
</div>
<div class="inforow">
<span class="infolabel">Publications:</span>
<span class="infovalue"><span class="redacted">[REDACTED]</span></span>
</div>
</p>
<br style="clear:both"/>
<!-- ############################################################ -->
<h1>annotation tasks compatible with brat</h1>
<p>A variety of annotation tasks that could be performed in brat
are introduced below using examples from annotated corpora.
</p>
<p>The example discussed in this section have been originally
created in various tools other than brat and converted into
brat format. Converters for many of the original formats are
distributed with brat.
</p>
<p>In the selection of examples included here, priotity has been
given to tasks with freely available data. (For example, although
any CoNLL shared task dataset could be visualized in brat, we
have only selected freely available subsets of those datasets
that do not require a separate licence.)
</p>
<br style="clear:both"/>
<!-- ############################################################ -->
<h2 class="nomargin">Entity mention detection</h2>
<a href="case-studies/esp.train-doc-536-full.png">
<img class="right" src="case-studies/esp.train-doc-536-small.png"/>
</a>
<h3>Example: CoNLL 2002 Shared Task: Language-Independent Named Entity Recognition</h3>
<p>The Conference on Computational Natural Language Learning
(CoNLL) 2002 shared task on Language-Independent Named Entity
Recognition provided two annotated corpora (Spanish and
Dutch) annotated with entities of four types (person,
organization, location and miscellaneous).
</p>
<p>A conversion script from the CoNLL 2002 shared task format into
the brat standoff format and a sample of the corpus annotations
are distributed with brat.
</p>
<p>The full shared task data are freely available from the shared
task website.
</p>
<p>
<div class="inforow">
<span class="infolabel">Examples in brat:</span>
<span class="infovalue"><a target="brat-example" href="http://li305-194.members.linode.com/~eacl/brat/#/static-examples/Named-entity-recognition/CoNLL-02-Spanish/">see it live on brat</a> (read-only data)</span>
</div>
<div class="inforow">
<span class="infolabel">Project website:</span>
<span class="infovalue"><a href="http://www.cnts.ua.ac.be/conll2002/ner/">CoNLL 2002 shared task website</a></span>
</div>
<div class="inforow">
<span class="infolabel">Publications:</span>
<span class="infovalue"><a href="http://www.cnts.ua.ac.be/conll2002/pdf/15558tjo.pdf">Tjong Kim Sang, 2002</a> [PDF]</span>
</div>
</p>
<br style="clear:both"/>
<!-- ############################################################ -->
<h2 class="nomargin">Event extraction</h2>
<a href="case-studies/PMID-20300060-full.png">
<img class="right" src="case-studies/PMID-20300060-small.png"/>
</a>
<h3>Example: BioNLP shared task: biomedical event extraction</h3>
<p>BioNLP shared task events held in 2009 and 2011 have included
four different event extraction tasks.
</p>
<p>The brat standoff format is compatible with the data
distribution format of the BioNLP shared task, and samples of
the annotations of the various corpora provided for the task
are distributed with brat.
</p>
<p>The full shared task data are freely available from the shared
task website.
</p>
<p>
<div class="inforow">
<span class="infolabel">Examples in brat:</span>
<span class="infovalue"><a target="brat-example" href="http://li305-194.members.linode.com/~eacl/brat/#/static-examples/Event-extraction/">see it live on brat</a> (read-only data)</span>
</div>
<div class="inforow">
<span class="infolabel">Project website:</span>
<span class="infovalue"><a href="http://2011.bionlp-st.org">BioNLP shared task 2011 website</a></span>
</div>
<div class="inforow">
<span class="infolabel">Publications:</span>
<span class="infovalue"><a href="http://aclweb.org/anthology-new/W/W09/W09-1401.pdf">Kim et al. 2009</a>, <a href="http://aclweb.org/anthology-new/W/W11/W11-1801.pdf">2011</a> [PDFs]</span>
</div>
</p>
<br style="clear:both"/>
<!-- ############################################################ -->
<h2 class="nomargin">Coreference resolution</h2>
<a href="case-studies/PMID-1492121-full.png">
<img class="right" src="case-studies/PMID-1492121-small.png"/>
</a>
<h3>Example: CO supporting task: coreference in scientific publications</h3>
<p>The BioNLP shared task 2011 included a supporting task
on coreference in scientific publications.
</p>
<p>The brat standoff format is compatible with the
representation used in the coreference task, and samples of
the annotations of the corpus provided for the task are
distributed with brat.
</p>
<p>The full shared task data are freely available from the shared
task website.
</p>
<p>
<div class="inforow">
<span class="infolabel">Examples in brat:</span>
<span class="infovalue"><a target="brat-example" href="http://li305-194.members.linode.com/~eacl/brat/#/">see it live on brat</a> (read-only data)</span>
</div>
<div class="inforow">
<span class="infolabel">Project website:</span>
<span class="infovalue"><a href="http://2011.bionlp-st.org/home/protein-gene-coreference-task">BioNLP ST 2011 CO task home page</a></span>
</div>
<div class="inforow">
<span class="infolabel">Publications:</span>
<span class="infovalue"><a href="http://aclweb.org/anthology-new/W/W11/W11-1811.pdf">Nguyen et al. 2001</a> [PDF]</span>
</div>
</p>
<br style="clear:both"/>
<!-- ############################################################ -->
<h2 class="nomargin">Chunking</h2>
<p>Chunking is the task of dividing text into non-overlapping
segments that are typically further assigned labels such as
<tt>NP</tt> (Noun Phrase).
</p>
<a href="case-studies/train.txt-doc-109-full.png">
<img class="right" src="case-studies/train.txt-doc-109-small.png"/>
</a>
<h3>Example: CoNLL 2000 Shared Task: Chunking</h3>
<p>The Conference on Computational Natural Language
Learning 2000 (CoNLL 2000) shared task on chunking provides
freely available training and test data.
</p>
<p>A conversion script from the CoNLL 2000 shared task format
into the brat standoff format and a sample of the corpus
annotations are distributed with brat.
</p>
<p>The full shared task data are freely available from the shared
task website.
</p>
<p>
<div class="inforow">
<span class="infolabel">Examples in brat:</span>
<span class="infovalue"><a target="brat-example" href="http://li305-194.members.linode.com/~eacl/brat/#/static-examples/Chunking/CoNLL-00/">see it live on brat</a> (read-only data)</span>
</div>
<div class="inforow">
<span class="infolabel">Project website:</span>
<span class="infovalue"><a href="http://www.clips.ua.ac.be/conll2000/chunking/">CoNLL 2000 shared task website</a></span>
</div>
<div class="inforow">
<span class="infolabel">Publications:</span>
<span class="infovalue"><a href="http://www.clips.ua.ac.be/conll2000/pdf/12732tjo.pdf">Tjong Kim Sang and Buchholz (2000)</a> [PDF]</span>
</div>
</p>
<br style="clear:both"/>
<!-- ############################################################ -->
<h2 class="nomargin">Dependency syntax</h2>
<p>Dependency parsing (syntactic analysis) is the task of
assigning binary relations between words to mark their
head-dependent relations.
</p>
<a href="case-studies/swedish_talbanken05_train.conll-doc-880-full.png">
<img class="right" src="case-studies/swedish_talbanken05_train.conll-doc-880-small.png"/>
</a>
<h3>Example: CoNLL-X Shared Task: Multi-lingual Dependency Parsing</h3>
<p>The Tenth Conference on Computational Natural Language
Learning (CoNLL-X) shared task on Multi-lingual Dependency
Parsing provided annotated corpora for 13 languages, four of
which are freely availabe (for Danish, Dutch, Portuguese and
Swedish).
</p>
<p>A conversion script from the CoNLL-X shared task format into
the brat standoff format and a sample of the corpus
annotations for these four languages are distributed with
brat.
</p>
<p>The full shared task data are freely available from the shared
task website.
</p>
<p>
<div class="inforow">
<span class="infolabel">Examples in brat:</span>
<span class="infovalue"><a target="brat-example" href="http://li305-194.members.linode.com/~eacl/brat/#/static-examples/Dependency-parsing/">see it live on brat</a> (read-only data)</span>
</div>
<div class="inforow">
<span class="infolabel">Project website:</span>
<span class="infovalue"><a href="http://ilk.uvt.nl/conll/">CoNLL-X shared task website</a></span>
</div>
<div class="inforow">
<span class="infolabel">Publications:</span>
<span class="infovalue"><a href="http://aclweb.org/anthology-new/W/W06/W06-2920.pdf">Buchholz and Marsi (2006)</a> [PDF]</span>
</div>
</p>
<br style="clear:both"/>
<div>
<h2 style="text-align:right"><a href="features.html">Next: features in overview</a></h2>
</div>
<div id="footer">
© 2010-2012 brat contributors
</div>
</div>
</body>
</html>