/
Morph_UNIX.txt
executable file
·246 lines (166 loc) · 8 KB
/
Morph_UNIX.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
This is a program of moprhological analysis (Russian, German, and English languages).
This program is distributed under the Library GNU Public Licence, which is in the file
COPYING.
This program was written by Andrey Putrin, Alexey Sokirko.
The project started in Moscow in Dialing
Company (Russian and English language). The German part was created
at Berlin-Brandenburg Academy of Sciences and Humanities in Berlin (the project DWDS).
The Russian lexicon is based upon Zaliznyak's Dictionary .
The German lexicon is based upon Morphy system (http://www-psycho.uni-paderborn.de/lezius/).
The English lexicon is based upon Wordnet.
The system has been developed under Windows 2000 (MS VS 6.0), but
has also been compiled and run under Linux(GCC). It should work with
minor changes on other systems.
Website of DDC: www.aot.ru, https://sf.net/projects/morph-lexicon/
I compiled all sources with gcc 3.2. Lower versions are not supported.
Contents of the this source archive
1. The main morphological library (Source/LemmatizerLib).
2. Library for grammatical codes (Source/AgrgamtabLib).
3. Test morphological program (Source/TestLem)..
4. Library for working with text version of the dictionaries (Source/MorphWizardLib).
5. Generator of morphological prediction base (Source/GenPredIdx).
6. Generator of binary format of the dictionaries (Source/MorphGen).
=================================================
====== Installation =====
=================================================
Unpacking
* Create a catalog and register a system variable RML, which points
to this catalog:
mkdir /home/sokirko/RML
export RML=/home/sokirko/RML
* Put "lemmatizer.tar.gz", "???-src-morph.tar.gz"
to this catalog, "???" can be "rus", "ger" or "eng"
according to what you have downloaded. Unpack it
tar xfz lemmatizer.tar.gz
tar xfz ???-src-morph.tar.gz
Compiling morphology
0. Do not forget to set RML_PCRE (see above)
1. cd $RML
2. ./compile_morph.sh
This step should create all libraries and a test program $RML\Bin\TestLem.
Building Morphological Dictionary
1. cd $RML
2. ./generate_morph_bin.sh <lang>
where <lang> can be Russian, German according to the dictionary
yo have downloaded.
The script should terminate with message "Everything is OK".
You can test the morphology
$RML\Bin\TestLem <lang>
If something goes wrong, write me to sokirko@yandex.ru.
======================================================
========== MRD-file ============
======================================================
This section describes the format of a mrd-file. Mrd-file is a text
file which contains one morphological dictionary for one natural language.
MRD is an abbreviation of "morphological dictionary".
The usual place for this file is
$RML/Dicts/SrcMorph/xxxSrc/morphs.mrd,
where xxx can be "Eng", "Rus" or "Ger" depending on the language.
The encoding of the file depends also upon the language:
* Russian - Windows 1251
* German - Windows 1252
* English - ASCII
Gramtab-files
A mrd-file refers to a gramtab-file, which is
language-dependent and which contains all possible full morphological
patterns for the words. One line in a gramtab-file looks like as follows:
<ancode> <unused_number> <part_of_speech> <grammems>
An ancode is an ID, which consists of two letters and which uniquely
identifies a morphological pattern. A morphological pattern consists of
<part_of_speech> and <grammems>. For example, here is a line from the English
gramtab:
te 1 VBE prsa,pl
Here "te" is an ancode, "VBE" is a part of speech, "prsa,pl" are grammems,
"1" is the obsolete unused number.
In mrd-files we use ancodes to refer to a morphological pattern.
Here is the list of all gramtab-files:
* Russian - $Rml/Dicts/Morph/gramtab.json
* German - $Rml/Dicts/Morph/gramtab.json
* English - $Rml/Dicts/Morph/gramtab.json
Common information
All words in a mrd-file are written in uppercase.
One mrd-file consists of the following sections:
1. Section of flexion and prefix models;
2. Section of accentual models;
3. Section of user sessions;
4. Section of prefix sets;
5. Section of lemmas.
Each section is a set of records, one per line. The number of all records
of the section is written in the very beginning of the section at
a separate line. For example, here is a possible variant
of the section of user sessions:
1
alex;17:10, 13 October 2003;17:12, 13 October 2003
"1" means that this section contains only one record, which is written
on the next line, thus this section contains only two lines.
Section of possible flexion and prefix models
Each record of this section is a list of items. Each item
describes how one word form in a paradigm should be built. The whole list
describes the whole paradigm (a set of word forms with morphological patterns).
The format of one item is the following:
%<flexion>*<ancode>
or %<flexion>*<ancode>*<prefix>
where
<flexion> is a flexion (a string, which should be added to right of the word base)
<prefix> is a prefix (a string, which should be added to left of the word base)
<ancode> is an ancode.
Let us consider an example of an English flexion and prefix model:
%F*na%VES*nb
Here we have two items:
1. <flexion> = F; <ancode> = na
2. <flexion> = VES; <ancode> = nb
In order to decipher ancodes we should go the English gramtab-file.
There we can find the following lines:
na NOUN narr,sg
nb NOUN narr,pl
If base "lea" would be ascribed to this model, then its paradigm
would be the following:
leaf NOUN narr,sg
leaves NOUN narr,pl
It is important, that each word of a morphological dictionary
should contain a reference to a line in this section.
Section of possible accentual models
Each record of this section is a comma-delimited list of numbers, where
each number is an index of a stressed vowel of a word form(counting
from the end). The whole list contains a position for each word
form in the paradigm.
If an item of an accentual model of word is equal to 255, then it
is undefined, and it means that this word form is unstressed.
Each word in the dictionary should have a reference to
an accentual model, even though this model can consist only of empty items.
For one word, the number and the order of items in the accentual model
should be equal to the number and the order of items in the flexion and
prefix model. For example we can ascribe to word "leaf" with the paradigm
leaf NOUN narr,sg
leaves NOUN narr,pl
the following accentual model:
2,3
It produces the following accented paradigm:
le'af NOUN narr,sg
le'aves NOUN narr,pl
Section of user section
This is a system section, which contains information about user edit
sessions.
Section of prefix sets
Each record of this section is a comma-delimited list of strings, where
each string is a prefix, which can be prefixed to the whole word. If a prefix
set is ascribed to a word, it means, that the words with these prefixes
can also exist in the language. For example, if "leaf" has
the prefix set "anti,contra", it follows the existence of words "antileaf",
"contraleaf".
A flexion and prefix model can contain
also a reference to a prefix, but this prefix is for
one separate word form, while a prefix set is ascribed to the whole word
paradigm.
Section of lemmas
A record of this section is a space-separated tuple of the following format:
<base> <flex_model_no> <accent_model_no> <session_no> <type_ancode> <prefix_set_no>
where
<base> is a base (a constant part of a word in its paradigm)
<flex_model_no> is an index of a flexion and prefix model
<accent_model_no> is an index of an accentual model
<session_no> is an index of the session, by which the last user edited this word
<type_ancode> is ancode, which is ascribed to the whole word
(intended: the common part of grammems in the paradigm)
"-" if it is undefined
<prefix_set_no> is an index of a prefix set, or "-" if it is undefined