forked from kaldi-asr/kaldi
-
Notifications
You must be signed in to change notification settings - Fork 0
/
matrix.dox
314 lines (272 loc) · 13.3 KB
/
matrix.dox
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
// doc/matrix.dox
// Copyright 2009-2011 Microsoft Corporation
// See ../../COPYING for clarification regarding multiple authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
// http://www.apache.org/licenses/LICENSE-2.0
// THIS CODE IS PROVIDED *AS IS* BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION ANY IMPLIED
// WARRANTIES OR CONDITIONS OF TITLE, FITNESS FOR A PARTICULAR PURPOSE,
// MERCHANTABLITY OR NON-INFRINGEMENT.
// See the Apache 2 License for the specific language governing permissions and
// limitations under the License.
namespace kaldi {
/**
\page matrix The Kaldi Matrix library
The Kaldi matrix library is mostly a C++ wrapper for standard BLAS and LAPACK
linear algebra routines. This documentation page provides an overview of
how to use the library. See \ref matrix_group for code-level documentation,
and see \ref matrixwrap for an explanation of how the matrix code makes use
of external libraries.
\section matrix_sec_types Matrix and vector types
The most important types defined by the matrix library are Matrix and Vector.
We can illustrate their use with an example:
\code
Vector<float> v(10), w(9);
for(int i=0; i < 9; i++) {
v(i) = i;
w(i) = i+1;
}
Matrix<float> M(10,9);
M.AddVecVec(1.0, v, w);
\endcode
The code above first sets up v and w as vectors of size 10 and
9 respectively (the last element of v will be zero as the constructor
sets the data to zero).
The parenthesis operator indexes the vectors; a matrix would be indexed
as M(i,j).
M is initialized to a 10 x 9 matrix. The last line of the code
adds the outer product \f$ v w^T \f$ to M. The naming scheme for the
function is based on writing the operation as an equation, in this case:
\f[ M \leftarrow M + \alpha v w^T \f]
The "+" becomes "Add" in the name, and the types of the non-scalar arguments
become parts of the name too, in this case "Vec" for v and "Vec" for w
(so AddVecVec). The type of M doesn't appear here as the function is its class member.
Also notice that we correctly sized M before adding the outer product (we could
have done this without using the constructor by calling \ref Matrix::Resize "Resize").
The matrix library never tries to "guess" what size you want something to be, it will
just crash if something is the wrong size.
The only things that will resize a matrix are the constructor,
\ref Matrix::Resize "Resize", and \ref Matrix::Read "Read" (and the assignment
operator, which is mainly included so that we can declare std::vector<Matrix<double> >
and the like).
To provide another example, if A, B and C are matrices and alpha
and beta are scalars, to execute
\f[ A = \alpha B C^T + \beta A \f]
we would use the line of code
\code
A.AddMatMat(alpha, B, kNoTrans, C, kTrans, beta);
\endcode
Here, kNoTrans and kTrans are enumeration values that indicate whether or
not the matrix argument is transposed. An examples that illustrates this
and some other issues is:
\code
Matrix<float> M(5, 10), N(5, 10), P(5, 5);
// initialize M and N somehow...
// next line: P := 1.0 * M * N^T + 0.0 * P.
P.AddMatMat(1.0, M, kNoTrans, N, kTrans, 0.0);
// tr(M N^T)
float f = TraceMatMat(M, N, kTrans),
g = P.Trace();
KALDI_ASSERT(f == g); // we use this macro for asserts in Kaldi
// code (prints stack trace and throws exception).
\endcode
Note that in the BLAS tradition, there is no operation that does just, say,
\f$ A = B C \f$; you have to use a special case of the example we just
gave (e.g. with alpha=1 and beta=0).
To get an idea of what mathematical member functions are available in the Matrix
and Vector classes, see the documentation for MatrixBase and VectorBase.
\section matrix_sec_sym Symmetric and triangular matrices
There are special types for symmetric matrices (\ref SpMatrix)
and triangular matrices (\ref TpMatrix). These are both represented
in memory as a "packed" lower-triangular matrix and both inherit
from \ref PackedMatrix. Of these, SpMatrix is the most useful
(TpMatrix is mostly just used to compute Cholesky factors).
A typical example of using these types is below. Notice the
"AddMat2" function. This illustrates a new feature of our naming
scheme: when a quantity appears twice in an expression we add "2"
to the corresponding part of the function name.
\code
Matrix<float> feats(1000, 39);
// ... initialize feats somehow ...
SpMatrix<float> scatter(39);
// next line does scatter = 0.001 * feats' * feats.
scatter.AddMat2(1.0/1000, feats, kTrans, 0.0);
TpMatrix<float> cholesky(39);
cholesky.Cholesky(scatter);
cholesky.Invert();
Matrix<float> whitened_feats(1000, 39);
// If scatter = C C^T, next line does:
// whitened_feats = feats * C^{-T}
whitened_feats.AddMatTp(1.0, feats, kNoTrans, cholesky, kTrans, 0.0);
\endcode
This code would crash (cholesky.Invert() would throw an exception) if we did not
initialize the features somehow where indicated.
\section matrix_sec_sub Sub-vectors and sub-matrices.
For matrix operations that involve only part of a vector or matrix,
the \ref SubVector and \ref SubMatrix classes are provided. Like
\ref Vector and \ref Matrix, these inherit from \ref VectorBase
and \ref MatrixBase respectively, so the implementation of operations
that don't involve resizing is shared (you can never resize
a SubVector or SubMatrix because it's treated as a pointer into
the underlying Vector or Matrix).
An example of using these classes is here:
\code
Vector<float> v(10), w(10);
Matrix<float> M(10, 10);
SubVector<float> vs(v, 1, 9), ws(w, 1, 9);
SubMatrix<float> Ms(M, 1, 9, 1, 9);
// next line would be v(2:10) += M(2:10,2:10)*w(2:10) in some
// math scripting languages.
vs.AddMatVec(1.0, Ms, kNoTrans, ws);
\endcode
There are other ways to obtain these types. If M is a matrix, M.Row(3) will
return row 3 of M, as a SubVector. There is a corresponding constructor,
for example:
\code
SubVector row_of_m(M, 0); // get zeroth row of M.
\endcode
The same doesn't work for columns. This is because our vector types are
contiguous in memory; we don't currently have a "stride" class-member (even though
the underlying BLAS supports this).
Another way to obtain these types is by using the Range functions of
Vector and Matrix. For instance:
\code
// get a sub-vector of length 5 starting from position 0; zero it.
v.Range(0, 5).SetZero();
// get a sub-matrix of size 2x2 starting from position (5,5); zero it.
M.Range(5, 2, 5, 2).SetZero();
\endcode
You have to be a little careful with SubVector and SubMatrix types. For
instance, if you create a SubVector and then destroy or resize the underlying
Vector or Matrix, the SubVector will be pointing to freed memory. Also,
SubVector and SubMatrix do not respect const-ness the way they ideally
should, so it is possible to modify a const Vector by creating a SubVector.
Fixing these issues would have made the library a lot more complex.
\section matrix_sec_calling Calling conventions for vectors and matrices
In general, when a function requires a vector or matrix type, for greater
flexibility we pass an argument of type e.g. VectorBase<BaseFloat> or MatrixBase<BaseFloat>
rather than Vector<BaseFloat> or Matrix<BaseFloat>. That way the code is still
callable if we are really passing a sub-matrix or sub-vector. The exception is
where the code needs to resize the vector or matrix in question, in which case
it would take a pointer of type e.g. Vector<BaseFloat>* or Matrix<BaseFloat>*;
it cannot take a reference because resizing is non-const, and as per the style
guide, functions cannot take non-const reference arguments.
\section matrix_sec_copy Copying matrix and vector types
There are different ways to copy vectors and matrices. The simplest ones are
the CopyFrom functions, for instance Matrix::CopyFromMat, Vector::CopyFromVec,
SpMatrix::CopyFromSp and so on. These will work even between float and double,
unlike most of the mathematical operations. However, they will not resize
the matrix for you and will crash if the sizes are wrong (you have to resize
yourself using Vector::Resize, Matrix::Resize and so on). The same types of functions are
available to copy between different matrix types, where this makes sense: for
example, Matrix::CopyFromTp, SpMatrix::CopyFromMat, TpMatrix::CopyFromMat and so
on. See the documentation of those functions for their exact behavior.
Wherever the above functions exist, corresponding constructors exist. These
will do the copying but also resize, for instance:
\code
Matrix<double> M(10, 10);
... initialize M ...
// next line copies M to Mf.
Matrix<float> Mf(M);
\endcode
There are also more special-purpose copying functions. You can copy the
concatenated rows of a matrix to a vector (Vector::CopyRowsFromMat), the same
for the columns (Vector::CopyColsFromMat), and do the reverse too
(Matrix::CopyRowsFromVec and Matrix::CopyColsFromVec). The same functions
exist for copying an individual row or column from a matrix to a vector
and vice versa: see Vector::CopyRowFromMat, Vector::CopyColFromMat,
Matrix::CopyRowFromVec and Matrix::CopyColFromVec. The row versions of these
operations are only provided for symmetry, because you could do the same
using SubVector in the row case (see \ref matrix_sec_sub).
\section matrix_sec_scalar Scalar products
Various functions that return scalars (and do not modify their parameters) are
defined not as class members but as functions.
See \ref matrix_funcs_scalar for a list of these.
Typical examples are:
\code
float f = VecVec(v, w), // v' * w
g = VecMatVec(v, M, w), // v' * M * w
h = TraceMatMat(M, N, kNoTrans); // tr(M * N)
\endcode
\section matric_sec_resizing Resizing
All matrix and vector types (except for SubMatrix and SubVector) may be resized.
Examples of this are below:
\code
Vector<float> v;
Matrix<float> M;
SpMatrix<float> S;
v.Resize(10);
M.Resize(5, 10);
S.Resize(10);
\endcode
The Resize functions will set the new data to zero unless you provide an optional
argument of type \ref MatrixResizeType. The possible values are:
- \ref kSetZero (the default): sets the data to zero
- \ref kUndefined : leaves the data undefined
- \ref kCopyData : Copies any old data that shared the same index as the new
data, leaving new indices at zero.
Thus, the code
\code
v.Resize(v.Dim() + 1, kCopyData);
\endcode
will append a zero to v. This is not a particularly efficient way to append
something to a list, though (it will newly allocate the whole array).
Constructors that take dimension arguments also optionally take
a ResizeType argument (the default is kSetZero), so constructors behave
in the same way as the Resize functions.
\section matrix_sec_io Matrix I/O
%Matrix I/O is done in the same style as other Kaldi code (see \ref io). A typical example
of reading and writing these types is:
\code
bool binary = false;
std::ofstream os( ... );
Vector<float> v;
v.Write(os, binary);
...
std::ifstream is( ... );
Vector<float> v;
v.Read(is, binary);
\endcode
There also stream operators << and >> that do the same thing (in text mode only, not
binary), but these are mainly intended for debugging. The text format of these
objects looks like (for a vector):
\code
[ 2.77914 1.50866 1.01017 0.263783 ]
\endcode
and for a matrix:
\code
[ -0.825915 -0.899772 0.158694 -0.731197
0.141949 0.827187 0.62493 -0.67114
-0.814922 -0.367702 -0.155766 -0.135577
0.286447 0.648148 -0.643773 0.724163
]
\endcode
\section matrix_sec_other Other matrix library functions
This page is only a brief introduction to the matrix library.
Most mathematical operations are members of the classes MatrixBase, VectorBase,
SpMatrix and TpMatrix, and can be discovered there. Functions returning
scalars are listed \ref matrix_funcs_scalar "here". Some miscellaneous functions,
including fourier transforms and the matrix exponential, are implemented as
separate functions and classes and may be found \ref matrix_funcs_misc "here".
We note that not all parts of the library have been optimized as fully as they could
be. Our approach has been to quickly implement functionality that we need,
and only optimize those parts that are taking a substantial amount of time
in some scenario.
\defgroup matrix_funcs_io I/O related functions and classes
\ingroup matrix
Various namespace-scope functions for I/O. Note that the << and >> operators
are not recommended for normal use in Kaldi-style I/O (for that, use the Read
and Write methods). The operators are mainly useful for logging and error output.
*/
/**
\defgroup matrix_group %Matrix and vector classes
For an overview of the matrix library, see \ref matrix.
\defgroup matrix_funcs_scalar Matrix-vector functions returning scalars
\ingroup matrix_group
\defgroup matrix_funcs_misc Miscellaneous matrix/vector library functions and classes
\ingroup matrix_group
*/
}