-
Notifications
You must be signed in to change notification settings - Fork 0
/
mgl-manual.html
5174 lines (4642 loc) · 506 KB
/
mgl-manual.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html xmlns='http://www.w3.org/1999/xhtml' xml:lang='en' lang='en'>
<head>
<title>MGL Manual
</title>
<link type='text/css' href='style.css' rel='stylesheet'>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="viewport" content="width=device-width">
<script src="jquery.min.js"></script>
<script src="toc.min.js"></script>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
tex2jax: {
inlineMath: [['$','$']],
processEscapes: true
}
});
</script>
<script async src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.7/MathJax.js?config=TeX-MML-AM_CHTML">
</script>
</head>
<body>
<div id="content-container">
<div id="toc">
<div id="toc-header"><ul><li><a href="index.html">PAX World</a></li></ul></div>
<div id="page-toc">
</div>
<div id="toc-footer"><ul><li><a href="https://github.com/melisgl/mgl-pax">[generated by MGL-PAX]</a></li></ul></div>
</div>
<div id="content">
<p><a id="x-28MGL-3A-40MGL-MANUAL-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL:@MGL-MANUAL%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#%22mgl%22%20ASDF%2FSYSTEM:SYSTEM" title=""mgl" ASDF/SYSTEM:SYSTEM">→</a> <a href="mgl-manual.html" title="MGL Manual">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/mgl.lisp#L3">λ</a></span></span></p>
<h1><a href="mgl-manual.html">MGL Manual</a></h1>
<h2>Table of Contents</h2>
<ul>
<li><a href="#%22mgl%22%20ASDF%2FSYSTEM:SYSTEM" title=""mgl" ASDF/SYSTEM:SYSTEM">1 The mgl ASDF System</a></li>
<li><a href="#MGL:@MGL-INTRODUCTION%20MGL-PAX:SECTION" title="Introduction">2 Introduction</a>
<ul>
<li><a href="#MGL:@MGL-OVERVIEW%20MGL-PAX:SECTION" title="Overview">2.1 Overview</a></li>
<li><a href="#MGL:@MGL-LINKS%20MGL-PAX:SECTION" title="Links">2.2 Links</a></li>
<li><a href="#MGL:@MGL-DEPENDENCIES%20MGL-PAX:SECTION" title="Dependencies">2.3 Dependencies</a></li>
<li><a href="#MGL:@MGL-CODE-ORGANIZATION%20MGL-PAX:SECTION" title="Code Organization">2.4 Code Organization</a></li>
<li><a href="#MGL:@MGL-GLOSSARY%20MGL-PAX:SECTION" title="Glossary">2.5 Glossary</a></li>
</ul></li>
<li><a href="#MGL-DATASET:@MGL-DATASET%20MGL-PAX:SECTION" title="Datasets">3 Datasets</a>
<ul>
<li><a href="#MGL-DATASET:@MGL-SAMPLER%20MGL-PAX:SECTION" title="Samplers">3.1 Samplers</a>
<ul>
<li><a href="#MGL-DATASET:@MGL-SAMPLER-FUNCTION-SAMPLER%20MGL-PAX:SECTION" title="Function Sampler">3.1.1 Function Sampler</a></li>
</ul></li>
</ul></li>
<li><a href="#MGL-RESAMPLE:@MGL-RESAMPLE%20MGL-PAX:SECTION" title="Resampling">4 Resampling</a>
<ul>
<li><a href="#MGL-RESAMPLE:@MGL-RESAMPLE-PARTITIONS%20MGL-PAX:SECTION" title="Partitions">4.1 Partitions</a></li>
<li><a href="#MGL-RESAMPLE:@MGL-RESAMPLE-CROSS-VALIDATION%20MGL-PAX:SECTION" title="Cross-validation">4.2 Cross-validation</a></li>
<li><a href="#MGL-RESAMPLE:@MGL-RESAMPLE-BAGGING%20MGL-PAX:SECTION" title="Bagging">4.3 Bagging</a></li>
<li><a href="#MGL-RESAMPLE:@MGL-RESAMPLE-CV-BAGGING%20MGL-PAX:SECTION" title="CV Bagging">4.4 CV Bagging</a></li>
<li><a href="#MGL-RESAMPLE:@MGL-RESAMPLE-MISC%20MGL-PAX:SECTION" title="Miscellaneous Operations">4.5 Miscellaneous Operations</a></li>
</ul></li>
<li><a href="#MGL-CORE:@MGL-CORE%20MGL-PAX:SECTION" title="Core">5 Core</a>
<ul>
<li><a href="#MGL-CORE:@MGL-PERSISTENCE%20MGL-PAX:SECTION" title="Persistence">5.1 Persistence</a></li>
<li><a href="#MGL-CORE:@MGL-MODEL-STRIPE%20MGL-PAX:SECTION" title="Batch Processing">5.2 Batch Processing</a></li>
<li><a href="#MGL-CORE:@MGL-EXECUTORS%20MGL-PAX:SECTION" title="Executors">5.3 Executors</a>
<ul>
<li><a href="#MGL-CORE:@MGL-PARAMETERIZED-EXECUTOR-CACHE%20MGL-PAX:SECTION" title="Parameterized Executor Cache">5.3.1 Parameterized Executor Cache</a></li>
</ul></li>
</ul></li>
<li><a href="#MGL-CORE:@MGL-MONITORING%20MGL-PAX:SECTION" title="Monitoring">6 Monitoring</a>
<ul>
<li><a href="#MGL-CORE:@MGL-MONITOR%20MGL-PAX:SECTION" title="Monitors">6.1 Monitors</a></li>
<li><a href="#MGL-CORE:@MGL-MEASURER%20MGL-PAX:SECTION" title="Measurers">6.2 Measurers</a></li>
<li><a href="#MGL-CORE:@MGL-COUNTER%20MGL-PAX:SECTION" title="Counters">6.3 Counters</a>
<ul>
<li><a href="#MGL-CORE:@MGL-ATTRIBUTES%20MGL-PAX:SECTION" title="Attributes">6.3.1 Attributes</a></li>
<li><a href="#MGL-CORE:@MGL-COUNTER-CLASSES%20MGL-PAX:SECTION" title="Counter classes">6.3.2 Counter classes</a></li>
</ul></li>
</ul></li>
<li><a href="#MGL-CORE:@MGL-CLASSIFICATION%20MGL-PAX:SECTION" title="Classification">7 Classification</a>
<ul>
<li><a href="#MGL-CORE:@MGL-CLASSIFICATION-MONITOR%20MGL-PAX:SECTION" title="Classification Monitors">7.1 Classification Monitors</a></li>
<li><a href="#MGL-CORE:@MGL-CLASSIFICATION-MEASURER%20MGL-PAX:SECTION" title="Classification Measurers">7.2 Classification Measurers</a></li>
<li><a href="#MGL-CORE:@MGL-CLASSIFICATION-COUNTER%20MGL-PAX:SECTION" title="Classification Counters">7.3 Classification Counters</a>
<ul>
<li><a href="#MGL-CORE:@MGL-CONFUSION-MATRIX%20MGL-PAX:SECTION" title="Confusion Matrices">7.3.1 Confusion Matrices</a></li>
</ul></li>
</ul></li>
<li><a href="#MGL-CORE:@MGL-FEATURES%20MGL-PAX:SECTION" title="Features">8 Features</a>
<ul>
<li><a href="#MGL-CORE:@MGL-FEATURE-SELECTION%20MGL-PAX:SECTION" title="Feature Selection">8.1 Feature Selection</a></li>
<li><a href="#MGL-CORE:@MGL-FEATURE-ENCODING%20MGL-PAX:SECTION" title="Feature Encoding">8.2 Feature Encoding</a></li>
</ul></li>
<li><a href="#MGL-OPT:@MGL-OPT%20MGL-PAX:SECTION" title="Gradient Based Optimization">9 Gradient Based Optimization</a>
<ul>
<li><a href="#MGL-OPT:@MGL-OPT-ITERATIVE-OPTIMIZER%20MGL-PAX:SECTION" title="Iterative Optimizer">9.1 Iterative Optimizer</a></li>
<li><a href="#MGL-OPT:@MGL-OPT-COST%20MGL-PAX:SECTION" title="Cost Function">9.2 Cost Function</a></li>
<li><a href="#MGL-GD:@MGL-GD%20MGL-PAX:SECTION" title="Gradient Descent">9.3 Gradient Descent</a>
<ul>
<li><a href="#MGL-GD:@MGL-GD-BATCH-GD-OPTIMIZER%20MGL-PAX:SECTION" title="Batch Based Optimizers">9.3.1 Batch Based Optimizers</a></li>
<li><a href="#MGL-GD:@MGL-GD-SEGMENTED-GD-OPTIMIZER%20MGL-PAX:SECTION" title="Segmented GD Optimizer">9.3.2 Segmented GD Optimizer</a></li>
<li><a href="#MGL-GD:@MGL-GD-PER-WEIGHT-OPTIMIZATION%20MGL-PAX:SECTION" title="Per-weight Optimization">9.3.3 Per-weight Optimization</a></li>
<li><a href="#MGL-GD:@MGL-GD-UTILITIES%20MGL-PAX:SECTION" title="Utilities">9.3.4 Utilities</a></li>
</ul></li>
<li><a href="#MGL-CG:@MGL-CG%20MGL-PAX:SECTION" title="Conjugate Gradient">9.4 Conjugate Gradient</a></li>
<li><a href="#MGL-OPT:@MGL-OPT-EXTENSION-API%20MGL-PAX:SECTION" title="Extension API">9.5 Extension API</a>
<ul>
<li><a href="#MGL-OPT:@MGL-OPT-OPTIMIZER%20MGL-PAX:SECTION" title="Implementing Optimizers">9.5.1 Implementing Optimizers</a></li>
<li><a href="#MGL-OPT:@MGL-OPT-GRADIENT-SOURCE%20MGL-PAX:SECTION" title="Implementing Gradient Sources">9.5.2 Implementing Gradient Sources</a></li>
<li><a href="#MGL-OPT:@MGL-OPT-GRADIENT-SINK%20MGL-PAX:SECTION" title="Implementing Gradient Sinks">9.5.3 Implementing Gradient Sinks</a></li>
</ul></li>
</ul></li>
<li><a href="#MGL-DIFFUN:@MGL-DIFFUN%20MGL-PAX:SECTION" title="Differentiable Functions">10 Differentiable Functions</a></li>
<li><a href="#MGL-BP:@MGL-BP%20MGL-PAX:SECTION" title="Backpropagation Neural Networks">11 Backpropagation Neural Networks</a>
<ul>
<li><a href="#MGL-BP:@MGL-BP-OVERVIEW%20MGL-PAX:SECTION" title="Backprop Overview">11.1 Backprop Overview</a></li>
<li><a href="#MGL-BP:@MGL-BP-EXTENSION-API%20MGL-PAX:SECTION" title="Clump API">11.2 Clump API</a></li>
<li><a href="#MGL-BP:@MGL-BPN%20MGL-PAX:SECTION" title="`bpn`s">11.3 <code>bpn</code>s</a>
<ul>
<li><a href="#MGL-BP:@MGL-BP-TRAINING%20MGL-PAX:SECTION" title="Training">11.3.1 Training</a></li>
<li><a href="#MGL-BP:@MGL-BP-MONITORING%20MGL-PAX:SECTION" title="Monitoring">11.3.2 Monitoring</a></li>
<li><a href="#MGL-BP:@MGL-FNN%20MGL-PAX:SECTION" title="Feed-Forward Nets">11.3.3 Feed-Forward Nets</a></li>
<li><a href="#MGL-BP:@MGL-RNN%20MGL-PAX:SECTION" title="Recurrent Neural Nets">11.3.4 Recurrent Neural Nets</a></li>
</ul></li>
<li><a href="#MGL-BP:@MGL-BP-LUMPS%20MGL-PAX:SECTION" title="Lumps">11.4 Lumps</a>
<ul>
<li><a href="#MGL-BP:@MGL-BP-LUMP%20MGL-PAX:SECTION" title="Lump Base Class">11.4.1 Lump Base Class</a></li>
<li><a href="#MGL-BP:@MGL-BP-INPUTS%20MGL-PAX:SECTION" title="Inputs">11.4.2 Inputs</a></li>
<li><a href="#MGL-BP:@MGL-BP-WEIGHT-LUMP%20MGL-PAX:SECTION" title="Weight Lump">11.4.3 Weight Lump</a></li>
<li><a href="#MGL-BP:@MGL-BP-ACTIVATIONS%20MGL-PAX:SECTION" title="Activations">11.4.4 Activations</a></li>
<li><a href="#MGL-BP:@MGL-BP-ACTIVATION-FUNCTIONS%20MGL-PAX:SECTION" title="Activation Functions">11.4.5 Activation Functions</a></li>
<li><a href="#MGL-BP:@MGL-BP-LOSSES%20MGL-PAX:SECTION" title="Losses">11.4.6 Losses</a></li>
<li><a href="#MGL-BP:@MGL-BP-STOCHASTICITY%20MGL-PAX:SECTION" title="Stochasticity">11.4.7 Stochasticity</a></li>
<li><a href="#MGL-BP:@MGL-BP-ARITHMETIC%20MGL-PAX:SECTION" title="Arithmetic">11.4.8 Arithmetic</a></li>
<li><a href="#MGL-BP:@MGL-BP-RNN-OPERATIONS%20MGL-PAX:SECTION" title="Operations for `rnn`s">11.4.9 Operations for <code>rnn</code>s</a></li>
</ul></li>
<li><a href="#MGL-BP:@MGL-BP-UTILITIES%20MGL-PAX:SECTION" title="Utilities">11.5 Utilities</a></li>
</ul></li>
<li><a href="#MGL:@MGL-BM%20MGL-PAX:SECTION" title="Boltzmann Machines">12 Boltzmann Machines</a></li>
<li><a href="#MGL:@MGL-GP%20MGL-PAX:SECTION" title="Gaussian Processes">13 Gaussian Processes</a></li>
<li><a href="#MGL-NLP:@MGL-NLP%20MGL-PAX:SECTION" title="Natural Language Processing">14 Natural Language Processing</a>
<ul>
<li><a href="#MGL-NLP:@MGL-NLP-BAG-OF-WORDS%20MGL-PAX:SECTION" title="Bag of Words">14.1 Bag of Words</a></li>
</ul></li>
</ul>
<h6>[in package MGL]</h6>
<p><a id="x-28-22mgl-22-20ASDF-2FSYSTEM-3ASYSTEM-29"></a>
<a id="%22mgl%22%20ASDF%2FSYSTEM:SYSTEM"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="mgl-manual.html" title="MGL Manual">←</a> <a href="mgl-manual.html" title="MGL Manual">↑</a> <a href="#MGL:@MGL-INTRODUCTION%20MGL-PAX:SECTION" title="Introduction">→</a> <a href="#%22mgl%22%20ASDF%2FSYSTEM:SYSTEM" title=""mgl" ASDF/SYSTEM:SYSTEM">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/mgl.asd#L1">λ</a></span></span></p>
<h2><a href="#%22mgl%22%20ASDF%2FSYSTEM:SYSTEM">1 The mgl ASDF System</a></h2>
<ul>
<li>Version: 0.1.0</li>
<li>Description: <code>mgl</code> is a machine learning library for backpropagation
neural networks, boltzmann machines, gaussian processes and more.</li>
<li>Licence: MIT, see COPYING.</li>
<li>Author: Gábor Melis <a href="mailto:mega@retes.hu">mailto:mega@retes.hu</a></li>
<li>Mailto: <a href="mailto:mega@retes.hu" >mega@retes.hu</a></li>
<li>Homepage: <a href="http://melisgl.github.io/mgl" >http://melisgl.github.io/mgl</a></li>
<li>Bug tracker: <a href="https://github.com/melisgl/mgl/issues" >https://github.com/melisgl/mgl/issues</a></li>
<li>Source control: <a href="https://github.com/melisgl/mgl.git" >GIT</a></li>
</ul>
<p><a id="x-28MGL-3A-40MGL-INTRODUCTION-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL:@MGL-INTRODUCTION%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#%22mgl%22%20ASDF%2FSYSTEM:SYSTEM" title=""mgl" ASDF/SYSTEM:SYSTEM">←</a> <a href="mgl-manual.html" title="MGL Manual">↑</a> <a href="#MGL:@MGL-OVERVIEW%20MGL-PAX:SECTION" title="Overview">→</a> <a href="#MGL:@MGL-INTRODUCTION%20MGL-PAX:SECTION" title="Introduction">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/mgl.lisp#L19">λ</a></span></span></p>
<h2><a href="#MGL:@MGL-INTRODUCTION%20MGL-PAX:SECTION">2 Introduction</a></h2>
<p><a id="x-28MGL-3A-40MGL-OVERVIEW-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL:@MGL-OVERVIEW%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL:@MGL-INTRODUCTION%20MGL-PAX:SECTION" title="Introduction">←</a> <a href="#MGL:@MGL-INTRODUCTION%20MGL-PAX:SECTION" title="Introduction">↑</a> <a href="#MGL:@MGL-LINKS%20MGL-PAX:SECTION" title="Links">→</a> <a href="#MGL:@MGL-OVERVIEW%20MGL-PAX:SECTION" title="Overview">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/mgl.lisp#L26">λ</a></span></span></p>
<h3><a href="#MGL:@MGL-OVERVIEW%20MGL-PAX:SECTION">2.1 Overview</a></h3>
<p>MGL is a Common Lisp machine learning library by <a href="http://quotenil.com" >Gábor
Melis</a> with some parts originally contributed
by Ravenpack International. It mainly concentrates on various forms
of neural networks (boltzmann machines, feed-forward and recurrent
backprop nets). Most of MGL is built on top of
<a href="mat-manual.html" title="MAT Manual">MGL-MAT</a> so it has BLAS and CUDA support.</p>
<p>In general, the focus is on power and performance not on ease of
use. Perhaps one day there will be a cookie cutter interface with
restricted functionality if a reasonable compromise is found between
power and utility.</p>
<p><a id="x-28MGL-3A-40MGL-LINKS-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL:@MGL-LINKS%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL:@MGL-OVERVIEW%20MGL-PAX:SECTION" title="Overview">←</a> <a href="#MGL:@MGL-INTRODUCTION%20MGL-PAX:SECTION" title="Introduction">↑</a> <a href="#MGL:@MGL-DEPENDENCIES%20MGL-PAX:SECTION" title="Dependencies">→</a> <a href="#MGL:@MGL-LINKS%20MGL-PAX:SECTION" title="Links">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/mgl.lisp#L39">λ</a></span></span></p>
<h3><a href="#MGL:@MGL-LINKS%20MGL-PAX:SECTION">2.2 Links</a></h3>
<p>Here is the <a href="https://github.com/melisgl/mgl" >official repository</a>
and the <a href="http://melisgl.github.io/mgl-pax-world/mgl-manual.html" >HTML
documentation</a>
for the latest version.</p>
<p><a id="x-28MGL-3A-40MGL-DEPENDENCIES-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL:@MGL-DEPENDENCIES%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL:@MGL-LINKS%20MGL-PAX:SECTION" title="Links">←</a> <a href="#MGL:@MGL-INTRODUCTION%20MGL-PAX:SECTION" title="Introduction">↑</a> <a href="#MGL:@MGL-CODE-ORGANIZATION%20MGL-PAX:SECTION" title="Code Organization">→</a> <a href="#MGL:@MGL-DEPENDENCIES%20MGL-PAX:SECTION" title="Dependencies">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/mgl.lisp#L45">λ</a></span></span></p>
<h3><a href="#MGL:@MGL-DEPENDENCIES%20MGL-PAX:SECTION">2.3 Dependencies</a></h3>
<p>MGL used to rely on <a href="https://github.com/tpapp/lla" >LLA</a> to
interface to BLAS and LAPACK. That's mostly history by now, but
configuration of foreign libraries is still done via LLA. See the
README in LLA on how to set things up. Note that these days OpenBLAS
is easier to set up and just as fast as ATLAS.</p>
<p><a href="https://github.com/takagi/cl-cuda" >CL-CUDA</a> and
<a href="https://github.com/melisgl/mgl" >MGL-MAT</a> are the two main
dependencies and also the ones not yet in quicklisp, so just drop
them into <code>quicklisp/local-projects/</code>. If there is no suitable GPU
on the system or the CUDA SDK is not installed, MGL will simply
fall back on using BLAS and Lisp code. Wrapping code in
<a href="mat-manual.html#MGL-MAT:WITH-CUDA*%20MGL-PAX:MACRO" title="MGL-MAT:WITH-CUDA* MGL-PAX:MACRO"><code>mgl-mat:with-cuda*</code></a> is basically all that's needed to run on the GPU,
and with <a href="mat-manual.html#MGL-MAT:CUDA-AVAILABLE-P%20FUNCTION" title="MGL-MAT:CUDA-AVAILABLE-P FUNCTION"><code>mgl-mat:cuda-available-p</code></a> one can check whether the GPU is
really being used.</p>
<p><a id="x-28MGL-3A-40MGL-CODE-ORGANIZATION-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL:@MGL-CODE-ORGANIZATION%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL:@MGL-DEPENDENCIES%20MGL-PAX:SECTION" title="Dependencies">←</a> <a href="#MGL:@MGL-INTRODUCTION%20MGL-PAX:SECTION" title="Introduction">↑</a> <a href="#MGL:@MGL-GLOSSARY%20MGL-PAX:SECTION" title="Glossary">→</a> <a href="#MGL:@MGL-CODE-ORGANIZATION%20MGL-PAX:SECTION" title="Code Organization">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/mgl.lisp#L62">λ</a></span></span></p>
<h3><a href="#MGL:@MGL-CODE-ORGANIZATION%20MGL-PAX:SECTION">2.4 Code Organization</a></h3>
<p>MGL consists of several packages dedicated to different tasks.
For example, package <code>mgl-resample</code> is about
<a href="#MGL-RESAMPLE:@MGL-RESAMPLE%20MGL-PAX:SECTION" title="Resampling">Resampling</a> and <code>mgl-gd</code> is about <a href="#MGL-GD:@MGL-GD%20MGL-PAX:SECTION" title="Gradient Descent">Gradient Descent</a>
and so on. On one hand, having many packages makes it easier to
cleanly separate API and implementation and also to explore into a
specific task. At other times, they can be a hassle, so the <code>mgl</code>
package itself reexports every external symbol found in all the
other packages that make up MGL and MGL-MAT (see
<a href="mat-manual.html" title="MAT Manual">MAT Manual</a>) on which it heavily relies.</p>
<p>One exception to this rule is the bundled, but independent
MGL-GNUPLOT library.</p>
<p>The built in tests can be run with:</p>
<pre><code>(ASDF:OOS 'ASDF:TEST-OP '#:MGL)
</code></pre>
<p>Note, that most of the tests are rather stochastic and can fail once
in a while.</p>
<p><a id="x-28MGL-3A-40MGL-GLOSSARY-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL:@MGL-GLOSSARY%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL:@MGL-CODE-ORGANIZATION%20MGL-PAX:SECTION" title="Code Organization">←</a> <a href="#MGL:@MGL-INTRODUCTION%20MGL-PAX:SECTION" title="Introduction">↑</a> <a href="#MGL-DATASET:@MGL-DATASET%20MGL-PAX:SECTION" title="Datasets">→</a> <a href="#MGL:@MGL-GLOSSARY%20MGL-PAX:SECTION" title="Glossary">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/mgl.lisp#L100">λ</a></span></span></p>
<h3><a href="#MGL:@MGL-GLOSSARY%20MGL-PAX:SECTION">2.5 Glossary</a></h3>
<p>Ultimately machine learning is about creating <strong>models</strong> of some
domain. The observations in the modelled domain are called
<strong>instances</strong> (also known as examples or samples). Sets of instances
are called <strong>datasets</strong>. Datasets are used when fitting a model or
when making <strong>predictions</strong>. Sometimes the word predictions is too
specific, and the results obtained from applying a model to some
instances are simply called <strong>results</strong>.</p>
<p><a id="x-28MGL-DATASET-3A-40MGL-DATASET-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL-DATASET:@MGL-DATASET%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL:@MGL-GLOSSARY%20MGL-PAX:SECTION" title="Glossary">←</a> <a href="mgl-manual.html" title="MGL Manual">↑</a> <a href="#MGL-DATASET:@MGL-SAMPLER%20MGL-PAX:SECTION" title="Samplers">→</a> <a href="#MGL-DATASET:@MGL-DATASET%20MGL-PAX:SECTION" title="Datasets">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/dataset.lisp#L3">λ</a></span></span></p>
<h2><a href="#MGL-DATASET:@MGL-DATASET%20MGL-PAX:SECTION">3 Datasets</a></h2>
<h6>[in package MGL-DATASET]</h6>
<p>An instance can often be any kind of object of the user's choice.
It is typically represented by a set of numbers which is called a
feature vector or by a structure holding the feature vector, the
label, etc. A dataset is a <a href="http://www.lispworks.com/documentation/HyperSpec/Body/t_seq.htm" title="SEQUENCE (MGL-PAX:CLHS CLASS)"><code>sequence</code></a> of such instances or a
<a href="#MGL-DATASET:@MGL-SAMPLER%20MGL-PAX:SECTION" title="Samplers">Samplers</a> object that produces instances.</p>
<p><a id="x-28MGL-DATASET-3AMAP-DATASET-20FUNCTION-29"></a>
<a id="MGL-DATASET:MAP-DATASET%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/dataset.lisp#L18">[function]</a></span> <span class="reference-object"><a href="#MGL-DATASET:MAP-DATASET%20FUNCTION" >map-dataset</a></span></span> <span class="locative-args">fn dataset</span></span></p>
<p>Call <code>fn</code> with each instance in <code>dataset</code>. This is basically equivalent
to iterating over the elements of a sequence or a sampler (see
<a href="#MGL-DATASET:@MGL-SAMPLER%20MGL-PAX:SECTION" title="Samplers">Samplers</a>).</p></li>
</ul>
<p><a id="x-28MGL-DATASET-3AMAP-DATASETS-20FUNCTION-29"></a>
<a id="MGL-DATASET:MAP-DATASETS%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/dataset.lisp#L26">[function]</a></span> <span class="reference-object"><a href="#MGL-DATASET:MAP-DATASETS%20FUNCTION" >map-datasets</a></span></span> <span class="locative-args">fn datasets &key (impute nil imputep)</span></span></p>
<p>Call <code>fn</code> with a list of instances, one from each dataset in
<code>datasets</code>. Return nothing. If <code>impute</code> is specified then iterate until
the largest dataset is consumed imputing <code>impute</code> for missing values.
If <code>impute</code> is not specified then iterate until the smallest dataset
runs out.</p>
<pre><code><span class="code"><span class="paren1">(<span class="code">map-datasets #'prin1 '<span class="paren2">(<span class="code"><span class="paren3">(<span class="code">0 1 2</span>)</span> <span class="paren3">(<span class="code"><span class="keyword">:a</span> <span class="keyword">:b</span></span>)</span></span>)</span></span>)</span>
.. <span class="paren1">(<span class="code">0 <span class="keyword">:A</span></span>)</span><span class="paren1">(<span class="code">1 <span class="keyword">:B</span></span>)</span>
<span class="paren1">(<span class="code">map-datasets #'prin1 '<span class="paren2">(<span class="code"><span class="paren3">(<span class="code">0 1 2</span>)</span> <span class="paren3">(<span class="code"><span class="keyword">:a</span> <span class="keyword">:b</span></span>)</span></span>)</span> <span class="keyword">:impute</span> nil</span>)</span>
.. <span class="paren1">(<span class="code">0 <span class="keyword">:A</span></span>)</span><span class="paren1">(<span class="code">1 <span class="keyword">:B</span></span>)</span><span class="paren1">(<span class="code">2 NIL</span>)</span></span></code></pre>
<p>It is of course allowed to mix sequences with samplers:</p>
<pre><code><span class="code"><span class="paren1">(<span class="code">map-datasets #'prin1
<span class="paren2">(<span class="code">list '<span class="paren3">(<span class="code">0 1 2</span>)</span>
<span class="paren3">(<span class="code">make-sequence-sampler '<span class="paren4">(<span class="code"><span class="keyword">:a</span> <span class="keyword">:b</span></span>)</span> <span class="keyword">:max-n-samples</span> 2</span>)</span></span>)</span></span>)</span>
.. <span class="paren1">(<span class="code">0 <span class="keyword">:A</span></span>)</span><span class="paren1">(<span class="code">1 <span class="keyword">:B</span></span>)</span></span></code></pre></li>
</ul>
<p><a id="x-28MGL-DATASET-3A-40MGL-SAMPLER-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL-DATASET:@MGL-SAMPLER%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL-DATASET:@MGL-DATASET%20MGL-PAX:SECTION" title="Datasets">←</a> <a href="#MGL-DATASET:@MGL-DATASET%20MGL-PAX:SECTION" title="Datasets">↑</a> <a href="#MGL-DATASET:@MGL-SAMPLER-FUNCTION-SAMPLER%20MGL-PAX:SECTION" title="Function Sampler">→</a> <a href="#MGL-DATASET:@MGL-SAMPLER%20MGL-PAX:SECTION" title="Samplers">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/dataset.lisp#L69">λ</a></span></span></p>
<h3><a href="#MGL-DATASET:@MGL-SAMPLER%20MGL-PAX:SECTION">3.1 Samplers</a></h3>
<p>Some algorithms do not need random access to the entire dataset and
can work with a stream observations. Samplers are simple generators
providing two functions: <a href="#MGL-DATASET:SAMPLE%20GENERIC-FUNCTION" title="MGL-DATASET:SAMPLE GENERIC-FUNCTION"><code>sample</code></a> and <a href="#MGL-DATASET:FINISHEDP%20GENERIC-FUNCTION" title="MGL-DATASET:FINISHEDP GENERIC-FUNCTION"><code>finishedp</code></a>.</p>
<p><a id="x-28MGL-DATASET-3ASAMPLE-20GENERIC-FUNCTION-29"></a>
<a id="MGL-DATASET:SAMPLE%20GENERIC-FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/dataset.lisp#L81">[generic-function]</a></span> <span class="reference-object"><a href="#MGL-DATASET:SAMPLE%20GENERIC-FUNCTION" >sample</a></span></span> <span class="locative-args">sampler</span></span></p>
<p>If <code>sampler</code> has not run out of data (see <a href="#MGL-DATASET:FINISHEDP%20GENERIC-FUNCTION" title="MGL-DATASET:FINISHEDP GENERIC-FUNCTION"><code>finishedp</code></a>)
<code>sample</code> returns an object that represents a sample from the world to
be experienced or, in other words, simply something the can be used
as input for training or prediction. It is not allowed to call
<code>sample</code> if <code>sampler</code> is <code>finishedp</code>.</p></li>
</ul>
<p><a id="x-28MGL-DATASET-3AFINISHEDP-20GENERIC-FUNCTION-29"></a>
<a id="MGL-DATASET:FINISHEDP%20GENERIC-FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/dataset.lisp#L88">[generic-function]</a></span> <span class="reference-object"><a href="#MGL-DATASET:FINISHEDP%20GENERIC-FUNCTION" >finishedp</a></span></span> <span class="locative-args">sampler</span></span></p>
<p>See if <code>sampler</code> has run out of examples.</p></li>
</ul>
<p><a id="x-28MGL-DATASET-3ALIST-SAMPLES-20FUNCTION-29"></a>
<a id="MGL-DATASET:LIST-SAMPLES%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/dataset.lisp#L91">[function]</a></span> <span class="reference-object"><a href="#MGL-DATASET:LIST-SAMPLES%20FUNCTION" >list-samples</a></span></span> <span class="locative-args">sampler max-size</span></span></p>
<p>Return a list of samples of length at most <code>max-size</code> or less if
<code>sampler</code> runs out.</p></li>
</ul>
<p><a id="x-28MGL-DATASET-3AMAKE-SEQUENCE-SAMPLER-20FUNCTION-29"></a>
<a id="MGL-DATASET:MAKE-SEQUENCE-SAMPLER%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/dataset.lisp#L98">[function]</a></span> <span class="reference-object"><a href="#MGL-DATASET:MAKE-SEQUENCE-SAMPLER%20FUNCTION" >make-sequence-sampler</a></span></span> <span class="locative-args">seq &key max-n-samples</span></span></p>
<p>Create a sampler that returns elements of <code>seq</code> in their original
order. If <code>max-n-samples</code> is non-nil, then at most <code>max-n-samples</code> are
sampled.</p></li>
</ul>
<p><a id="x-28MGL-DATASET-3AMAKE-RANDOM-SAMPLER-20FUNCTION-29"></a>
<a id="MGL-DATASET:MAKE-RANDOM-SAMPLER%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/dataset.lisp#L106">[function]</a></span> <span class="reference-object"><a href="#MGL-DATASET:MAKE-RANDOM-SAMPLER%20FUNCTION" >make-random-sampler</a></span></span> <span class="locative-args">seq &key max-n-samples (reorder #'mgl-resample:shuffle)</span></span></p>
<p>Create a sampler that returns elements of <code>seq</code> in random order. If
<code>max-n-samples</code> is non-nil, then at most <code>max-n-samples</code> are sampled.
The first pass over a shuffled copy of <code>seq</code>, and this copy is
reshuffled whenever the sampler reaches the end of it. Shuffling is
performed by calling the <code>reorder</code> function.</p></li>
</ul>
<p><a id="x-28MGL-DATASET-3A-2AINFINITELY-EMPTY-DATASET-2A-20VARIABLE-29"></a>
<a id="MGL-DATASET:*INFINITELY-EMPTY-DATASET*%20VARIABLE"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/dataset.lisp#L177">[variable]</a></span> <span class="reference-object"><a href="#MGL-DATASET:*INFINITELY-EMPTY-DATASET*%20VARIABLE" >*infinitely-empty-dataset*</a></span></span> <span class="locative-args">#<function-sampler "infinitely empty" ></span></span></p>
<p>This is the default dataset for <a href="#MGL-OPT:MINIMIZE%20FUNCTION" title="MGL-OPT:MINIMIZE FUNCTION"><code>mgl-opt:minimize</code></a>. It's an infinite
stream of <code>nil</code>s.</p></li>
</ul>
<p><a id="x-28MGL-DATASET-3A-40MGL-SAMPLER-FUNCTION-SAMPLER-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL-DATASET:@MGL-SAMPLER-FUNCTION-SAMPLER%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL-DATASET:@MGL-SAMPLER%20MGL-PAX:SECTION" title="Samplers">←</a> <a href="#MGL-DATASET:@MGL-SAMPLER%20MGL-PAX:SECTION" title="Samplers">↑</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE%20MGL-PAX:SECTION" title="Resampling">→</a> <a href="#MGL-DATASET:@MGL-SAMPLER-FUNCTION-SAMPLER%20MGL-PAX:SECTION" title="Function Sampler">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/dataset.lisp#L118">λ</a></span></span></p>
<h4><a href="#MGL-DATASET:@MGL-SAMPLER-FUNCTION-SAMPLER%20MGL-PAX:SECTION">3.1.1 Function Sampler</a></h4>
<p><a id="x-28MGL-DATASET-3AFUNCTION-SAMPLER-20CLASS-29"></a>
<a id="MGL-DATASET:FUNCTION-SAMPLER%20CLASS"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/dataset.lisp#L125">[class]</a></span> <span class="reference-object"><a href="#MGL-DATASET:FUNCTION-SAMPLER%20CLASS" >function-sampler</a></span></span></span></p>
<p>A sampler with a function in its <a href="#MGL-DATASET:GENERATOR%20%28MGL-PAX:READER%20MGL-DATASET:FUNCTION-SAMPLER%29" title="MGL-DATASET:GENERATOR (MGL-PAX:READER MGL-DATASET:FUNCTION-SAMPLER)"><code>generator</code></a> that
produces a stream of samples which may or may not be finite
depending on <a href="#MGL-DATASET:MAX-N-SAMPLES%20%28MGL-PAX:ACCESSOR%20MGL-DATASET:FUNCTION-SAMPLER%29" title="MGL-DATASET:MAX-N-SAMPLES (MGL-PAX:ACCESSOR MGL-DATASET:FUNCTION-SAMPLER)"><code>max-n-samples</code></a>. <a href="#MGL-DATASET:FINISHEDP%20GENERIC-FUNCTION" title="MGL-DATASET:FINISHEDP GENERIC-FUNCTION"><code>finishedp</code></a> returns <code>t</code> iff <code>max-n-samples</code> is
non-nil, and it's not greater than the number of samples
generated (<a href="#MGL-DATASET:N-SAMPLES%20%28MGL-PAX:READER%20MGL-DATASET:FUNCTION-SAMPLER%29" title="MGL-DATASET:N-SAMPLES (MGL-PAX:READER MGL-DATASET:FUNCTION-SAMPLER)"><code>n-samples</code></a>).</p>
<pre><code>(list-samples (make-instance 'function-sampler
:generator (lambda ()
(random 10))
:max-n-samples 5)
10)
=> (3 5 2 3 3)
</code></pre></li>
</ul>
<p><a id="x-28MGL-DATASET-3AGENERATOR-20-28MGL-PAX-3AREADER-20MGL-DATASET-3AFUNCTION-SAMPLER-29-29"></a>
<a id="MGL-DATASET:GENERATOR%20%28MGL-PAX:READER%20MGL-DATASET:FUNCTION-SAMPLER%29"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/dataset.lisp#L126">[reader]</a></span> <span class="reference-object"><a href="#MGL-DATASET:GENERATOR%20%28MGL-PAX:READER%20MGL-DATASET:FUNCTION-SAMPLER%29" >generator</a></span></span> <span class="locative-args">function-sampler (:generator)</span></span></p>
<p>A generator function of no arguments that returns
the next sample.</p></li>
</ul>
<p><a id="x-28MGL-DATASET-3AMAX-N-SAMPLES-20-28MGL-PAX-3AACCESSOR-20MGL-DATASET-3AFUNCTION-SAMPLER-29-29"></a>
<a id="MGL-DATASET:MAX-N-SAMPLES%20%28MGL-PAX:ACCESSOR%20MGL-DATASET:FUNCTION-SAMPLER%29"></a></p>
<ul>
<li><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/dataset.lisp#L136">[accessor]</a></span> <span class="reference-object"><a href="#MGL-DATASET:MAX-N-SAMPLES%20%28MGL-PAX:ACCESSOR%20MGL-DATASET:FUNCTION-SAMPLER%29" >max-n-samples</a></span></span> <span class="locative-args">function-sampler (:max-n-samples = nil)</span></span></li>
</ul>
<p><a id="x-28MGL-COMMON-3ANAME-20-28MGL-PAX-3AREADER-20MGL-DATASET-3AFUNCTION-SAMPLER-29-29"></a>
<a id="MGL-COMMON:NAME%20%28MGL-PAX:READER%20MGL-DATASET:FUNCTION-SAMPLER%29"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/dataset.lisp#L140">[reader]</a></span> <span class="reference-object"><a href="#MGL-COMMON:NAME%20%28MGL-PAX:READER%20MGL-DATASET:FUNCTION-SAMPLER%29" >name</a></span></span> <span class="locative-args">function-sampler (:name = nil)</span></span></p>
<p>An arbitrary object naming the sampler. Only used
for printing the sampler object.</p></li>
</ul>
<p><a id="x-28MGL-DATASET-3AN-SAMPLES-20-28MGL-PAX-3AREADER-20MGL-DATASET-3AFUNCTION-SAMPLER-29-29"></a>
<a id="MGL-DATASET:N-SAMPLES%20%28MGL-PAX:READER%20MGL-DATASET:FUNCTION-SAMPLER%29"></a></p>
<ul>
<li><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/dataset.lisp#L131">[reader]</a></span> <span class="reference-object"><a href="#MGL-DATASET:N-SAMPLES%20%28MGL-PAX:READER%20MGL-DATASET:FUNCTION-SAMPLER%29" >n-samples</a></span></span> <span class="locative-args">function-sampler (:n-samples = 0)</span></span></li>
</ul>
<p><a id="x-28MGL-RESAMPLE-3A-40MGL-RESAMPLE-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL-RESAMPLE:@MGL-RESAMPLE%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL-DATASET:@MGL-SAMPLER-FUNCTION-SAMPLER%20MGL-PAX:SECTION" title="Function Sampler">←</a> <a href="mgl-manual.html" title="MGL Manual">↑</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE-PARTITIONS%20MGL-PAX:SECTION" title="Partitions">→</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE%20MGL-PAX:SECTION" title="Resampling">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L3">λ</a></span></span></p>
<h2><a href="#MGL-RESAMPLE:@MGL-RESAMPLE%20MGL-PAX:SECTION">4 Resampling</a></h2>
<h6>[in package MGL-RESAMPLE]</h6>
<p>The focus of this package is on resampling methods such as
cross-validation and bagging which can be used for model evaluation,
model selection, and also as a simple form of ensembling. Data
partitioning and sampling functions are also provided because they
tend to be used together with resampling.</p>
<p><a id="x-28MGL-RESAMPLE-3A-40MGL-RESAMPLE-PARTITIONS-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL-RESAMPLE:@MGL-RESAMPLE-PARTITIONS%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE%20MGL-PAX:SECTION" title="Resampling">←</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE%20MGL-PAX:SECTION" title="Resampling">↑</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE-CROSS-VALIDATION%20MGL-PAX:SECTION" title="Cross-validation">→</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE-PARTITIONS%20MGL-PAX:SECTION" title="Partitions">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L40">λ</a></span></span></p>
<h3><a href="#MGL-RESAMPLE:@MGL-RESAMPLE-PARTITIONS%20MGL-PAX:SECTION">4.1 Partitions</a></h3>
<p>The following functions partition a dataset (currently only
<a href="http://www.lispworks.com/documentation/HyperSpec/Body/t_seq.htm" title="SEQUENCE (MGL-PAX:CLHS CLASS)"><code>sequence</code></a>s are supported) into a number of partitions. For each
element in the original dataset there is exactly one partition that
contains it.</p>
<p><a id="x-28MGL-RESAMPLE-3AFRACTURE-20FUNCTION-29"></a>
<a id="MGL-RESAMPLE:FRACTURE%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L49">[function]</a></span> <span class="reference-object"><a href="#MGL-RESAMPLE:FRACTURE%20FUNCTION" >fracture</a></span></span> <span class="locative-args">fractions seq &key weight</span></span></p>
<p>Partition <code>seq</code> into a number of subsequences. <code>fractions</code> is either a
positive integer or a list of non-negative real numbers. <code>weight</code> is
<code>nil</code> or a function that returns a non-negative real number when
called with an element from <code>seq</code>. If <code>fractions</code> is a positive integer
then return a list of that many subsequences with equal sum of
weights bar rounding errors, else partition <code>seq</code> into subsequences,
where the sum of weights of subsequence I is proportional to element
I of <code>fractions</code>. If <code>weight</code> is <code>nil</code>, then it's element is assumed to
have the same weight.</p>
<p>To split into 5 sequences:</p>
<pre><code><span class="code"><span class="paren1">(<span class="code">fracture 5 '<span class="paren2">(<span class="code">0 1 2 3 4 5 6 7 8 9</span>)</span></span>)</span>
=> <span class="paren1">(<span class="code"><span class="paren2">(<span class="code">0 1</span>)</span> <span class="paren2">(<span class="code">2 3</span>)</span> <span class="paren2">(<span class="code">4 5</span>)</span> <span class="paren2">(<span class="code">6 7</span>)</span> <span class="paren2">(<span class="code">8 9</span>)</span></span>)</span></span></code></pre>
<p>To split into two sequences whose lengths are proportional to 2 and
3:</p>
<pre><code><span class="code"><span class="paren1">(<span class="code">fracture '<span class="paren2">(<span class="code">2 3</span>)</span> '<span class="paren2">(<span class="code">0 1 2 3 4 5 6 7 8 9</span>)</span></span>)</span>
=> <span class="paren1">(<span class="code"><span class="paren2">(<span class="code">0 1 2 3</span>)</span> <span class="paren2">(<span class="code">4 5 6 7 8 9</span>)</span></span>)</span></span></code></pre></li>
</ul>
<p><a id="x-28MGL-RESAMPLE-3ASTRATIFY-20FUNCTION-29"></a>
<a id="MGL-RESAMPLE:STRATIFY%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L112">[function]</a></span> <span class="reference-object"><a href="#MGL-RESAMPLE:STRATIFY%20FUNCTION" >stratify</a></span></span> <span class="locative-args">seq &key (key #'identity) (test #'eql)</span></span></p>
<p>Return the list of strata of <code>seq</code>. <code>seq</code> is a sequence of elements for
which the function <code>key</code> returns the class they belong to. Such
classes are opaque objects compared for equality with <code>test</code>. A
stratum is a sequence of elements with the same (under <code>test</code>) <code>key</code>.</p>
<pre><code><span class="code"><span class="paren1">(<span class="code">stratify '<span class="paren2">(<span class="code">0 1 2 3 4 5 6 7 8 9</span>)</span> <span class="keyword">:key</span> #'evenp</span>)</span>
=> <span class="paren1">(<span class="code"><span class="paren2">(<span class="code">0 2 4 6 8</span>)</span> <span class="paren2">(<span class="code">1 3 5 7 9</span>)</span></span>)</span></span></code></pre></li>
</ul>
<p><a id="x-28MGL-RESAMPLE-3AFRACTURE-STRATIFIED-20FUNCTION-29"></a>
<a id="MGL-RESAMPLE:FRACTURE-STRATIFIED%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L156">[function]</a></span> <span class="reference-object"><a href="#MGL-RESAMPLE:FRACTURE-STRATIFIED%20FUNCTION" >fracture-stratified</a></span></span> <span class="locative-args">fractions seq &key (key #'identity) (test #'eql) weight</span></span></p>
<p>Similar to <a href="#MGL-RESAMPLE:FRACTURE%20FUNCTION" title="MGL-RESAMPLE:FRACTURE FUNCTION"><code>fracture</code></a>, but also makes sure that keys are evenly
distributed among the partitions (see <a href="#MGL-RESAMPLE:STRATIFY%20FUNCTION" title="MGL-RESAMPLE:STRATIFY FUNCTION"><code>stratify</code></a>). It can be useful
for classification tasks to partition the data set while keeping the
distribution of classes the same.</p>
<p>Note that the sets returned are not in random order. In fact, they
are sorted internally by <code>key</code>.</p>
<p>For example, to make two splits with approximately the same number
of even and odd numbers:</p>
<pre><code><span class="code"><span class="paren1">(<span class="code">fracture-stratified 2 '<span class="paren2">(<span class="code">0 1 2 3 4 5 6 7 8 9</span>)</span> <span class="keyword">:key</span> #'evenp</span>)</span>
=> <span class="paren1">(<span class="code"><span class="paren2">(<span class="code">0 2 1 3</span>)</span> <span class="paren2">(<span class="code">4 6 8 5 7 9</span>)</span></span>)</span></span></code></pre></li>
</ul>
<p><a id="x-28MGL-RESAMPLE-3A-40MGL-RESAMPLE-CROSS-VALIDATION-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL-RESAMPLE:@MGL-RESAMPLE-CROSS-VALIDATION%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE-PARTITIONS%20MGL-PAX:SECTION" title="Partitions">←</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE%20MGL-PAX:SECTION" title="Resampling">↑</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE-BAGGING%20MGL-PAX:SECTION" title="Bagging">→</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE-CROSS-VALIDATION%20MGL-PAX:SECTION" title="Cross-validation">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L187">λ</a></span></span></p>
<h3><a href="#MGL-RESAMPLE:@MGL-RESAMPLE-CROSS-VALIDATION%20MGL-PAX:SECTION">4.2 Cross-validation</a></h3>
<p><a id="x-28MGL-RESAMPLE-3ACROSS-VALIDATE-20FUNCTION-29"></a>
<a id="MGL-RESAMPLE:CROSS-VALIDATE%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L193">[function]</a></span> <span class="reference-object"><a href="#MGL-RESAMPLE:CROSS-VALIDATE%20FUNCTION" >cross-validate</a></span></span> <span class="locative-args">data fn &key (n-folds 5) (folds (alexandria:iota n-folds)) (split-fn #'split-fold/mod) pass-fold</span></span></p>
<p>Map <code>fn</code> over the <code>folds</code> of <code>data</code> split with <code>split-fn</code> and collect the
results in a list. The simplest demonstration is:</p>
<pre><code><span class="code"><span class="paren1">(<span class="code">cross-validate '<span class="paren2">(<span class="code">0 1 2 3 4</span>)</span>
<span class="paren2">(<span class="code"><i><span class="symbol">lambda</span></i> <span class="paren3">(<span class="code">test training</span>)</span>
<span class="paren3">(<span class="code">list test training</span>)</span></span>)</span>
<span class="keyword">:n-folds</span> 5</span>)</span>
=> <span class="paren1">(<span class="code"><span class="paren2">(<span class="code"><span class="paren3">(<span class="code">0</span>)</span> <span class="paren3">(<span class="code">1 2 3 4</span>)</span></span>)</span>
<span class="paren2">(<span class="code"><span class="paren3">(<span class="code">1</span>)</span> <span class="paren3">(<span class="code">0 2 3 4</span>)</span></span>)</span>
<span class="paren2">(<span class="code"><span class="paren3">(<span class="code">2</span>)</span> <span class="paren3">(<span class="code">0 1 3 4</span>)</span></span>)</span>
<span class="paren2">(<span class="code"><span class="paren3">(<span class="code">3</span>)</span> <span class="paren3">(<span class="code">0 1 2 4</span>)</span></span>)</span>
<span class="paren2">(<span class="code"><span class="paren3">(<span class="code">4</span>)</span> <span class="paren3">(<span class="code">0 1 2 3</span>)</span></span>)</span></span>)</span></span></code></pre>
<p>Of course, in practice one would typically train a model and return
the trained model and/or its score on <code>test</code>. Also, sometimes one may
want to do only some of the folds and remember which ones they were:</p>
<pre><code><span class="code"><span class="paren1">(<span class="code">cross-validate '<span class="paren2">(<span class="code">0 1 2 3 4</span>)</span>
<span class="paren2">(<span class="code"><i><span class="symbol">lambda</span></i> <span class="paren3">(<span class="code">fold test training</span>)</span>
<span class="paren3">(<span class="code">list <span class="keyword">:fold</span> fold test training</span>)</span></span>)</span>
<span class="keyword">:folds</span> '<span class="paren2">(<span class="code">2 3</span>)</span>
<span class="keyword">:pass-fold</span> t</span>)</span>
=> <span class="paren1">(<span class="code"><span class="paren2">(<span class="code"><span class="keyword">:fold</span> 2 <span class="paren3">(<span class="code">2</span>)</span> <span class="paren3">(<span class="code">0 1 3 4</span>)</span></span>)</span>
<span class="paren2">(<span class="code"><span class="keyword">:fold</span> 3 <span class="paren3">(<span class="code">3</span>)</span> <span class="paren3">(<span class="code">0 1 2 4</span>)</span></span>)</span></span>)</span></span></code></pre>
<p>Finally, the way the data is split can be customized. By default
<a href="#MGL-RESAMPLE:SPLIT-FOLD%2FMOD%20FUNCTION" title="MGL-RESAMPLE:SPLIT-FOLD/MOD FUNCTION"><code>split-fold/mod</code></a> is called with the arguments <code>data</code>, the fold (from
among <code>folds</code>) and <code>n-folds</code>. <code>split-fold/mod</code> returns two values which
are then passed on to <code>fn</code>. One can use <a href="#MGL-RESAMPLE:SPLIT-FOLD%2FCONT%20FUNCTION" title="MGL-RESAMPLE:SPLIT-FOLD/CONT FUNCTION"><code>split-fold/cont</code></a> or
<a href="#MGL-RESAMPLE:SPLIT-STRATIFIED%20FUNCTION" title="MGL-RESAMPLE:SPLIT-STRATIFIED FUNCTION"><code>split-stratified</code></a> or any other function that works with these
arguments. The only real constraint is that <code>fn</code> has to take as many
arguments (plus the fold argument if <code>pass-fold</code>) as <code>split-fn</code>
returns.</p></li>
</ul>
<p><a id="x-28MGL-RESAMPLE-3ASPLIT-FOLD-2FMOD-20FUNCTION-29"></a>
<a id="MGL-RESAMPLE:SPLIT-FOLD%2FMOD%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L244">[function]</a></span> <span class="reference-object"><a href="#MGL-RESAMPLE:SPLIT-FOLD%2FMOD%20FUNCTION" >split-fold/mod</a></span></span> <span class="locative-args">seq fold n-folds</span></span></p>
<p>Partition <code>seq</code> into two sequences: one with elements of <code>seq</code> with
indices whose remainder is <code>fold</code> when divided with <code>n-folds</code>, and a
second one with the rest. The second one is the larger set. The
order of elements remains stable. This function is suitable as the
<code>split-fn</code> argument of <a href="#MGL-RESAMPLE:CROSS-VALIDATE%20FUNCTION" title="MGL-RESAMPLE:CROSS-VALIDATE FUNCTION"><code>cross-validate</code></a>.</p></li>
</ul>
<p><a id="x-28MGL-RESAMPLE-3ASPLIT-FOLD-2FCONT-20FUNCTION-29"></a>
<a id="MGL-RESAMPLE:SPLIT-FOLD%2FCONT%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L254">[function]</a></span> <span class="reference-object"><a href="#MGL-RESAMPLE:SPLIT-FOLD%2FCONT%20FUNCTION" >split-fold/cont</a></span></span> <span class="locative-args">seq fold n-folds</span></span></p>
<p>Imagine dividing <code>seq</code> into <code>n-folds</code> subsequences of the same
size (bar rounding). Return the subsequence of index <code>fold</code> as the
first value and the all the other subsequences concatenated into one
as the second value. The order of elements remains stable. This
function is suitable as the <code>split-fn</code> argument of <a href="#MGL-RESAMPLE:CROSS-VALIDATE%20FUNCTION" title="MGL-RESAMPLE:CROSS-VALIDATE FUNCTION"><code>cross-validate</code></a>.</p></li>
</ul>
<p><a id="x-28MGL-RESAMPLE-3ASPLIT-STRATIFIED-20FUNCTION-29"></a>
<a id="MGL-RESAMPLE:SPLIT-STRATIFIED%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L284">[function]</a></span> <span class="reference-object"><a href="#MGL-RESAMPLE:SPLIT-STRATIFIED%20FUNCTION" >split-stratified</a></span></span> <span class="locative-args">seq fold n-folds &key (key #'identity) (test #'eql) weight</span></span></p>
<p>Split <code>seq</code> into <code>n-folds</code> partitions (as in <a href="#MGL-RESAMPLE:FRACTURE-STRATIFIED%20FUNCTION" title="MGL-RESAMPLE:FRACTURE-STRATIFIED FUNCTION"><code>fracture-stratified</code></a>).
Return the partition of index <code>fold</code> as the first value, and the
concatenation of the rest as the second value. This function is
suitable as the <code>split-fn</code> argument of <a href="#MGL-RESAMPLE:CROSS-VALIDATE%20FUNCTION" title="MGL-RESAMPLE:CROSS-VALIDATE FUNCTION"><code>cross-validate</code></a> (mostly likely
as a closure with <code>key</code>, <code>test</code>, <code>weight</code> bound).</p></li>
</ul>
<p><a id="x-28MGL-RESAMPLE-3A-40MGL-RESAMPLE-BAGGING-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL-RESAMPLE:@MGL-RESAMPLE-BAGGING%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE-CROSS-VALIDATION%20MGL-PAX:SECTION" title="Cross-validation">←</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE%20MGL-PAX:SECTION" title="Resampling">↑</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE-CV-BAGGING%20MGL-PAX:SECTION" title="CV Bagging">→</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE-BAGGING%20MGL-PAX:SECTION" title="Bagging">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L299">λ</a></span></span></p>
<h3><a href="#MGL-RESAMPLE:@MGL-RESAMPLE-BAGGING%20MGL-PAX:SECTION">4.3 Bagging</a></h3>
<p><a id="x-28MGL-RESAMPLE-3ABAG-20FUNCTION-29"></a>
<a id="MGL-RESAMPLE:BAG%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L304">[function]</a></span> <span class="reference-object"><a href="#MGL-RESAMPLE:BAG%20FUNCTION" >bag</a></span></span> <span class="locative-args">seq fn &key (ratio 1) n weight (replacement t) key (test #'eql) (random-state *random-state*)</span></span></p>
<p>Sample from <code>seq</code> with <a href="#MGL-RESAMPLE:SAMPLE-FROM%20FUNCTION" title="MGL-RESAMPLE:SAMPLE-FROM FUNCTION"><code>sample-from</code></a> (passing <code>ratio</code>, <code>weight</code>,
<code>replacement</code>), or <a href="#MGL-RESAMPLE:SAMPLE-STRATIFIED%20FUNCTION" title="MGL-RESAMPLE:SAMPLE-STRATIFIED FUNCTION"><code>sample-stratified</code></a> if <code>key</code> is not <code>nil</code>. Call <code>fn</code> with
the sample. If <code>n</code> is <code>nil</code> then keep repeating this until <code>fn</code> performs a
non-local exit. Else <code>n</code> must be a non-negative integer, <code>n</code> iterations
will be performed, the primary values returned by <code>fn</code> collected into
a list and returned. See <code>sample-from</code> and <code>sample-stratified</code> for
examples.</p></li>
</ul>
<p><a id="x-28MGL-RESAMPLE-3ASAMPLE-FROM-20FUNCTION-29"></a>
<a id="MGL-RESAMPLE:SAMPLE-FROM%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L326">[function]</a></span> <span class="reference-object"><a href="#MGL-RESAMPLE:SAMPLE-FROM%20FUNCTION" >sample-from</a></span></span> <span class="locative-args">ratio seq &key weight replacement (random-state *random-state*)</span></span></p>
<p>Return a sequence constructed by sampling with or without
<code>replacement</code> from <code>seq</code>. The sum of weights in the result sequence will
approximately be the sum of weights of <code>seq</code> times <code>ratio</code>. If <code>weight</code> is
<code>nil</code> then elements are assumed to have equal weights, else <code>weight</code>
should return a non-negative real number when called with an element
of <code>seq</code>.</p>
<p>To randomly select half of the elements:</p>
<pre><code><span class="code"><span class="paren1">(<span class="code">sample-from 1/2 '<span class="paren2">(<span class="code">0 1 2 3 4 5</span>)</span></span>)</span>
=> <span class="paren1">(<span class="code">5 3 2</span>)</span></span></code></pre>
<p>To randomly select some elements such that the sum of their weights
constitute about half of the sum of weights across the whole
sequence:</p>
<pre><code><span class="code"><span class="paren1">(<span class="code">sample-from 1/2 '<span class="paren2">(<span class="code">0 1 2 3 4 5 6 7 8 9</span>)</span> <span class="keyword">:weight</span> #'identity</span>)</span>
=> <span class="comment">;; sums to 28 that's near 45/2
</span> <span class="paren1">(<span class="code">9 4 1 6 8</span>)</span></span></code></pre>
<p>To sample with replacement (that is, allowing the element to be
sampled multiple times):</p>
<pre><code><span class="code"><span class="paren1">(<span class="code">sample-from 1 '<span class="paren2">(<span class="code">0 1 2 3 4 5</span>)</span> <span class="keyword">:replacement</span> t</span>)</span>
=> <span class="paren1">(<span class="code">1 1 5 1 4 4</span>)</span></span></code></pre></li>
</ul>
<p><a id="x-28MGL-RESAMPLE-3ASAMPLE-STRATIFIED-20FUNCTION-29"></a>
<a id="MGL-RESAMPLE:SAMPLE-STRATIFIED%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L410">[function]</a></span> <span class="reference-object"><a href="#MGL-RESAMPLE:SAMPLE-STRATIFIED%20FUNCTION" >sample-stratified</a></span></span> <span class="locative-args">ratio seq &key weight replacement (key #'identity) (test #'eql) (random-state *random-state*)</span></span></p>
<p>Like <a href="#MGL-RESAMPLE:SAMPLE-FROM%20FUNCTION" title="MGL-RESAMPLE:SAMPLE-FROM FUNCTION"><code>sample-from</code></a> but makes sure that the weighted proportion of
classes in the result is approximately the same as the proportion in
<code>seq</code>. See <a href="#MGL-RESAMPLE:STRATIFY%20FUNCTION" title="MGL-RESAMPLE:STRATIFY FUNCTION"><code>stratify</code></a> for the description of <code>key</code> and <code>test</code>.</p></li>
</ul>
<p><a id="x-28MGL-RESAMPLE-3A-40MGL-RESAMPLE-CV-BAGGING-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL-RESAMPLE:@MGL-RESAMPLE-CV-BAGGING%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE-BAGGING%20MGL-PAX:SECTION" title="Bagging">←</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE%20MGL-PAX:SECTION" title="Resampling">↑</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE-MISC%20MGL-PAX:SECTION" title="Miscellaneous Operations">→</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE-CV-BAGGING%20MGL-PAX:SECTION" title="CV Bagging">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L428">λ</a></span></span></p>
<h3><a href="#MGL-RESAMPLE:@MGL-RESAMPLE-CV-BAGGING%20MGL-PAX:SECTION">4.4 CV Bagging</a></h3>
<p><a id="x-28MGL-RESAMPLE-3ABAG-CV-20FUNCTION-29"></a>
<a id="MGL-RESAMPLE:BAG-CV%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L431">[function]</a></span> <span class="reference-object"><a href="#MGL-RESAMPLE:BAG-CV%20FUNCTION" >bag-cv</a></span></span> <span class="locative-args">data fn &key n (n-folds 5) (folds (alexandria:iota n-folds)) (split-fn #'split-fold/mod) pass-fold (random-state *random-state*)</span></span></p>
<p>Perform cross-validation on different shuffles of <code>data</code> <code>n</code> times and
collect the results. Since <a href="#MGL-RESAMPLE:CROSS-VALIDATE%20FUNCTION" title="MGL-RESAMPLE:CROSS-VALIDATE FUNCTION"><code>cross-validate</code></a> collects the return values
of <code>fn</code>, the return value of this function is a list of lists of <code>fn</code>
results. If <code>n</code> is <code>nil</code>, don't collect anything just keep doing
repeated CVs until <code>fn</code> performs a non-local exit.</p>
<p>The following example simply collects the test and training sets for
2-fold CV repeated 3 times with shuffled data:</p>
<pre><code><span class="code"><span class="comment">;;; This is non-deterministic.
</span><span class="paren1">(<span class="code">bag-cv '<span class="paren2">(<span class="code">0 1 2 3 4</span>)</span> #'list <span class="keyword">:n</span> 3 <span class="keyword">:n-folds</span> 2</span>)</span>
=> <span class="paren1">(<span class="code"><span class="paren2">(<span class="code"><span class="paren3">(<span class="code"><span class="paren4">(<span class="code">2 3 4</span>)</span> <span class="paren4">(<span class="code">1 0</span>)</span></span>)</span>
<span class="paren3">(<span class="code"><span class="paren4">(<span class="code">1 0</span>)</span> <span class="paren4">(<span class="code">2 3 4</span>)</span></span>)</span></span>)</span>
<span class="paren2">(<span class="code"><span class="paren3">(<span class="code"><span class="paren4">(<span class="code">2 1 0</span>)</span> <span class="paren4">(<span class="code">4 3</span>)</span></span>)</span>
<span class="paren3">(<span class="code"><span class="paren4">(<span class="code">4 3</span>)</span> <span class="paren4">(<span class="code">2 1 0</span>)</span></span>)</span></span>)</span>
<span class="paren2">(<span class="code"><span class="paren3">(<span class="code"><span class="paren4">(<span class="code">1 0 3</span>)</span> <span class="paren4">(<span class="code">2 4</span>)</span></span>)</span>
<span class="paren3">(<span class="code"><span class="paren4">(<span class="code">2 4</span>)</span> <span class="paren4">(<span class="code">1 0 3</span>)</span></span>)</span></span>)</span></span>)</span></span></code></pre>
<p>CV bagging is useful when a single CV is not producing stable
results. As an ensemble method, CV bagging has the advantage over
bagging that each example will occur the same number of times and
after the first CV is complete there is a complete but less reliable
estimate for each example which gets refined by further CVs.</p></li>
</ul>
<p><a id="x-28MGL-RESAMPLE-3A-40MGL-RESAMPLE-MISC-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL-RESAMPLE:@MGL-RESAMPLE-MISC%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE-CV-BAGGING%20MGL-PAX:SECTION" title="CV Bagging">←</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE%20MGL-PAX:SECTION" title="Resampling">↑</a> <a href="#MGL-CORE:@MGL-CORE%20MGL-PAX:SECTION" title="Core">→</a> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE-MISC%20MGL-PAX:SECTION" title="Miscellaneous Operations">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L468">λ</a></span></span></p>
<h3><a href="#MGL-RESAMPLE:@MGL-RESAMPLE-MISC%20MGL-PAX:SECTION">4.5 Miscellaneous Operations</a></h3>
<p><a id="x-28MGL-RESAMPLE-3ASPREAD-STRATA-20FUNCTION-29"></a>
<a id="MGL-RESAMPLE:SPREAD-STRATA%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L472">[function]</a></span> <span class="reference-object"><a href="#MGL-RESAMPLE:SPREAD-STRATA%20FUNCTION" >spread-strata</a></span></span> <span class="locative-args">seq &key (key #'identity) (test #'eql)</span></span></p>
<p>Return a sequence that's a reordering of <code>seq</code> such that elements
belonging to different strata (under <code>key</code> and <code>test</code>, see <a href="#MGL-RESAMPLE:STRATIFY%20FUNCTION" title="MGL-RESAMPLE:STRATIFY FUNCTION"><code>stratify</code></a>) are
distributed evenly. The order of elements belonging to the same
stratum is unchanged.</p>
<p>For example, to make sure that even and odd numbers are distributed
evenly:</p>
<pre><code><span class="code"><span class="paren1">(<span class="code">spread-strata '<span class="paren2">(<span class="code">0 2 4 6 8 1 3 5 7 9</span>)</span> <span class="keyword">:key</span> #'evenp</span>)</span>
=> <span class="paren1">(<span class="code">0 1 2 3 4 5 6 7 8 9</span>)</span></span></code></pre>
<p>Same thing with unbalanced classes:</p>
<pre><code><span class="code"><span class="paren1">(<span class="code">spread-strata <span class="paren2">(<span class="code">vector 0 2 3 5 6 1 4</span>)</span>
<span class="keyword">:key</span> <span class="paren2">(<span class="code"><i><span class="symbol">lambda</span></i> <span class="paren3">(<span class="code">x</span>)</span>
<span class="paren3">(<span class="code"><i><span class="symbol">if</span></i> <span class="paren4">(<span class="code">member x '<span class="paren5">(<span class="code">1 4</span>)</span></span>)</span>
t
nil</span>)</span></span>)</span></span>)</span>
=> #<span class="paren1">(<span class="code">0 1 2 3 4 5 6</span>)</span></span></code></pre></li>
</ul>
<p><a id="x-28MGL-RESAMPLE-3AZIP-EVENLY-20FUNCTION-29"></a>
<a id="MGL-RESAMPLE:ZIP-EVENLY%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/resample.lisp#L499">[function]</a></span> <span class="reference-object"><a href="#MGL-RESAMPLE:ZIP-EVENLY%20FUNCTION" >zip-evenly</a></span></span> <span class="locative-args">seqs &key result-type</span></span></p>
<p>Make a single sequence out of the sequences in <code>seqs</code> so that in the
returned sequence indices of elements belonging to the same source
sequence are spread evenly across the whole range. The result is a
list is <code>result-type</code> is <code>list</code>(<a href="http://www.lispworks.com/documentation/HyperSpec/Body/t_list.htm" title="LIST (MGL-PAX:CLHS CLASS)"><code>0</code></a> <a href="http://www.lispworks.com/documentation/HyperSpec/Body/f_list_.htm" title="LIST (MGL-PAX:CLHS FUNCTION)"><code>1</code></a>), it's a vector if <code>result-type</code> is <code>vector</code>(<a href="http://www.lispworks.com/documentation/HyperSpec/Body/t_vector.htm" title="VECTOR (MGL-PAX:CLHS CLASS)"><code>0</code></a> <a href="http://www.lispworks.com/documentation/HyperSpec/Body/f_vector.htm" title="VECTOR (MGL-PAX:CLHS FUNCTION)"><code>1</code></a>).
If <code>result-type</code> is <code>nil</code>, then it's determined by the type of the first
sequence in <code>seqs</code>.</p>
<pre><code><span class="code"><span class="paren1">(<span class="code">zip-evenly '<span class="paren2">(<span class="code"><span class="paren3">(<span class="code">0 2 4</span>)</span> <span class="paren3">(<span class="code">1 3</span>)</span></span>)</span></span>)</span>
=> <span class="paren1">(<span class="code">0 1 2 3 4</span>)</span></span></code></pre></li>
</ul>
<p><a id="x-28MGL-CORE-3A-40MGL-CORE-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL-CORE:@MGL-CORE%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL-RESAMPLE:@MGL-RESAMPLE-MISC%20MGL-PAX:SECTION" title="Miscellaneous Operations">←</a> <a href="mgl-manual.html" title="MGL Manual">↑</a> <a href="#MGL-CORE:@MGL-PERSISTENCE%20MGL-PAX:SECTION" title="Persistence">→</a> <a href="#MGL-CORE:@MGL-CORE%20MGL-PAX:SECTION" title="Core">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L3">λ</a></span></span></p>
<h2><a href="#MGL-CORE:@MGL-CORE%20MGL-PAX:SECTION">5 Core</a></h2>
<h6>[in package MGL-CORE]</h6>
<p><a id="x-28MGL-CORE-3A-40MGL-PERSISTENCE-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL-CORE:@MGL-PERSISTENCE%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL-CORE:@MGL-CORE%20MGL-PAX:SECTION" title="Core">←</a> <a href="#MGL-CORE:@MGL-CORE%20MGL-PAX:SECTION" title="Core">↑</a> <a href="#MGL-CORE:@MGL-MODEL-STRIPE%20MGL-PAX:SECTION" title="Batch Processing">→</a> <a href="#MGL-CORE:@MGL-PERSISTENCE%20MGL-PAX:SECTION" title="Persistence">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L9">λ</a></span></span></p>
<h3><a href="#MGL-CORE:@MGL-PERSISTENCE%20MGL-PAX:SECTION">5.1 Persistence</a></h3>
<p><a id="x-28MGL-CORE-3ALOAD-STATE-20FUNCTION-29"></a>
<a id="MGL-CORE:LOAD-STATE%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L17">[function]</a></span> <span class="reference-object"><a href="#MGL-CORE:LOAD-STATE%20FUNCTION" >load-state</a></span></span> <span class="locative-args">filename object</span></span></p>
<p>Load weights of <code>object</code> from <code>filename</code>. Return <code>object</code>.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3ASAVE-STATE-20FUNCTION-29"></a>
<a id="MGL-CORE:SAVE-STATE%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L27">[function]</a></span> <span class="reference-object"><a href="#MGL-CORE:SAVE-STATE%20FUNCTION" >save-state</a></span></span> <span class="locative-args">filename object &key (if-exists :error) (ensure t)</span></span></p>
<p>Save weights of <code>object</code> to <code>filename</code>. If <code>ensure</code>, then
<a href="http://www.lispworks.com/documentation/HyperSpec/Body/f_ensu_1.htm" title="ENSURE-DIRECTORIES-EXIST (MGL-PAX:CLHS FUNCTION)"><code>ensure-directories-exist</code></a> is called on <code>filename</code>. <code>if-exists</code> is passed
on to <a href="http://www.lispworks.com/documentation/HyperSpec/Body/f_open.htm" title="OPEN (MGL-PAX:CLHS FUNCTION)"><code>open</code></a>. Return <code>object</code>.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3AREAD-STATE-20FUNCTION-29"></a>
<a id="MGL-CORE:READ-STATE%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L40">[function]</a></span> <span class="reference-object"><a href="#MGL-CORE:READ-STATE%20FUNCTION" >read-state</a></span></span> <span class="locative-args">object stream</span></span></p>
<p>Read the weights of <code>object</code> from the bivalent <code>stream</code> where weights
mean the learnt parameters. There is currently no sanity checking of
data which will most certainly change in the future together with
the serialization format. Return <code>object</code>.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3AWRITE-STATE-20FUNCTION-29"></a>
<a id="MGL-CORE:WRITE-STATE%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L48">[function]</a></span> <span class="reference-object"><a href="#MGL-CORE:WRITE-STATE%20FUNCTION" >write-state</a></span></span> <span class="locative-args">object stream</span></span></p>
<p>Write weight of <code>object</code> to the bivalent <code>stream</code>. Return <code>object</code>.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3AREAD-STATE-2A-20GENERIC-FUNCTION-29"></a>
<a id="MGL-CORE:READ-STATE*%20GENERIC-FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L53">[generic-function]</a></span> <span class="reference-object"><a href="#MGL-CORE:READ-STATE*%20GENERIC-FUNCTION" >read-state*</a></span></span> <span class="locative-args">object stream context</span></span></p>
<p>This is the extension point for <a href="#MGL-CORE:READ-STATE%20FUNCTION" title="MGL-CORE:READ-STATE FUNCTION"><code>read-state</code></a>. It is
guaranteed that primary <code>read-state*</code> methods will be called only once
for each <code>object</code> (under <a href="http://www.lispworks.com/documentation/HyperSpec/Body/f_eq.htm" title="EQ (MGL-PAX:CLHS FUNCTION)"><code>eq</code></a>). <code>context</code> is an opaque object and must be
passed on to any recursive <code>read-state*</code> calls.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3AWRITE-STATE-2A-20GENERIC-FUNCTION-29"></a>
<a id="MGL-CORE:WRITE-STATE*%20GENERIC-FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L63">[generic-function]</a></span> <span class="reference-object"><a href="#MGL-CORE:WRITE-STATE*%20GENERIC-FUNCTION" >write-state*</a></span></span> <span class="locative-args">object stream context</span></span></p>
<p>This is the extension point for <a href="#MGL-CORE:WRITE-STATE%20FUNCTION" title="MGL-CORE:WRITE-STATE FUNCTION"><code>write-state</code></a>. It is
guaranteed that primary <code>write-state*</code> methods will be called only
once for each <code>object</code> (under <a href="http://www.lispworks.com/documentation/HyperSpec/Body/f_eq.htm" title="EQ (MGL-PAX:CLHS FUNCTION)"><code>eq</code></a>). <code>context</code> is an opaque object and must
be passed on to any recursive <code>write-state*</code> calls.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3A-40MGL-MODEL-STRIPE-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL-CORE:@MGL-MODEL-STRIPE%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL-CORE:@MGL-PERSISTENCE%20MGL-PAX:SECTION" title="Persistence">←</a> <a href="#MGL-CORE:@MGL-CORE%20MGL-PAX:SECTION" title="Core">↑</a> <a href="#MGL-CORE:@MGL-EXECUTORS%20MGL-PAX:SECTION" title="Executors">→</a> <a href="#MGL-CORE:@MGL-MODEL-STRIPE%20MGL-PAX:SECTION" title="Batch Processing">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L74">λ</a></span></span></p>
<h3><a href="#MGL-CORE:@MGL-MODEL-STRIPE%20MGL-PAX:SECTION">5.2 Batch Processing</a></h3>
<p>Processing instances one by one during training or prediction can
be slow. The models that support batch processing for greater
efficiency are said to be <em>striped</em>.</p>
<p>Typically, during or after creating a model, one sets <a href="#MGL-CORE:MAX-N-STRIPES%20GENERIC-FUNCTION" title="MGL-CORE:MAX-N-STRIPES GENERIC-FUNCTION"><code>max-n-stripes</code></a>
on it a positive integer. When a batch of instances is to be fed to
the model it is first broken into subbatches of length that's at
most <code>max-n-stripes</code>. For each subbatch, <a href="#MGL-CORE:SET-INPUT%20GENERIC-FUNCTION" title="MGL-CORE:SET-INPUT GENERIC-FUNCTION"><code>set-input</code></a> (FIXDOC) is called
and a before method takes care of setting <a href="#MGL-CORE:N-STRIPES%20GENERIC-FUNCTION" title="MGL-CORE:N-STRIPES GENERIC-FUNCTION"><code>n-stripes</code></a> to the actual
number of instances in the subbatch. When <code>max-n-stripes</code> is set
internal data structures may be resized which is an expensive
operation. Setting <code>n-stripes</code> is a comparatively cheap operation,
often implemented as matrix reshaping.</p>
<p>Note that for models made of different parts (for example,
<a href="#MGL-BP:BPN%20CLASS" title="MGL-BP:BPN CLASS"><code>mgl-bp:bpn</code></a> consists of <a href="#MGL-BP:LUMP%20CLASS" title="MGL-BP:LUMP CLASS"><code>mgl-bp:lump</code></a>s) , setting these
values affects the constituent parts, but one should never change
the number stripes of the parts directly because that would lead to
an internal inconsistency in the model.</p>
<p><a id="x-28MGL-CORE-3AMAX-N-STRIPES-20GENERIC-FUNCTION-29"></a>
<a id="MGL-CORE:MAX-N-STRIPES%20GENERIC-FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L105">[generic-function]</a></span> <span class="reference-object"><a href="#MGL-CORE:MAX-N-STRIPES%20GENERIC-FUNCTION" >max-n-stripes</a></span></span> <span class="locative-args">object</span></span></p>
<p>The number of stripes with which the <code>object</code> is
capable of dealing simultaneously. </p></li>
</ul>
<p><a id="x-28MGL-CORE-3ASET-MAX-N-STRIPES-20GENERIC-FUNCTION-29"></a>
<a id="MGL-CORE:SET-MAX-N-STRIPES%20GENERIC-FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L109">[generic-function]</a></span> <span class="reference-object"><a href="#MGL-CORE:SET-MAX-N-STRIPES%20GENERIC-FUNCTION" >set-max-n-stripes</a></span></span> <span class="locative-args">max-n-stripes object</span></span></p>
<p>Allocate the necessary stuff to allow for
<code>max-n-stripes</code> number of stripes to be worked with simultaneously in
<code>object</code>. This is called when <code>max-n-stripes</code> is <a href="http://www.lispworks.com/documentation/HyperSpec/Body/m_setf_.htm" title="SETF (MGL-PAX:CLHS MGL-PAX:MACRO)"><code>setf</code></a>'ed.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3AN-STRIPES-20GENERIC-FUNCTION-29"></a>
<a id="MGL-CORE:N-STRIPES%20GENERIC-FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L117">[generic-function]</a></span> <span class="reference-object"><a href="#MGL-CORE:N-STRIPES%20GENERIC-FUNCTION" >n-stripes</a></span></span> <span class="locative-args">object</span></span></p>
<p>The number of stripes currently present in <code>object</code>.
This is at most <a href="#MGL-CORE:MAX-N-STRIPES%20GENERIC-FUNCTION" title="MGL-CORE:MAX-N-STRIPES GENERIC-FUNCTION"><code>max-n-stripes</code></a>.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3ASET-N-STRIPES-20GENERIC-FUNCTION-29"></a>
<a id="MGL-CORE:SET-N-STRIPES%20GENERIC-FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L121">[generic-function]</a></span> <span class="reference-object"><a href="#MGL-CORE:SET-N-STRIPES%20GENERIC-FUNCTION" >set-n-stripes</a></span></span> <span class="locative-args">n-stripes object</span></span></p>
<p>Set the number of stripes (out of <a href="#MGL-CORE:MAX-N-STRIPES%20GENERIC-FUNCTION" title="MGL-CORE:MAX-N-STRIPES GENERIC-FUNCTION"><code>max-n-stripes</code></a>)
that are in use in <code>object</code>. This is called when <code>n-stripes</code> is
<a href="http://www.lispworks.com/documentation/HyperSpec/Body/m_setf_.htm" title="SETF (MGL-PAX:CLHS MGL-PAX:MACRO)"><code>setf</code></a>'ed.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3AWITH-STRIPES-20MGL-PAX-3AMACRO-29"></a>
<a id="MGL-CORE:WITH-STRIPES%20MGL-PAX:MACRO"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L129">[macro]</a></span> <span class="reference-object"><a href="#MGL-CORE:WITH-STRIPES%20MGL-PAX:MACRO" >with-stripes</a></span></span> <span class="locative-args">specs &body body</span></span></p>
<p>Bind start and optionally end indices belonging to stripes in
striped objects.</p>
<pre><code>(WITH-STRIPES ((STRIPE1 OBJECT1 START1 END1)
(STRIPE2 OBJECT2 START2)
...)
...)
</code></pre>
<p>This is how one's supposed to find the index range corresponding to
the Nth input in an input lump of a bpn:</p>
<pre><code> (with-stripes ((n input-lump start end))
(loop for i upfrom start below end
do (setf (mref (nodes input-lump) i) 0d0)))
</code></pre>
<p>Note how the input lump is striped, but the matrix into which we are
indexing (<a href="#MGL-COMMON:NODES%20GENERIC-FUNCTION" title="MGL-COMMON:NODES GENERIC-FUNCTION"><code>nodes</code></a>) is not known to <code>with-stripes</code>. In fact, for lumps
the same stripe indices work with <code>nodes</code> and <a href="#MGL-BP:DERIVATIVES%20GENERIC-FUNCTION" title="MGL-BP:DERIVATIVES GENERIC-FUNCTION"><code>mgl-bp:derivatives</code></a>.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3ASTRIPE-START-20GENERIC-FUNCTION-29"></a>
<a id="MGL-CORE:STRIPE-START%20GENERIC-FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L160">[generic-function]</a></span> <span class="reference-object"><a href="#MGL-CORE:STRIPE-START%20GENERIC-FUNCTION" >stripe-start</a></span></span> <span class="locative-args">stripe object</span></span></p>
<p>Return the start index of <code>stripe</code> in some array or
matrix of <code>object</code>.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3ASTRIPE-END-20GENERIC-FUNCTION-29"></a>
<a id="MGL-CORE:STRIPE-END%20GENERIC-FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L164">[generic-function]</a></span> <span class="reference-object"><a href="#MGL-CORE:STRIPE-END%20GENERIC-FUNCTION" >stripe-end</a></span></span> <span class="locative-args">stripe object</span></span></p>
<p>Return the end index (exclusive) of <code>stripe</code> in some
array or matrix of <code>object</code>.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3ASET-INPUT-20GENERIC-FUNCTION-29"></a>
<a id="MGL-CORE:SET-INPUT%20GENERIC-FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L168">[generic-function]</a></span> <span class="reference-object"><a href="#MGL-CORE:SET-INPUT%20GENERIC-FUNCTION" >set-input</a></span></span> <span class="locative-args">instances model</span></span></p>
<p>Set <code>instances</code> as inputs in <code>model</code>. <code>instances</code> is
always a <a href="http://www.lispworks.com/documentation/HyperSpec/Body/t_seq.htm" title="SEQUENCE (MGL-PAX:CLHS CLASS)"><code>sequence</code></a> of instances even for models not capable of batch
operation. It sets <a href="#MGL-CORE:N-STRIPES%20GENERIC-FUNCTION" title="MGL-CORE:N-STRIPES GENERIC-FUNCTION"><code>n-stripes</code></a> to (<a href="http://www.lispworks.com/documentation/HyperSpec/Body/f_length.htm" title="LENGTH (MGL-PAX:CLHS FUNCTION)"><code>length</code></a> <code>instances</code>) in a <code>:before</code>
method.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3AMAP-BATCHES-FOR-MODEL-20FUNCTION-29"></a>
<a id="MGL-CORE:MAP-BATCHES-FOR-MODEL%20FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L174">[function]</a></span> <span class="reference-object"><a href="#MGL-CORE:MAP-BATCHES-FOR-MODEL%20FUNCTION" >map-batches-for-model</a></span></span> <span class="locative-args">fn dataset model</span></span></p>
<p>Call <code>fn</code> with batches of instances from <code>dataset</code> suitable for <code>model</code>.
The number of instances in a batch is <a href="#MGL-CORE:MAX-N-STRIPES%20GENERIC-FUNCTION" title="MGL-CORE:MAX-N-STRIPES GENERIC-FUNCTION"><code>max-n-stripes</code></a> of <code>model</code> or less
if there are no more instances left.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3ADO-BATCHES-FOR-MODEL-20MGL-PAX-3AMACRO-29"></a>
<a id="MGL-CORE:DO-BATCHES-FOR-MODEL%20MGL-PAX:MACRO"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L185">[macro]</a></span> <span class="reference-object"><a href="#MGL-CORE:DO-BATCHES-FOR-MODEL%20MGL-PAX:MACRO" >do-batches-for-model</a></span></span> <span class="locative-args">(batch (dataset model)) &body body</span></span></p>
<p>Convenience macro over <a href="#MGL-CORE:MAP-BATCHES-FOR-MODEL%20FUNCTION" title="MGL-CORE:MAP-BATCHES-FOR-MODEL FUNCTION"><code>map-batches-for-model</code></a>.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3A-40MGL-EXECUTORS-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL-CORE:@MGL-EXECUTORS%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL-CORE:@MGL-MODEL-STRIPE%20MGL-PAX:SECTION" title="Batch Processing">←</a> <a href="#MGL-CORE:@MGL-CORE%20MGL-PAX:SECTION" title="Core">↑</a> <a href="#MGL-CORE:@MGL-PARAMETERIZED-EXECUTOR-CACHE%20MGL-PAX:SECTION" title="Parameterized Executor Cache">→</a> <a href="#MGL-CORE:@MGL-EXECUTORS%20MGL-PAX:SECTION" title="Executors">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L190">λ</a></span></span></p>
<h3><a href="#MGL-CORE:@MGL-EXECUTORS%20MGL-PAX:SECTION">5.3 Executors</a></h3>
<p><a id="x-28MGL-CORE-3AMAP-OVER-EXECUTORS-20GENERIC-FUNCTION-29"></a>
<a id="MGL-CORE:MAP-OVER-EXECUTORS%20GENERIC-FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L195">[generic-function]</a></span> <span class="reference-object"><a href="#MGL-CORE:MAP-OVER-EXECUTORS%20GENERIC-FUNCTION" >map-over-executors</a></span></span> <span class="locative-args">fn instances prototype-executor</span></span></p>
<p>Divide <code>instances</code> between executors that perform the
same function as <code>prototype-executor</code> and call <code>fn</code> with the instances
and the executor for which the instances are.</p>
<p>Some objects conflate function and call: the forward pass of a
<a href="#MGL-BP:BPN%20CLASS" title="MGL-BP:BPN CLASS"><code>mgl-bp:bpn</code></a> computes output from inputs so it is like a
function but it also doubles as a function call in the sense that
the bpn (function) object changes state during the computation of
the output. Hence not even the forward pass of a bpn is thread safe.
There is also the restriction that all inputs must be of the same
size.</p>
<p>For example, if we have a function that builds bpn a for an input of
a certain size, then we can create a factory that creates bpns for a
particular call. The factory probably wants to keep the weights the
same though. In <a href="#MGL-CORE:@MGL-PARAMETERIZED-EXECUTOR-CACHE%20MGL-PAX:SECTION" title="Parameterized Executor Cache">Parameterized Executor Cache</a>,
<a href="#MGL-CORE:MAKE-EXECUTOR-WITH-PARAMETERS%20GENERIC-FUNCTION" title="MGL-CORE:MAKE-EXECUTOR-WITH-PARAMETERS GENERIC-FUNCTION"><code>make-executor-with-parameters</code></a> is this factory.</p>
<p>Parallelization of execution is another possibility
<code>map-over-executors</code> allows, but there is no prebuilt solution for it,
yet.</p>
<p>The default implementation simply calls <code>fn</code> with <code>instances</code> and
<code>prototype-executor</code>.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3ADO-EXECUTORS-20MGL-PAX-3AMACRO-29"></a>
<a id="MGL-CORE:DO-EXECUTORS%20MGL-PAX:MACRO"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L223">[macro]</a></span> <span class="reference-object"><a href="#MGL-CORE:DO-EXECUTORS%20MGL-PAX:MACRO" >do-executors</a></span></span> <span class="locative-args">(instances object) &body body</span></span></p>
<p>Convenience macro on top of <a href="#MGL-CORE:MAP-OVER-EXECUTORS%20GENERIC-FUNCTION" title="MGL-CORE:MAP-OVER-EXECUTORS GENERIC-FUNCTION"><code>map-over-executors</code></a>.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3A-40MGL-PARAMETERIZED-EXECUTOR-CACHE-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL-CORE:@MGL-PARAMETERIZED-EXECUTOR-CACHE%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL-CORE:@MGL-EXECUTORS%20MGL-PAX:SECTION" title="Executors">←</a> <a href="#MGL-CORE:@MGL-EXECUTORS%20MGL-PAX:SECTION" title="Executors">↑</a> <a href="#MGL-CORE:@MGL-MONITORING%20MGL-PAX:SECTION" title="Monitoring">→</a> <a href="#MGL-CORE:@MGL-PARAMETERIZED-EXECUTOR-CACHE%20MGL-PAX:SECTION" title="Parameterized Executor Cache">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L230">λ</a></span></span></p>
<h4><a href="#MGL-CORE:@MGL-PARAMETERIZED-EXECUTOR-CACHE%20MGL-PAX:SECTION">5.3.1 Parameterized Executor Cache</a></h4>
<p><a id="x-28MGL-CORE-3APARAMETERIZED-EXECUTOR-CACHE-MIXIN-20CLASS-29"></a>
<a id="MGL-CORE:PARAMETERIZED-EXECUTOR-CACHE-MIXIN%20CLASS"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L236">[class]</a></span> <span class="reference-object"><a href="#MGL-CORE:PARAMETERIZED-EXECUTOR-CACHE-MIXIN%20CLASS" >parameterized-executor-cache-mixin</a></span></span></span></p>
<p>Mix this into a model, implement
<a href="#MGL-CORE:INSTANCE-TO-EXECUTOR-PARAMETERS%20GENERIC-FUNCTION" title="MGL-CORE:INSTANCE-TO-EXECUTOR-PARAMETERS GENERIC-FUNCTION"><code>instance-to-executor-parameters</code></a> and <a href="#MGL-CORE:MAKE-EXECUTOR-WITH-PARAMETERS%20GENERIC-FUNCTION" title="MGL-CORE:MAKE-EXECUTOR-WITH-PARAMETERS GENERIC-FUNCTION"><code>make-executor-with-parameters</code></a>
and <a href="#MGL-CORE:DO-EXECUTORS%20MGL-PAX:MACRO" title="MGL-CORE:DO-EXECUTORS MGL-PAX:MACRO"><code>do-executors</code></a> will be to able build executors suitable for
different instances. The canonical example is using a BPN to compute
the means and convariances of a gaussian process. Since each
instance is made of a variable number of observations, the size of
the input is not constant, thus we have a bpn (an executor) for each
input dimension (the parameters).</p></li>
</ul>
<p><a id="x-28MGL-CORE-3AMAKE-EXECUTOR-WITH-PARAMETERS-20GENERIC-FUNCTION-29"></a>
<a id="MGL-CORE:MAKE-EXECUTOR-WITH-PARAMETERS%20GENERIC-FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L249">[generic-function]</a></span> <span class="reference-object"><a href="#MGL-CORE:MAKE-EXECUTOR-WITH-PARAMETERS%20GENERIC-FUNCTION" >make-executor-with-parameters</a></span></span> <span class="locative-args">parameters cache</span></span></p>
<p>Create a new executor for <code>parameters</code>. <code>cache</code> is a
<a href="#MGL-CORE:PARAMETERIZED-EXECUTOR-CACHE-MIXIN%20CLASS" title="MGL-CORE:PARAMETERIZED-EXECUTOR-CACHE-MIXIN CLASS"><code>parameterized-executor-cache-mixin</code></a>. In the BPN gaussian process
example, <code>parameters</code> would be a list of input dimensions.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3AINSTANCE-TO-EXECUTOR-PARAMETERS-20GENERIC-FUNCTION-29"></a>
<a id="MGL-CORE:INSTANCE-TO-EXECUTOR-PARAMETERS%20GENERIC-FUNCTION"></a></p>
<ul>
<li><p><span class=reference-bullet><span class=reference><span class="locative-type"><a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/core.lisp#L254">[generic-function]</a></span> <span class="reference-object"><a href="#MGL-CORE:INSTANCE-TO-EXECUTOR-PARAMETERS%20GENERIC-FUNCTION" >instance-to-executor-parameters</a></span></span> <span class="locative-args">instance cache</span></span></p>
<p>Return the parameters for an executor able to
handle <code>instance</code>. Called by <a href="#MGL-CORE:MAP-OVER-EXECUTORS%20GENERIC-FUNCTION" title="MGL-CORE:MAP-OVER-EXECUTORS GENERIC-FUNCTION"><code>map-over-executors</code></a> on <code>cache</code> (that's a
<a href="#MGL-CORE:PARAMETERIZED-EXECUTOR-CACHE-MIXIN%20CLASS" title="MGL-CORE:PARAMETERIZED-EXECUTOR-CACHE-MIXIN CLASS"><code>parameterized-executor-cache-mixin</code></a>). The returned parameters are
keys in an <a href="http://www.lispworks.com/documentation/HyperSpec/Body/f_equal.htm" title="EQUAL (MGL-PAX:CLHS FUNCTION)"><code>equal</code></a> parameters->executor hash table.</p></li>
</ul>
<p><a id="x-28MGL-CORE-3A-40MGL-MONITORING-20MGL-PAX-3ASECTION-29"></a>
<a id="MGL-CORE:@MGL-MONITORING%20MGL-PAX:SECTION"></a></p>
<p><span class="outer-navigation"><span class="navigation"> <a href="#MGL-CORE:@MGL-PARAMETERIZED-EXECUTOR-CACHE%20MGL-PAX:SECTION" title="Parameterized Executor Cache">←</a> <a href="mgl-manual.html" title="MGL Manual">↑</a> <a href="#MGL-CORE:@MGL-MONITOR%20MGL-PAX:SECTION" title="Monitors">→</a> <a href="#MGL-CORE:@MGL-MONITORING%20MGL-PAX:SECTION" title="Monitoring">↺</a> <a href="https://github.com/melisgl/mgl/blob/46d47949278e163463834aca572649b996f28419/src/monitor.lisp#L3">λ</a></span></span></p>
<h2><a href="#MGL-CORE:@MGL-MONITORING%20MGL-PAX:SECTION">6 Monitoring</a></h2>
<h6>[in package MGL-CORE]</h6>
<p>When training or applying a model, one often wants to track various
statistics. For example, in the case of training a neural network
with cross-entropy loss, these statistics could be the average
cross-entropy loss itself, classification accuracy, or even the
entire confusion matrix and sparsity levels in hidden layers. Also,
there is the question of what to do with the measured values (log
and forget, add to some counter or a list).</p>
<p>So there may be several phases of operation when we want to keep an
eye on. Let's call these <strong>events</strong>. There can also be many fairly
independent things to do in response to an event. Let's call these
<strong>monitors</strong>. Some monitors are a composition of two operations: one
that extracts some measurements and another that aggregates those
measurements. Let's call these two <strong>measurers</strong> and <strong>counters</strong>,
respectively.</p>
<p>For example, consider training a backpropagation neural network. We
want to look at the state of of network just after the backward
pass. <a href="#MGL-BP:BP-LEARNER%20CLASS" title="MGL-BP:BP-LEARNER CLASS"><code>mgl-bp:bp-learner</code></a> has a <a href="#MGL-CORE:MONITORS%20%28MGL-PAX:ACCESSOR%20MGL-BP:BP-LEARNER%29" title="MGL-CORE:MONITORS (MGL-PAX:ACCESSOR MGL-BP:BP-LEARNER)"><code>monitors</code></a> event hook corresponding to the moment after
backpropagating the gradients. Suppose we are interested in how the
training cost evolves:</p>
<pre><code>(push (make-instance 'monitor
:measurer (lambda (instances bpn)
(declare (ignore instances))
(mgl-bp:cost bpn))
:counter (make-instance 'basic-counter))
(monitors learner))
</code></pre>
<p>During training, this monitor will track the cost of training
examples behind the scenes. If we want to print and reset this
monitor periodically we can put another monitor on
<a href="#MGL-OPT:ITERATIVE-OPTIMIZER%20CLASS" title="MGL-OPT:ITERATIVE-OPTIMIZER CLASS"><code>mgl-opt:iterative-optimizer</code></a>'s <a href="#MGL-OPT:ON-N-INSTANCES-CHANGED%20%28MGL-PAX:ACCESSOR%20MGL-OPT:ITERATIVE-OPTIMIZER%29" title="MGL-OPT:ON-N-INSTANCES-CHANGED (MGL-PAX:ACCESSOR MGL-OPT:ITERATIVE-OPTIMIZER)"><code>mgl-opt:on-n-instances-changed</code></a>
accessor:</p>
<pre><code>(push (lambda (optimizer gradient-source n-instances)
(declare (ignore optimizer))
(when (zerop (mod n-instances 1000))
(format t "n-instances: ~S~%" n-instances)
(dolist (monitor (monitors gradient-source))
(when (counter monitor)
(format t "~A~%" (counter monitor))