-
Notifications
You must be signed in to change notification settings - Fork 3
/
oschema.html
1889 lines (1646 loc) · 65.2 KB
/
oschema.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">
<head>
<!-- 2021-02-24 Wed 15:25 -->
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title><code>moo</code> 無 <code>oschema</code></title>
<meta name="generator" content="Org mode" />
<meta name="author" content="Brett Viren" />
<style type="text/css">
<!--/*--><![CDATA[/*><!--*/
.title { text-align: center;
margin-bottom: .2em; }
.subtitle { text-align: center;
font-size: medium;
font-weight: bold;
margin-top:0; }
.todo { font-family: monospace; color: red; }
.done { font-family: monospace; color: green; }
.priority { font-family: monospace; color: orange; }
.tag { background-color: #eee; font-family: monospace;
padding: 2px; font-size: 80%; font-weight: normal; }
.timestamp { color: #bebebe; }
.timestamp-kwd { color: #5f9ea0; }
.org-right { margin-left: auto; margin-right: 0px; text-align: right; }
.org-left { margin-left: 0px; margin-right: auto; text-align: left; }
.org-center { margin-left: auto; margin-right: auto; text-align: center; }
.underline { text-decoration: underline; }
#postamble p, #preamble p { font-size: 90%; margin: .2em; }
p.verse { margin-left: 3%; }
pre {
border: 1px solid #ccc;
box-shadow: 3px 3px 3px #eee;
padding: 8pt;
font-family: monospace;
overflow: auto;
margin: 1.2em;
}
pre.src {
position: relative;
overflow: auto;
padding-top: 1.2em;
}
pre.src:before {
display: none;
position: absolute;
background-color: white;
top: -10px;
right: 10px;
padding: 3px;
border: 1px solid black;
}
pre.src:hover:before { display: inline; margin-top: 14px;}
/* Languages per Org manual */
pre.src-asymptote:before { content: 'Asymptote'; }
pre.src-awk:before { content: 'Awk'; }
pre.src-C:before { content: 'C'; }
/* pre.src-C++ doesn't work in CSS */
pre.src-clojure:before { content: 'Clojure'; }
pre.src-css:before { content: 'CSS'; }
pre.src-D:before { content: 'D'; }
pre.src-ditaa:before { content: 'ditaa'; }
pre.src-dot:before { content: 'Graphviz'; }
pre.src-calc:before { content: 'Emacs Calc'; }
pre.src-emacs-lisp:before { content: 'Emacs Lisp'; }
pre.src-fortran:before { content: 'Fortran'; }
pre.src-gnuplot:before { content: 'gnuplot'; }
pre.src-haskell:before { content: 'Haskell'; }
pre.src-hledger:before { content: 'hledger'; }
pre.src-java:before { content: 'Java'; }
pre.src-js:before { content: 'Javascript'; }
pre.src-latex:before { content: 'LaTeX'; }
pre.src-ledger:before { content: 'Ledger'; }
pre.src-lisp:before { content: 'Lisp'; }
pre.src-lilypond:before { content: 'Lilypond'; }
pre.src-lua:before { content: 'Lua'; }
pre.src-matlab:before { content: 'MATLAB'; }
pre.src-mscgen:before { content: 'Mscgen'; }
pre.src-ocaml:before { content: 'Objective Caml'; }
pre.src-octave:before { content: 'Octave'; }
pre.src-org:before { content: 'Org mode'; }
pre.src-oz:before { content: 'OZ'; }
pre.src-plantuml:before { content: 'Plantuml'; }
pre.src-processing:before { content: 'Processing.js'; }
pre.src-python:before { content: 'Python'; }
pre.src-R:before { content: 'R'; }
pre.src-ruby:before { content: 'Ruby'; }
pre.src-sass:before { content: 'Sass'; }
pre.src-scheme:before { content: 'Scheme'; }
pre.src-screen:before { content: 'Gnu Screen'; }
pre.src-sed:before { content: 'Sed'; }
pre.src-sh:before { content: 'shell'; }
pre.src-sql:before { content: 'SQL'; }
pre.src-sqlite:before { content: 'SQLite'; }
/* additional languages in org.el's org-babel-load-languages alist */
pre.src-forth:before { content: 'Forth'; }
pre.src-io:before { content: 'IO'; }
pre.src-J:before { content: 'J'; }
pre.src-makefile:before { content: 'Makefile'; }
pre.src-maxima:before { content: 'Maxima'; }
pre.src-perl:before { content: 'Perl'; }
pre.src-picolisp:before { content: 'Pico Lisp'; }
pre.src-scala:before { content: 'Scala'; }
pre.src-shell:before { content: 'Shell Script'; }
pre.src-ebnf2ps:before { content: 'ebfn2ps'; }
/* additional language identifiers per "defun org-babel-execute"
in ob-*.el */
pre.src-cpp:before { content: 'C++'; }
pre.src-abc:before { content: 'ABC'; }
pre.src-coq:before { content: 'Coq'; }
pre.src-groovy:before { content: 'Groovy'; }
/* additional language identifiers from org-babel-shell-names in
ob-shell.el: ob-shell is the only babel language using a lambda to put
the execution function name together. */
pre.src-bash:before { content: 'bash'; }
pre.src-csh:before { content: 'csh'; }
pre.src-ash:before { content: 'ash'; }
pre.src-dash:before { content: 'dash'; }
pre.src-ksh:before { content: 'ksh'; }
pre.src-mksh:before { content: 'mksh'; }
pre.src-posh:before { content: 'posh'; }
/* Additional Emacs modes also supported by the LaTeX listings package */
pre.src-ada:before { content: 'Ada'; }
pre.src-asm:before { content: 'Assembler'; }
pre.src-caml:before { content: 'Caml'; }
pre.src-delphi:before { content: 'Delphi'; }
pre.src-html:before { content: 'HTML'; }
pre.src-idl:before { content: 'IDL'; }
pre.src-mercury:before { content: 'Mercury'; }
pre.src-metapost:before { content: 'MetaPost'; }
pre.src-modula-2:before { content: 'Modula-2'; }
pre.src-pascal:before { content: 'Pascal'; }
pre.src-ps:before { content: 'PostScript'; }
pre.src-prolog:before { content: 'Prolog'; }
pre.src-simula:before { content: 'Simula'; }
pre.src-tcl:before { content: 'tcl'; }
pre.src-tex:before { content: 'TeX'; }
pre.src-plain-tex:before { content: 'Plain TeX'; }
pre.src-verilog:before { content: 'Verilog'; }
pre.src-vhdl:before { content: 'VHDL'; }
pre.src-xml:before { content: 'XML'; }
pre.src-nxml:before { content: 'XML'; }
/* add a generic configuration mode; LaTeX export needs an additional
(add-to-list 'org-latex-listings-langs '(conf " ")) in .emacs */
pre.src-conf:before { content: 'Configuration File'; }
table { border-collapse:collapse; }
caption.t-above { caption-side: top; }
caption.t-bottom { caption-side: bottom; }
td, th { vertical-align:top; }
th.org-right { text-align: center; }
th.org-left { text-align: center; }
th.org-center { text-align: center; }
td.org-right { text-align: right; }
td.org-left { text-align: left; }
td.org-center { text-align: center; }
dt { font-weight: bold; }
.footpara { display: inline; }
.footdef { margin-bottom: 1em; }
.figure { padding: 1em; }
.figure p { text-align: center; }
.equation-container {
display: table;
text-align: center;
width: 100%;
}
.equation {
vertical-align: middle;
}
.equation-label {
display: table-cell;
text-align: right;
vertical-align: middle;
}
.inlinetask {
padding: 10px;
border: 2px solid gray;
margin: 10px;
background: #ffffcc;
}
#org-div-home-and-up
{ text-align: right; font-size: 70%; white-space: nowrap; }
textarea { overflow-x: auto; }
.linenr { font-size: smaller }
.code-highlighted { background-color: #ffff00; }
.org-info-js_info-navigation { border-style: none; }
#org-info-js_console-label
{ font-size: 10px; font-weight: bold; white-space: nowrap; }
.org-info-js_search-highlight
{ background-color: #ffff00; color: #000000; font-weight: bold; }
.org-svg { width: 90%; }
/*]]>*/-->
</style>
<link rel="stylesheet" type="text/css" href="other/readtheorg/css/htmlize.css"/>
<link rel="stylesheet" type="text/css" href="other/readtheorg/css/readtheorg.css"/>
<script type="text/javascript" src="other/lib/js/jquery.min.js"></script>
<script type="text/javascript" src="other/lib/js/bootstrap.min.js"></script>
<script type="text/javascript" src="other/lib/js/jquery.stickytableheaders.min.js"></script>
<script type="text/javascript" src="other/readtheorg/js/readtheorg.js"></script>
<style> #content{max-width:1800px;}</style>
<style> p{max-width:800px;}</style>
<style> li{max-width:800px;}</style>
<style> pre.src{border-radius: 5px; background-color:#333; color:#0f0;}</style>
<style> pre.example{border-radius: 5px; background-color:#333; color:#0f0;}</style>
<style> code{border-radius: 5px; background-color:#333; color:#0f0;}</style>
<script type="text/javascript">
// @license magnet:?xt=urn:btih:e95b018ef3580986a04669f1b5879592219e2a7a&dn=public-domain.txt Public Domain
<!--/*--><![CDATA[/*><!--*/
function CodeHighlightOn(elem, id)
{
var target = document.getElementById(id);
if(null != target) {
elem.classList.add("code-highlighted");
target.classList.add("code-highlighted");
}
}
function CodeHighlightOff(elem, id)
{
var target = document.getElementById(id);
if(null != target) {
elem.classList.remove("code-highlighted");
target.classList.remove("code-highlighted");
}
}
/*]]>*///-->
// @license-end
</script>
</head>
<body>
<div id="content">
<h1 class="title"><code>moo</code> 無 <code>oschema</code>
<br />
<span class="subtitle">Describing schema as objects</span>
</h1>
<div id="table-of-contents">
<h2>Table of Contents</h2>
<div id="text-table-of-contents">
<ul>
<li><a href="#concepts">Concepts</a>
<ul>
<li><a href="#classes">Schema classes</a></li>
<li><a href="#atomic">Atomic types</a></li>
<li><a href="#aggregate">Aggregate types</a></li>
<li><a href="#org28b3dd8">Type containers</a></li>
</ul>
</li>
<li><a href="#Schema">Schema Examples</a>
<ul>
<li><a href="#schema-in-jsonnet">Jsonnet</a></li>
<li><a href="#schema-in-python">Python</a></li>
<li><a href="#json-schema">JSON Schema</a></li>
</ul>
</li>
<li><a href="#transforms">Models</a>
<ul>
<li><a href="#diy-trans">DIY</a></li>
<li><a href="#jsonnet-tla-trans">Jsonnet</a></li>
<li><a href="#python-trans">Python</a></li>
</ul>
</li>
<li><a href="#codegen">Codegen</a>
<ul>
<li><a href="#model">Model</a></li>
<li><a href="#struct">Template</a></li>
</ul>
</li>
<li><a href="#objects">Objects</a></li>
</ul>
</div>
</div>
<div id="outline-container-concepts" class="outline-2">
<h2 id="concepts">Concepts</h2>
<div class="outline-text-2" id="text-concepts">
<p>
A <b>schema</b> defines information about the <b>type</b> of some <b>value</b>. It may
describe an atomic type ("integer", "boolean") or types which are
aggregates of other types ("record", "sequence"). The language forms
we choose to use to provide this information defines a <b>schema system</b>
(essentially itself a "meta schema"). <b>moo</b> supports a multiple schema
systems but the predominant one is called <i>oschema</i>.
</p>
<div class="note" id="org8b25ebb">
<p>
The "o" stands for "object" as <i>oschema</i> describes a type using an
object representation. A lesser used <b>moo</b> schema system is <i>fschema</i>
which as you may guess uses a functional representation.
</p>
</div>
<p>
Many schema systems will specify a persistent representation (file
format) for expressing a schema. Eg, XML has XSD and JSON has JSON
Schema (itself expressed as JSON). <b>moo</b> <i>oschema</i> system is defined
in terms of a transient representation (structured data in memory).
Thus, a variety of persistent representations may be used to
provide <i>oschema</i> schema. Users may select file formats they know and
love if they wish. Below we will describe two (Jsonnet and Python) for which <b>moo</b> provides direct support.
</p>
</div>
<div id="outline-container-classes" class="outline-3">
<h3 id="classes">Schema classes</h3>
<div class="outline-text-3" id="text-classes">
<p>
<b>moo</b> <i>oschema</i> is defined conceptually starting with a fixed set of
<b>schema classes</b> that are listed below. A <b>moo</b> <i>oschema</i> <b>type</b>
(called in some parts of <b>moo</b> an <i>otype</i>) is considered an instance
of exactly one <b>schema class</b>. Dropping down one rung of the semantic
ladder, a <b>model</b> (aka "value") is an instance of a <b>type</b>.
</p>
<p>
The <b>moo</b> <i>oschema</i> schema classes are summarized:
</p>
<dl class="org-dl">
<dt><code>boolean</code></dt><dd>a type which may take value "true" or "false"</dd>
<dt><code>number</code></dt><dd>a numeric type of given format and size and optional numeric constraints</dd>
<dt><code>string</code></dt><dd>a character string type possibly matching some pattern or format</dd>
<dt><code>sequence</code></dt><dd>an array or vector with elements of one type</dd>
<dt><code>tuple</code></dt><dd>(not yet supported, but you can guess what it will be)</dd>
<dt><code>record</code></dt><dd>a collection of <code>fields</code> (named type references) directly held or indirectly held by zero or more records named as <code>bases</code>.</dd>
<dt><code>enum</code></dt><dd>a type that may take one value from a predefined, limited set of values</dd>
<dt><code>any</code></dt><dd>a type that may take a value of any type (eg such as <code>void*</code>, <code>boost::any</code>, <code>nlohmann::json</code>)</dd>
<dt><code>anyOf</code></dt><dd>a type that may take a value of any type in a predefined, limited set of types.</dd>
<dt><code>namespace</code></dt><dd>a collection of named types, (distinct from <code>record</code> to match eg C++/Python semantics)</dd>
</dl>
<p>
Every <i>oschema</i> <b>type</b> then provides a set of <b>attributes</b> as
determined by its schema class. Some attributes are common across all
schema classes and may be required or optional:
</p>
<dl class="org-dl">
<dt><code>name</code></dt><dd>(required) type name unique to the type context (see <code>path</code>)</dd>
<dt><code>schema</code></dt><dd>(required) string identifying the schema class taken from above list</dd>
<dt><code>doc</code></dt><dd>(optional, default empty) document string briefly describing the type</dd>
<dt><code>path</code></dt><dd>(required, potentially empty), ordered array of names
representing the absolute context of the type (eg as a C++
<code>namespace</code> or Python module path).</dd>
</dl>
<p>
The following sections go through this list of schema classes in more
detail.
</p>
</div>
</div>
<div id="outline-container-atomic" class="outline-3">
<h3 id="atomic">Atomic types</h3>
<div class="outline-text-3" id="text-atomic">
<p>
A type in <b>moo</b> schema is considered <i>atomic</i> if it holds no references to
other types. Some details of the atomic types are provided in this
section.
</p>
</div>
<div id="outline-container-boolean" class="outline-4">
<h4 id="boolean">Boolean</h4>
<div class="outline-text-4" id="text-boolean">
<p>
An instance of the <code>boolean</code> schema class is a type that may hold a <i>true</i>
or <i>false</i> value.
</p>
<div class="note" id="orgd6f2c22">
<p>
Like all schema classes, the user is free to make multiple instances
of <code>boolean</code>. However, unlike the other schema classes, these types
will not actually provide any variety of meaning. One <code>boolean</code> type is
like any other even if they are differentiated by their type name.
</p>
</div>
</div>
</div>
<div id="outline-container-number" class="outline-4">
<h4 id="number">Number</h4>
<div class="outline-text-4" id="text-number">
<p>
Instances of the moo <code>number</code> schema class describe numeric types in
terms of representation size, format and optional constraints.
</p>
<p>
The size and format is specified as code which should be a valid
<code>numpy.dtype</code>. For example, <code>i8</code> is a 8 <b>byte</b> (not bit!) signed integer
type and <code>f4</code> is a single-precision floating point type. Numpy supports
a wide variety of "spelling" of the format and size codes however <b>moo</b>
schema is restricted to 2-character <code>dtype</code> codes.
</p>
<p>
The <code>dtype</code> is intended for codegen. For validation, a <code>number</code> schema
instance may also supply numeric constraints with names matching those
from <a href="https://json-schema.org/understanding-json-schema/reference/numeric.html">JSON Schema numeric types</a>. Specifically:
</p>
<dl class="org-dl">
<dt><code>multipleOf</code></dt><dd>a valid value must be a multiple of the given number</dd>
<dt><code>exclusiveMaximum</code></dt><dd>a valid value must be strictly less than the given number</dd>
<dt><code>exclusiveMinimum</code></dt><dd>a valid value must be strictly more than the given number</dd>
<dt><code>maximum</code></dt><dd>a valid value must be less than or equal to the given number</dd>
<dt><code>minimum</code></dt><dd>a valid value must be more than or equal to the given number</dd>
</dl>
<div class="tip" id="org99a5bfa">
<p>
To produce a schema that validates a number has having an integral
value, specify the constraint <code>multipleOf=1</code>.
</p>
</div>
<div class="note" id="org77b1640">
<p>
The JSON Schema "draft 4" interpretation of the exclusive variants is
rejected. That is, all numeric constraints themselves take a numeric
value and not Boolean.
</p>
</div>
<div class="warning" id="orgaad1507">
<p>
Numeric constraints are given as literal numeric values and thus are
subject to limitations of the language in which they are specified.
In particular, Jsonnet treats all numeric values as IEEE754 64-bit
floating point numbers.
</p>
</div>
</div>
</div>
<div id="outline-container-string" class="outline-4">
<h4 id="string">String</h4>
<div class="outline-text-4" id="text-string">
</div>
</div>
<div id="outline-container-enum" class="outline-4">
<h4 id="enum">Enum</h4>
<div class="outline-text-4" id="text-enum">
</div>
</div>
</div>
<div id="outline-container-aggregate" class="outline-3">
<h3 id="aggregate">Aggregate types</h3>
<div class="outline-text-3" id="text-aggregate">
<p>
Some types will hold <b>references</b> to one or more other types. These are
<i>aggregate</i> types. A <b>type reference</b> is represented as a <i>fully-qualified
type name</i> (FQTN) which is formed as the dot-separated concatenation of
the elements of <code>path</code> (if any) and the <code>name</code>. A FQTN shall not begin
nor end with a period. As the <code>path</code> is absolute there is no concept of
a "relative" type reference. See the section <a href="#namespace">Namespace</a> below for
related concepts.
</p>
<p>
Thus, every <b>type</b> exists at an absolutely determined location in a
name hierarchy, and it carries this location with it as <code>path</code> +
<code>name</code>. Types that must reference another type do so by its <code>path</code>
and <code>name</code>. Of course, to resolve a reference the referred type must
be available in order to match the type reference against possible
<code>path</code> and <code>name</code>. For this reason a <b>schema</b> is considered
<b>complete</b> if every type referenced by any of the schema's types are
also included.
</p>
</div>
<div id="outline-container-sequence" class="outline-4">
<h4 id="sequence">Sequence</h4>
<div class="outline-text-4" id="text-sequence">
</div>
</div>
<div id="outline-container-record" class="outline-4">
<h4 id="record">Record</h4>
<div class="outline-text-4" id="text-record">
<p>
The <b>moo</b> <code>record</code> schema class is analogous to Python <code>class</code> or C++ <code>struct</code>
but with it we may only define data members, which in <b>moo</b> are called
<code>fields</code>, and not methods.
</p>
<p>
Each <code>field</code> is itself a small data structure but <b>moo</b> does not
considered it a first class "type". A <code>field</code> merely associates the
following information in the context of the <code>record</code>:
</p>
<dl class="org-dl">
<dt><code>name</code></dt><dd>unique identifier (native type string type)</dd>
<dt><code>item</code></dt><dd>reference the type of the value that the field represents.</dd>
<dt><code>default</code></dt><dd>optionally provide a value directly in the schema (see
note)</dd>
<dt><code>doc</code></dt><dd>optionally provide a brief description of the field.</dd>
<dt><code>optional</code></dt><dd>optionally indicate if this field is optional (<code>true</code> value) or required (<code>false</code>).</dd>
</dl>
<p>
When optional values are not provided, the attribute will be omitted
from the resulting <code>record</code> structure.
</p>
<div class="note" id="org8360be9">
<p>
A field may be given a <code>default</code> value expressed literally in the
language used to define the schema (ie, Jsonnet or Python). This
value may be used by whatever consumes the schema (eg, a template) to
provide a value for the field.
</p>
<p>
When providing a <code>default</code> value, care must be given to assure it is in
a valid form for the <code>field</code> type held by <code>item</code>. In particular, when a
<code>field</code> is itself of a type <code>record</code> type, a full and matching object must
be provided.
</p>
<p>
Special care is typically needed when handling a <code>default</code> value by the
consumer of a schema due to its representation as a literal value.
For example, a <code>default</code> value which is an empty string does not itself
carry any quotes.
</p>
</div>
<p>
A <code>record</code> may also be constructed with an attribute called <code>bases</code> which
provides an array of zero or more references to other <code>record</code> types.
The <code>fields</code> of all <code>record</code> types referenced in <code>bases</code> should be
considered held by the <code>record</code> itself. That is, <code>bases</code> provides a
simple form of object inheritance.
</p>
<p>
Like all aspects of schema, it is up to a consumer of the schema to
determine how to reflect this inheritance information. As examples,
the <code>ostructs.hpp.j2</code> template, described more below, directly reflects
<code>bases</code> into a list of base <code>struct</code>'s in simple C++ inheritance.
Somewhat differently the <code>onljs.hpp.j2</code> template which provides
serialization an <code>ostruct.hpp.j2</code> generated <code>struct</code> and <code>nlohman::json</code>
representations interpret <code>bases</code> simply as providing additional fields.
Thus it enacts a "duck-typed" interface between the type-free JSON
object and the strongly-typed C++ <code>struct</code>.
</p>
</div>
</div>
<div id="outline-container-any" class="outline-4">
<h4 id="any">Any</h4>
<div class="outline-text-4" id="text-any">
<p>
The <code>any</code> schema class provides the "type erasure" pattern. That is an
<code>any</code> type may hold any type. It is like a <code>void*</code> in C or a <code>std::any</code> in
C++. As it may represent any type it holds no type information other
than <code>name</code> and <code>doc</code>.
</p>
<p>
With the <code>ostructs.hpp.j2</code> and <code>onljs.hpp.j2</code> templates, the <code>any</code> type is
mapped to a <code>nlohmann::json</code> type. This can be used to delay
serialization of specific parts of a <code>record</code> type until the instance
can be passed into some more specifically typed C++ context.
</p>
<div class="note" id="org98275c4">
<p>
The <code>any</code> type must hold a value which is itself of a type under the
schema. As there is currently no support in <b>moo</b> to <i>realize</i> moo schema
types in Jsonnet there is no way to provide a <code>default</code> value to a
<code>record</code> with a <code>field</code> of type <code>any</code>. See the <i>otypes</i> document for details
about realizing types and their values in Python.
</p>
</div>
</div>
</div>
<div id="outline-container-xxxof" class="outline-4">
<h4 id="xxxof">xxxOf</h4>
<div class="outline-text-4" id="text-xxxof">
<p>
Three similar aggregate schema classes are: <code>anyOf</code>, <code>allOf</code> and <code>oneOf</code>.
Each type is a collection of references to other types for which
</p>
<dl class="org-dl">
<dt><code>anyOf</code></dt><dd>any may apply</dd>
<dt><code>allOf</code></dt><dd>all apply</dd>
<dt><code>oneOf</code></dt><dd>exactly one may apply</dd>
</dl>
<p>
As always, how they reflect depends on the consumers and not all may
have useful meanings in all contexts. Generally, all three have
meaningful reflections in a validation context. Indeed they are in
<b>moo</b> in order to support JSON Schema validation. The <code>oneOf</code> could be
considered to map to union types such as <code>std::variant</code>. The <code>allOf</code>
could be considered to represent a component with all of the required
interfaces. Given those two definitions, <code>anyOf</code> lacks an obvious use
for code generation.
</p>
</div>
</div>
</div>
<div id="outline-container-org28b3dd8" class="outline-3">
<h3 id="org28b3dd8">Type containers</h3>
<div class="outline-text-3" id="text-org28b3dd8">
</div>
<div id="outline-container-namespace" class="outline-4">
<h4 id="namespace">Namespace</h4>
<div class="outline-text-4" id="text-namespace">
<p>
Every type is defined in a context called a <code>path</code>. Conceptually, a
<code>namespace</code> is a <code>path</code> shared by all types "in" or "under" the namespace.
<b>moo</b> provides Python and Jsonnet helper code which provides a <code>namespace</code>
as a type factory constructing types in a given namespace.
</p>
</div>
</div>
<div id="outline-container-schema-array" class="outline-4">
<h4 id="schema-array">Schema array</h4>
<div class="outline-text-4" id="text-schema-array">
<p>
The most simple way to express a set of <b>moo</b> types is as a <i>schema
array</i>. This is simply a Jsonnet or JSON array or Python <code>list</code> holding
<b>moo</b> type structures. Generally, schema arrays must be <i>topologically
sorted</i> so that no type structure references any other type later than
itself in the array. <b>moo</b> provides Python and Jsonnet code to perform
this topological sort and examples are given below.
</p>
</div>
</div>
</div>
</div>
<div id="outline-container-Schema" class="outline-2">
<h2 id="Schema">Schema Examples</h2>
<div class="outline-text-2" id="text-Schema">
<p>
This section describes how to define a schema using either the <b>Jsonnet</b>
or the <b>Python</b> programming language as a persistent representation. It
also describes how to convert moo schema (from any format) into other
schema systems in particular <b>JSON Schema</b>.
</p>
<p>
To illustrate some of the patterns seen in real,
large-scale projects the example will factor the schema into two
parts:
</p>
<dl class="org-dl">
<dt><i>sys</i></dt><dd>a set of types relevant to some system (eg a framework)</dd>
<dt><i>app</i></dt><dd>a set of types relevant to an application based on the system</dd>
</dl>
</div>
<div id="outline-container-schema-in-jsonnet" class="outline-3">
<h3 id="schema-in-jsonnet">Jsonnet</h3>
<div class="outline-text-3" id="text-schema-in-jsonnet">
<p>
<b>moo</b> <i>oschema</i> may be described easily in the
<a href="https://jsonnet.org/">Jsonnet</a> language as Jsonnet was designed for
defining data structures.
</p>
<p>
We start by simply presenting the <code>sys</code> schema:
</p>
<div class="org-src-container">
<pre class="src src-jsonnet">// examples/oschema/sys.jsonnet
local moo = import "moo.jsonnet";
local sys = moo.oschema.schema("sys");
[sys.number("Count", "u4")]
</pre>
</div>
<p>
Let's walk through this short example line by line:
</p>
<div class="org-src-container">
<pre class="src src-jsonnet">local moo = import "moo.jsonnet";
</pre>
</div>
<p>
This shows Jsonnet's "module" system in action. A file is loaded and its
contents available via the <code>moo</code> variable. The <code>moo.jsonnet</code> file holds
various Jsonnet functions and data that will help build our <i>oschema</i>.
</p>
<div class="org-src-container">
<pre class="src src-jsonnet">local sys = moo.oschema.schema("sys");
</pre>
</div>
<p>
This call of the <code>schema()</code> function of the <code>moo.oschema</code> object
returns another object held by the local variable <code>sys</code> which will
provide sort of a "schema factory" that operates "in" the type path of
"sys". We see it in action:
</p>
<div class="org-src-container">
<pre class="src src-jsonnet">[sys.number("Count", "u4")]
</pre>
</div>
<p>
This last line defines the data structure which is the "return" value
of the entire <code>sys.jsonnet</code> file. That is, this file "compiles" to an
array holding a single <i>oschema</i> type.
</p>
<div class="note" id="org7825af8">
<p>
All <b>moo</b> "schema files" are expected to provide an
array-of-type-objects. We will later see how the order of this array
matters and how to assure it is correct. We will also later see how
this array becomes just one element of a more complex structure called
a <b>model</b> which we will form prior to applying to a <b>template</b>.
</p>
</div>
<p>
We can see how the <i>sys</i> schema expands or compiles to a JSON
representation using <b>moo</b>:
</p>
<div class="org-src-container">
<pre class="src src-shell">moo compile examples/oschema/sys.jsonnet
</pre>
</div>
<div class="org-src-container">
<pre class="src src-json">[
{
"deps": [],
"dtype": "u4",
"name": "Count",
"path": [
"sys"
],
"schema": "number"
}
]
</pre>
</div>
<p>
This is then a schema with a single type called <code>Count</code> which is of
schema class <code>number</code> that has numeric Numpy-style type code <code>dtype</code> of an
unsigned integer in four bytes <code>"u4"</code> and is in a context <code>path</code> of simply
<code>["sys"]</code>.
</p>
<div class="note" id="org2204da8">
<p>
The final structural form of <i>oschema</i> (ie, the JSON above) is not
something that a developer of a schema strictly needs to know. <b>moo</b>
support for Jsonnet and Python provide native language code to assist
in building a schema without exposing all the details of the type
object structure. But these details shall need to be understood if
any new native language support is developed.
</p>
</div>
<p>
Next, we imagine an application schema with a more rich set of types:
</p>
<div class="org-src-container">
<pre class="src src-jsonnet">// examples/oschema/app.jsonnet
local moo = import "moo.jsonnet";
local sa = import "sys.jsonnet";
local sh = moo.oschema.hier(sa);
local as = moo.oschema.schema("app");
local hier = {
counts: as.sequence("Counts",sh.sys.Count,
doc="All the counts"),
email: as.string("Email", format="email",
doc="Electronic mail address"),
affil: as.any("Affiliation",
doc="An associated object of any type"),
mbti: as.enum("MBTI",["introversion","extroversion",
"sensing","intuition",
"thinking","feeling",
"judging","perceiving"]),
make: as.string("Make"),
model: as.string("Model"),
autotype: as.enum("VehicleClass", ["boring", "fun"]),
vehicle : as.record("Vehicle", [
as.field("make", self.make, default="Subaru"),
as.field("model", self.model, default="WRX"),
as.field("type", self.autotype, default="fun")]),
person: as.record("Person", [
as.field("email",self.email,
doc="E-mail address"),
as.field("email2",self.email,
doc="E-mail address", default="me@example.com"),
as.field("counts",self.counts,
doc="Count of some things"),
as.field("counts2",self.counts,
doc="Count of some things", default=[0,1,2]),
as.field("affil", self.affil,
doc="Some affiliation"),
as.field("mbti", self.mbti,
doc="Personality"),
as.field("vehicle", self.vehicle, doc="Example of nested record"),
as.field("vehicle2", self.vehicle, default={model:"CrossTrek", type:"boring"},
doc="Example of nested record with default"),
as.field("vehicle3", self.vehicle, default={model:"BRZ"},
doc="Example of nested record with default"),
], doc="Describe everything there is to know about an individual human"),
};
sa + moo.oschema.sort_select(hier, "app")
</pre>
</div>
<p>
We will go through some of these lines of Jsonnet in order to explain
some of the forms. Starting with the first few lines:
</p>
<div class="org-src-container">
<pre class="src src-jsonnet">local moo = import "moo.jsonnet";
local sa = import "sys.jsonnet";
local sh = moo.oschema.hier(sa);
local as = moo.oschema.schema("app");
</pre>
</div>
<p>
As with <code>sys</code> we import the support module <code>moo.jsonnet</code>. We also import
the <i>sys</i> schema that we made above. Thus, <code>sa</code> holds a single-element
array.
</p>
<p>
Next we call the <code>moo.oschema.hier()</code> method on this array. This
transforms the ordered array of types into an (unordered) object where
each type is available by its name. We'll see <code>sh</code> in use next.
</p>
<p>
Finally for this preamble, we make another "schema factory" which is
this time "in" the path: <code>app</code>. We'll use <code>as</code> many times to build up
elements of our schema.
</p>
<p>
Next we build our <i>sys</i> schema and do so inside a "working object".
This lets us use Jsonnet language feature to refer to one of our types
in another. Let's look at the start:
</p>
<div class="org-src-container">
<pre class="src src-jsonnet">local hier = {
counts: as.sequence("Counts",sh.sys.Count,
doc="All the counts"),
</pre>
</div>
<p>
Here we start our object and save it in a local variable <code>hier</code> and give
it an initial entry. The key name <code>counts</code> is a temporary convenience
and the value is what will ultimately matter. The type we make is a
sequence that references the <code>sys.Count</code> type made in the <i>sys</i> schema.
This shows how "cross schema" references can be made.
</p>
<p>
The next set of types are nothing special but illustrate how instances
of some of the different schema classes in Jsonnet are made:
</p>
<p>
Next, let's jump to the definition of the <code>Person</code> type which begins with:
</p>
<div class="org-src-container">
<pre class="src src-jsonnet">email: as.string("Email", format="email",
doc="Electronic mail address"),
affil: as.any("Affiliation",
doc="An associated object of any type"),
mbti: as.enum("MBTI",["introversion","extroversion",
"sensing","intuition",
"thinking","feeling",
"judging","perceiving"]),
</pre>
</div>
<p>
Now we define an instance of the <code>record</code> schema class:
</p>
<div class="org-src-container">
<pre class="src src-jsonnet">make: as.string("Make"),
model: as.string("Model"),
autotype: as.enum("VehicleClass", ["boring", "fun"]),
vehicle : as.record("Vehicle", [
as.field("make", self.make, default="Subaru"),
as.field("model", self.model, default="WRX"),
as.field("type", self.autotype, default="fun")]),
</pre>
</div>
<p>
In addition to showing how an instance of a <code>record</code> schema class this
example shows how to reference types within our "working object"
through a <code>self</code> object. We will skip the res of our "working object"
other than to say that this <code>Vehicle</code> type is referenced in the <code>Person</code>
type described in the remaining and that this shows an example of
"nested" records.
</p>
<p>
We end the <i>app</i> schema file by producing a "sorted" array of types:
</p>
<div class="org-src-container">
<pre class="src src-jsonnet">sa + moo.oschema.sort_select(hier, "app")
</pre>
</div>
<p>
Like any <i>oschema</i>, this array must include <b>all</b> of the types needed to
satisfy any type references and in an order such that any referenced
types come first. That is, a <a href="https://en.wikipedia.org/wiki/Topological_sorting">topological sort</a> must be applied to the
graph built between types and their references. This is not so
trivial of an operation, but the <code>moo</code> Jsonnet support provides the
required algorithm. As the <i>sys</i> schema is simple and independent from
<i>app</i> by construction we merely prepend it.
</p>
<p>
Sparing the long output here, the full schema compiled to JSON can be
produced with this command:
</p>
<pre class="example" id="org4450887">
moo compile examples/oschema/app.jsonnet
</pre>
<div class="note" id="orgb81a10e">
<p>
While developing a schema, it is very useful to run <code>moo compile</code>
frequently in order to check for Jsonnet syntax or logic errors.
</p>
</div>
</div>
</div>
<div id="outline-container-schema-in-python" class="outline-3">
<h3 id="schema-in-python">Python</h3>
<div class="outline-text-3" id="text-schema-in-python">
<p>
<b>moo</b> provides support for defining <i>oschema</i> in Python which has some
similarities to its Jsonnet support. However, Python is a far more
expressive language than Jsonnet and thus <b>moo</b> Python support provides
more options to the developer.
</p>
<p>
Like the Jsonnet support there are layers of representation of schema
information, transformations between the layers. These are
summarized:
</p>