-
-
Notifications
You must be signed in to change notification settings - Fork 5
/
README.bigb
11716 lines (8803 loc) · 426 KB
/
README.bigb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
= OurBigBook Project
{c}
{id=ourbigbook}
= OurBigBook Project
{c}
{synonym}
\b[https://OurBigBook.com[]]: The \b[OurBigBook Project] is creating the ultimate tools to <publish> \b[textbooks]/\b[<#personal knowledge bases>]/\b[<#Zettelkasten>]/\b[<#digital gardens>] in the <#learn in public> philosophy. It is our best shot yet at the final \b[real-world <#Encyclopedia Galactica>] by allowing effective \b[<#mindmeld>[mind melding]]/\b[<#collective intelligence>] via the \b[<topics feature>].
\Image[logo.svg]
{height=150}
{title=<project logo>[Logo of] the <OurBigBook Project>}
\b[Mission]: to live in a world where you can learn university-level #mathematics{p=1}, #physics{p=1}, <#chemistry>, #biology and #engineering from perfect free open source books that anyone can write to get famous.
\b[Ultimate goal]: destroy the currently grossly inefficient education system and replace it with a much more inspiring system where people learn what they want as fast as possible to reach their goals faster without so much useless pain.
\b[How to get there]: <OurBigBook Web>[create a website] (live at \b[https://OurBigBook.com[]]) that incentivizes learners (notably university students taking courses) to write freely licensed university-level natural science books in their own words for free. Their motivation for doing that are:
* getting their knowledge globally recognized and thus better jobs
* improving the world
* learning by teaching
\Video[https://www.youtube.com/watch?v=7JOJYx0mmhg]
{title=Introduction to the <OurBigBook Project>}
{height=500}
\b[Notable features]:
\Ol[
* \b[<topics>]: groups the articles of different users about the same topic, sorted by upvote to achieve <#mindmeld>[mind melding]/<#collective intelligence>. This makes your articles easier for others to find.
\Image[feature/topics/derivative.png]
{title=The <topics feature> allows you to find the best version of a subject written by other users user}
{description=Live demo: <#derivative>.}
{provider=github}
{height=800}
{border}
\Video[https://www.youtube.com/watch?v=54nxjC9BWTg]
{title=<OurBigBook Web topics> demo}
{disambiguate=intro}
{height=700}
{width=700}
* \b[local editing]: you can store all your <#personal knowledge base> content locally in a <OurBigBook Markup>[plaintext markup format] that can be edited locally and <published> with <OurBigBook CLI> to either:
* \b[to <OurBigBook.com>]: to get awesome multi-user features like <OurBigBook Web topics>
* \b[as HTML files to a <static website>]: you can host yourself for free on many external providers like <GitHub Pages>, and remain in full control
This way you can be sure that even if <ourbigbook.com> were to go down one day (which we have no plans to as it is quite cheap to host!), your content will still be perfectly readable as a static site.
\Image[feature/local-editing/bigb-publish-to-web-or-static.svg]
{title=With OurBigBook you can <publish> local <OurBigBook markup>[lightweight markup files] to either <OurBigBook Web> or as a <static website>}
{provider=github}
{height=600}
{disambiguate=intro}
\Video[https://www.youtube.com/watch?v=Ghvlztiu6rI]
{title=Edit locally and <publish> demo}
{disambiguate=intro}
{description=This shows editing <OurBigBook Markup> and <publishing> it using the <VS Code> extension.}
{height=720}
{width=720}
]
\b[Key links]:
* https://OurBigBook.com[]: reference <OurBigBook Web> instance
* donate to the OurBigBook Project: <donate>
* project announcements: <news>{full}. Also posted in shorter form to <official accounts>{full} such as:
* https://twitter.com/OurBigBook[]
* https://mastodon.social/@OurBigBook
* https://cirosantilli.com/ourbigbook-com[]: further rationale behind the project by the founder <Ciro Santilli>
* https://cirosantilli.com[]: showcase static demo document with interesting content, published with <OurBigBook CLI>. Primary inspiration for OurBigBook development.
* https://cirosantilli.com/oxford-nanopore-river-bacteria[]: a self-contained tutorial style part of the above. Note how internal links integrate seamlessly into the more global topic of biology, e.g. when talking about DNA we link to the global topic https://cirosantilli.com/dna[].
* https://github.com/cirosantilli/cirosantilli.github.io[] and https://github.com/cirosantilli/cirosantilli.github.io/blob/dev/oxford-nanopore-river-bacteria.bigb[]: source of the above showcase documents
* <design goals>{full}: <OurBigBook Markup> and <OurBigBook CLI> feature overview
* https://github.com/ourbigbook/ourbigbook[]: OurBigBook source code
* https://github.com/ourbigbook/ourbigbook/issues[]: project issue tracker
* https://github.com/ourbigbook/ourbigbook/blob/master/README.bigb[]: source for this document
* https://docs.ourbigbook.com[]: rendered version of this document
* https://docs.ourbigbook.com/_obb/dist/editor[]: live in-browser editor demo
* https://github.com/ourbigbook/template[]: good template to get started with <OurBigBook CLI>, see <OurBigBook CLI quick start>{full}
* https://ourbigbook.github.io/template[]: the above <published to GitHub Pages>
* https://cirosantilli.com/ourbigbook-media[]: media for the project such as for documentation and <publicity>, more info: <OurBigBook media repository>{full}
= Donate
{parent=OurBigBook}
To donate:
* https://cirosantilli.com/sponsor[]: give money directly to <Ciro Santilli>
* buy project merchandise, see: <merchandise>{full}
All donated money currently just goes directly to <Ciro Santilli>'s personal bank account, the project founder and current lead. If things ever take off we will set up a legal entity to make things cleaner. One may dream. But for now, it would just add unnecessary overhead. Related: <Project governance>{full}.
Ciro announces funding milestones and transparently accounts for all donations at: https://cirosantilli.com/sponsor[]. When milestones are reached, he quits his day job and works full time on the project for a given ammount of time.
We are also happy to discuss paid contracts to implement specific features, to get in touch see: <contact>.
= Quick start
{parent=OurBigBook}
The following sections cover different ways to use tools from the OurBigBook:
* <OurBigBook Web user manual>: manual for <OurBigBook Web>, the dynamic website. This is what most users will use by default when it is matured;.
* <OurBigBook CLI quick start>: allows you to convert local `.bigb` files from the command line to either:
* a static website. This is already very usable.
* upload to <OurBigBook Web>. Almost very usable, we're getting there.
* <OurBigBook Markup quick start>: covers specifically <OurBigBook Markup>, which is the markup language you use to write content in OurBigBook, both <OurBigBook Web> and <OurBigBook CLI>.
This is currently the only way to write OurBigBook content, but we would really like to add <todo/WYSIWYG>[WYSIWYG] editor support one day!
= OurBigBook Markup and CLI overview
{parent=OurBigBook}
= Features
{parent=OurBigBook Markup and CLI overview}
* <cross references> to any <header> (including e.g. h2, h3, etc. <internal cross file references>[in other files]), <images>, etc. with amazing <error reporting>[error checking and reporting]: never break internal links without knoing again, and quickly find out what broke when you do. E.g.:
animal.bigb
``
= Animal
<Bats> are <flying animals>.
``
Mammal.bigb
``
= Flying animal
== Bat
``
`Animal.bigb` would render something like:
``
<a href="flying-animal.html#bat">Bats</a> are <a href="flying-animal.html">flying animals</a>.
``
The following would fail and point you out the file and line of the failure:
* nonexistent id:
``
<Weird animal not documented>
``
* duplicate IDs:
``
= Animal
== Dog
== Cat
== Dog
``
* https://katex.org/[KaTeX] server side <mathematics>, works on browsers with JavaScript disabled:
``
I like $\sqrt{2}$, but I adore this \x[equation-quadratic-equation]:
$$
x^2 + 2x + 1
$$
{title=Quadratic equation}
``
* multi-file features out of the box so you don't need a separate wrapper like Jekyll to make a multi-page website:
* <cross file references>
* single-source multi-format output based on <includes> and build options:
* by default, one HTML per source with includes rendered as links between pages, e.g.:
README.bigb
``
= My website
== h2
\Include[not-readme]
``
not-readme.bigb
``
= Not readme
== Not readme h2
``
produces `index.html` and `not-readme.html`
* with the <split headers> option, you can output each header of an input file into a separate output file. The previous filesystem would produce:
* `index.html`: which contains the full `README.bigb` output
* `split.html`: split version of the above containing only the `= My website` header and not `h2`
* `h2.html`: only contains the `h2` header
* `not-readme.html` contains the full output of `not-readme.bigb`
* `not-readme-split.html`: only contains the `= Not readme` header
* `not-readme-h2.html`: only contains the `= Not readme h2` header
Each of those pages automatically gets a <table of contents>
* <--embed-includes> single file output from multiple input files. Includes are parsed smartly, not just source copy pasted, e.g. included headers are shifted from `h1` to `h2` correctly.
On the previous sample filesystem, it would produce a single output file `index.html` which would contain a header structure like:
``
= My website
== h2
=== Not readme
==== Not readme h2
``
* supports both local serverless rendering to HTML files for local viewing, and server oriented rendering such as GitHub pages, e.g. <cross references> automatically get `.html` extension and or not. E.g.:
* locally, a link `\x[not-readme]` would render as `<a href="not-readme.html">` and `not-readme.bigb` produces `not-readme.html`
* when publishing, `\x[not-readme]` would render as `<a href="not-readme">` and `not-readme.bigb` also produces `not-readme.html`, which the server converts to just `http://my-website.com/not-readme`
* cross file configuration files to factor out common page parts like headers, footers and other metadata, e.g.:
* `ourbigbook.liquid.html`: https://github.com/Shopify/liquid[Liquid template] used for all pages, see example at: <play with the template>{full}
* `main.scss`: CSS stylesheet generated from https://sass-lang.com/[SASS] input, see example at: <play with the template>{full}
* `ourbigbook.tex`: global LaTeX math definitions, e.g.:
``
\newcommand{\abs}[1]{\left|#1\right|}
``
and then you can use:
``
$\abs{x}$
``
in any .bigb file of the project.
* <ourbigbook.json>: per repository configuration options
* <table of contents> that crosses input files via includes. E.g. in:
README.bigb
``
= My website
== h2
\Include[not-readme]
``
not-readme.bigb
``
= Not readme
== Not readme h2
``
the table of contents for `index.html` also contains the headers for `not-readme.bigb` producing:
* My website
* h2
* Not readme
* Not readme h2
This means that you can split large <\H splitDefault argument>[splitDefault] input files if rendering starts to slow you down, and things will still render exactly the same.
* check that local files and images linked to actually exist: <\a external argument>. E.g.:
``
\a[i-don-exist.txt]
``
would lead to a build error.
* associate headers to files or directories with the <\H file argument> e.g.:
``
Here's an example of a nice image: \x[path/to/my/image.png]{file}.
= path/to/my/image.png
{file}
This image was taken when I was on vacation!
``
would automatically add a preview of the image on the output. Display files and their metadata nicely directly on your static website rather than relying exclusively on GitHub as a file browser.
* advanced header/ID related features:
* <ID-based header levels>:
``
= Furry animal
I like \x[furry-animal]{p}, especially my cat, here is his photo: \x[image-my-cat].
== Cat
\Image[My_cat.jpg]
{title=My cat}
``
* <scopes> either with directories or with within a single file:
``
See the important conclusion of my experiment: \x[report-of-my-experiment/conclusion]
= Report of my experiment
{scope}
== Introduction
== Middle
== Conclusion
``
* <cross reference title inflection> for capitalization and pluralization, e.g.;
``
= Dog
== Snoopy
{c}
\x[dog]{c}{p} are fun. But the \x[dog] I like the most is \x[snoopy]!
``
would render:
* `\x[dog]{c}{p}` as `Dogs`: capitalized because of `{c}` and pluralized because of `{p}`
* `\x[dog]` as `dogs`: auto lowercased because its header `= Dog` does not have `{c}`
* `\x[snoopy]` as `Snoopy`: title capitalization kept to upper case due to `{c}` on the header `== Snoopy`
* <synonyms>, e.g.:
``
= User interface
= UI
{c}
{synonym}
{title2}
\x[user-interface]{c} is too long, I just say \x[ui].
``
would render something like:
``
<a href="#user-interface">User interface</a> is too long, I just say <a href="user-interface">UI</a>
``
Furthermore, this also generates a output file:
``
ui.html
``
which redirects to the ain `user-interface.html`, so it serves as a way to have backward compatibility on page renames.
And the `title2` makes it appears on the main title under parenthesis, something like:
``
<h1>User interface (UI)</h1>
``
* <\H disambiguate argument>[header disambiguation], e.g.:
``
My favorite fruits are \x[apple-fruit]{p}!
My favorite least favorite brand is is \x[apple-company]! \x[apple] computers are too expensive.
== Apple
{disambiguate=fruit}
== Apple
{c}
{disambiguate=company}
= Apple
{c}
{synonym}
``
which renders something like:
* `\x[apple-fruit]{p}`: `<a href="apple-fruit">apples</a>`
* `\x[apple-company]`: `<a href="apple-company">Apple</a>`
* `\x[apple]`: also `<a href="apple-company">Apple</a>` because of the synonym
* `== Apple\n{disambiguate=fruit}`: `<h2 id="apple-fruit">Apple (fruit)</h2>`
* `== Apple\n{disambiguate=company}`: `<h2 id="apple-company">Apple (company)</h2>`
* tags are regular headers: <\H child argument>, <\x child argument>
``
= Animal
== Dog
{tag=domestic}
{tag=cute}
== Cat
{tag=domestic}
{tag=cute}
== Bat
{tag=flying}
= Flying
= Cute
= Domestic
``
* <unlimited header levels>, levels higher than 6 are rendered in HTML as an appropriately styled `div`s with an ID:
``
= h1
== h2
=== h3
==== h4
===== h5
====== h6
======= h7
======== h8
``
* generate lists of <incoming links> between internal headers: it shows every internal link coming into the current page
* automatic file upload and directory listing of non OurBigBook files: <`_raw` directory>, e.g.:
* link to a file:
\OurBigBookExample[[
The file \a[index.js] is cool.
]]
* link to a directory:
\OurBigBookExample[[
The directory \a[file_demo] is cooler.
]]
* is written in JavaScript and therefore runs natively on the browser to allow live previews as shown at: https://docs.ourbigbook.com/_obb/dist/editor[]
* helps you with the publishing:
* <ourbigbook --publish> publishes in a single command to the configured target (default <GitHub Pages>)
* OurBigBook tries to deal with media such as images and video intelligently for you, e.g.: <where to store images>{full}. E.g. you can keep media in a separate media repository, `my-media-repository`, and then by configuring on <ourbigbook.json>:
``
"media-providers": {
"github": {
"default-for": ["image", "video"],
"path": "media",
"remote": "yourname/myproject-media"
}
}
``
you can use images in that repository with:
``
\Image[My_image_basename.jpg]
``
instead of:
``
\Image[https://raw.githubusercontent.com/cirosantilli/myproject--media/master/My_image_basename.jpg]
``
* `inotifywait` watch and automatically rebuild with <watch>:
``
ourbigbook --watch input-file.bigb
``
* automatic code formatting: <--format-source>
= Design goals
{parent=OurBigBook Markup and CLI overview}
OurBigBook is designed entirely to allow writing complex professional HTML and PDF scientific books, blogs, articles and encyclopedias.
OurBigBook aims to be the ultimate <latex output format>[LaTeX] "killer", allowing books to be finally published as either HTML or PDF painlessly (LaTeX being only a backend to PDF generation).
It aims to be <features>[more powerful] and <saner> and than Markdown and Asciidoctor.
= Saner
{parent=Design goals}
Originally, OurBigBook was is meant to be both saner and more powerful than Markdown and Asciidoctor.
But alas, as Ciro started implementing and using it, he started to bring some Markdown <insane macro shortcut>[insanity he missed back in].
And so this "degraded" slightly into a language slightly saner than Asciidoctor but with an amazing Node.js implementation that makes it better for book writing and website publishing.
Notably, we hope that our escaping will be a bit saner backslash escapes everything instead of Asciidoctor's "different escapes for every case" approach: https://github.com/asciidoctor/asciidoctor/issues/901
But hopefully, having starting from a saner point will still produce a saner end result, e.g. there are sane constructs for every insane one.
It is intended that this will be an acceptable downside as OurBigBook will be used primarily large complex content such as books rather than forum posts, and will therefore primarily written either:
* in text editors locally, where users have more features than in random browser textareas
* in a dedicated website that will revolutionize education, and therefore have a good JavaScript editing interface: https://github.com/cirosantilli/write-free-science-books-to-get-famous-website
For example, originally OurBigBook had exactly five magic characters, with similar functions as in LaTeX:
* `\` backslash to start a macro, like LaTeX
* `{` and `}`: left and right square brackets to delimit <positional vs named arguments>[optional macro arguments]
* `[` and `]`: left and right curly braces bracket to start an optional arguments
and double blank newlines for <paragraphs> if you are pedantic, but this later degenerated into many more with <insane macro shortcuts>.
We would like to have only square brackets for both optional and mandatory to have even less magic characters, but that would make the language difficult to parse for computer and humans. LaTeX was right for once!
This produces a very regular syntax that is easy to learn, including doing:
* arbitrary nesting of elements
* adding arbitrary properties to elements
This sanity also makes the end tail learning curve of the endless edge cases found in Markdown and Asciidoctor disappear.
The language is designed to be philosophically isomorphic to HTML to:
* further reduce the learning curve
* ensure that most of HTML constructs can be reached, including arbitrary nesting
More precisely:
* macro names map to tag names, e.g.: `\\a` to `<a`
* one of the arguments of macros, maps to the content of the HTML element, and the others map to attributes.
E.g., in a link:
``
\a[http://example.com][Link text\]
``
the first macro argument:
``
http://example.com
``
maps to the `href` of `<a`, and the second macro argument:
``
Link text
``
maps to the internal content of `<a>Link text<>`.
= More powerful
{parent=Design goals}
The <saner>[high sanity of OurBigBook], also makes creating new macro extensions extremely easy and intuitive.
All built-in language features use the exact same API as new extensions, which ensures that the extension API is sane forever.
Markdown is clearly missing many key features such as block attributes and <cross references>, and has no standardized extension mechanism.
The "more powerful than Asciidoctor" part is only partially true, since Asciidoctor is very featureful can do basically anything through extensions.
The difference is mostly that OurBigBook is completely and entirely focused on making amazing scientific books, and so will have key features for that application out-of-the box, notably:
* amazing header/ToC/ID features including proper error reports: never have a internal broken link or duplicate ID again
* <mathematics>[server side pre-rendered maths with KaTeX]: all divs and spans are ready, browser only applies CSS, no JavaScript gets executed
* <publish>: we take care of website publishing for you out-of-the-box, no need to integrate into an external project like Jekyll
* <split headers>:
* https://github.com/asciidoctor/asciidoctor/issues/626 feature request
* https://github.com/owenh000/asciidoctor-multipage third party plugin that does it
and we feel that some of those features have required specialized code that could not be easily implemented as a standalone macro.
Another advantage over Asciidoctor is that the reference implementation of OurBigBook is in JavaScript, and can therefore be used on browser live preview out of the box. Asciidoctor does Transpile to JS with https://github.com/opal/opal[Opal], but who wants to deal with that layer of complexity?
= Related projects
{parent=Design goals}
Static wiki generators: this is perhaps the best way of classifying this project :-)
* https://github.com/gollum/gollum[]: already has a local server editor! But no WYSIWYG nor live preview. Git integration by default, so when you save on the UI already generates a Git commit. We could achieve that with: https://github.com/isomorphic-git/isomorphic-git[], would be really nice. Does not appear to have built-in static generation:
* https://stackoverflow.com/questions/7210391/have-anyone-use-gollum-site-to-generete-markdown-wikis-and-host-it-on-heroku
* https://github.com/dreverri/gollum-site
Does not appear to check that any links are correct.
* https://github.com/wcchin/markypydia
* https://obsidian.md/ closed source, Markdown with <cross file reference> + a SaaS. Appears to require payment for any publishing. 28k followers 2021: https://twitter.com/obsdmd[]. Founders are likely Canadians of Asian descent from Waterloo University: https://www.linkedin.com/in/lishid/ | https://www.linkedin.com/in/ericaxu/ also working in parallel on https://dynalist.io/ 2020 review at: https://www.youtube.com/watch?v=aK2fOQRNSxc Has offline editor with side-by-side preview. Compares with https://roamresearch.com/[Roam] and https://roamresearch.com/[Notion], but can't find any public publishing on those, seem to be enterprise only things.
Static book generators:
* https://github.com/rstudio/bookdown[], https://bookdown.org/[]. Very similar feature set to what we want!!! Transpiles to markdown, and then goes through Pandoc: https://bookdown.org/yihui/bookdown/pandoc.html[], thus will never run on browser without huge translation layers. But does have an obscene amount of output formats however.
* https://gohugo.io/[Hugo]. Pretty good, similar feature set to ours. But Go based, so hard on browser, and adds adhoc features on top of markdown once again
* https://en.wikipedia.org/wiki/Personal_wiki
* https://github.com/vimwiki/vimwiki
* https://github.com/hplgit/doconce
* https://www.gwern.net/About#source is pretty interesting, uses https://github.com/jaspervdj/Hakyll/ + some custom stuff.
* https://github.com/JerrySievert/bookmarkdown
* https://www.gitbook.com/
* https://github.com/rust-lang/mdBook[]. Impressive integrated search feature. Like Gitbook but implemented in Rust.
* https://github.com/facebook/docusaurus React + markdown based, written in TypeScript. So how can it be build fast? Gotta benchmark.
* vimdoc: http://vimdoc.sourceforge.net/ They do have perfectly working <Internal cross file references>, see any page e.g. http://vimdoc.sourceforge.net/htmldoc/pattern.html[].
* typst: https://github.com/typst/typst An attempt at a LaTeX killer. Has its own typesetting engine, does not simply transpile to LaTeX. Meant to be faster and simpler to write. No HTML output as of writing: https://github.com/typst/typst/issues/721
Less related but of interest, similar philosophy to what Ciro wants, but no explicitly reusable system:
* http://www.uprtcl.io/
* https://libretexts.org
* https://physics.info/
* https://hypertextbook.com/
* https://tutorial.math.lamar.edu/
= Motivation
{parent=Design goals}
<Ciro Santilli> developed OurBigBook to perfectly satisfy his writing style, which is basically "create one humongous document where you document everything you know about a subject so everyone can understand it, and just keep adding to it".
https://cirosantilli.com[] is the first major document that he has created in OurBigBook.
He decided to finally create this new system after having repeatedly facing limitations of Asciidoctor which were ignored/wontfixed upstream, because Ciro's writing style is not as common/targeted by Asciidoctor.
Following large documents Ciro worked extensively on:
* https://github.com/cirosantilli/china-dictatorship
* https://github.com/cirosantilli/linux-kernel-module-cheat
made the limitations of Asciidoctor clear to Ciro, and were major motivation in this work.
The key limitations have repeatedly annoyed Ciro were:
* cannot go over header level 6, addressed at: <unlimited header levels>
* the need for <split headers> to avoid one too large HTML output that will never get indexed properly by search engines, and takes a few seconds to load on any browser, which is unacceptable user experience
= OurBigBook Markup
{c}
{parent=OurBigBook}
= `.bigb` extension
{synonym}
{title2}
OurBigBook Markup is the https://en.wikipedia.org/wiki/Lightweight_markup_language[lightweight markup language] used in the <OurBigBook> project.
It works both on the <OurBigBook Web> dynamic website, and on <OurBigBook CLI> static websites from the command line.
<OurBigBook Markup> files use the `.bigb` extension.
= OurBigBook Markup quick start
{c}
{parent=OurBigBook Markup}
<Paragraphs> are made by simplying adding an empty line, e.g.:
\OurBigBookExample[[
My first paragraph.
And now my second paragraph.
Third one to finish.
]]
<Headers> are created by starting the line with equal signs. The more equal signs the deeper you are, e.g.:
``
= Animal
== Mammal
=== Dog
=== Cat
== Bird
=== Pigeon
=== Chicken
``
On <OurBigBook Web>, the toplevel header of each page goes into a separate title box, so there things would just look like:
* title box: "Animal"
* body:
``
== Mammal
=== Dog
=== Cat
== Bird
=== Pigeon
=== Chicken
``
You can can use any header as a <`\H` `tag` argument>[tag] of any other header, e.g.:
``
= Animal
== Dog
{tag=Cute animal}
== Turtle
{tag=Ugly animal}
== Animal cuteness
=== Cute animal
=== Ugly animal
``
Headers have several powerful features that you can read more about under <`\H` arguments>, e.g. <`\H` `synonym` argument> and <`\H` `disambiguate` argument>.
To <internal cross references>[link to any of your other pages], you can use angle brackets (less than/greater than) signs:
``
I have a <cute animal>. <Birds> are too noisy.
``
Note how <cross reference title inflection>[capitalization and pluralization generally just work].
To use a custom link text on a reference, use the following syntax:
``
I have a <cute animal>[furry animal]. <Birds>[feathery animals] are too noisy.
``
External links can be input directly as:
\OurBigBookExample[[
This is a great website: https://example.com
I really like https://example.com[this website].
]]
<Code blocks> are done with backticks \c[[`]]. With just one backtick, you get a code block inside the text:
\OurBigBookExample[[
The function call `f(x + 1, "abc")` is wrong.
]]
and with two ore more backticks you get a code block on its own line, and possibly with multiple code lines:
\OurBigBookExample[[
The function:
``
function f(x, s) {
return x + s
}
``
is wrong.
]]
<Mathematics> syntax is very similar to code blocks, you can just enter you LaTeX code in it:
\OurBigBookExample[[
The number $\sqrt{2}$ is irrational.
The same goes for:
$$
\frac{1}{\sqrt{2}}
$$
]]
We also have <Built-in math macros>[a bunch of predefined macros from popular packages], e.g. `\dv` from the https://mirrors.ibiblio.org/CTAN/macros/latex/contrib/physics/physics.pdf[`physics` package] for derivatives:
\OurBigBookExample[[
$$
\dv{x^2}{x} = 2x
$$
]]
You can <internal cross references>[refer] to specific equations like this:
\OurBigBookExample[[
As shown in <equation Very important equation>, this is true.
$$
\frac{1}{\sqrt{2}}
$$
{title=Very important equation}
]]
<Images> and <videos> are also easy to add and <internal cross references>[refer] to:
\OurBigBookExample[[
As shown at <image Cute chicken chick>, chicks are cute.
\Image[https://upload.wikimedia.org/wikipedia/commons/thumb/c/c9/H%C3%BChnerk%C3%BCken_02.jpg/800px-H%C3%BChnerk%C3%BCken_02.jpg?20200716091201]
{title=Cute chicken chick}
\Video[https://www.youtube.com/watch?v=j_fl4xoGTKU]
{title=Top Down 2D Continuous Game by Ciro Santilli (2018)}
]]
Images can take a bunch of options, about which you can read more about at <image arguments>. Most should be self explanatory, here is an image with a bunch of useful arguments:
\OurBigBookExample[[
\Image[https://upload.wikimedia.org/wikipedia/commons/thumb/c/c9/H%C3%BChnerk%C3%BCken_02.jpg/800px-H%C3%BChnerk%C3%BCken_02.jpg?20200716091201]
{title=Ultra cute chicken chick}
{description=
The chicken is yellow, and the hand is brown.
The background is green.
}
{border}
{height=400}
{source=https://commons.wikimedia.org/wiki/File:H%C3%BChnerk%C3%BCken_02.jpg}
]]
<Lists> are written by starting the line with an asterisk `*`:
\OurBigBookExample[[
* first item
* second item
* and the third
]]
A nested list:
\OurBigBookExample[[
* first item
* first item version 1
* first item version 2
* first item version 2 1
* first item version 2 2
* second item
* and the third
]]
Lists items can contain any markup, e.g. paragraphs. You just need to keep the same number of spaces, e.g.:
\OurBigBookExample[[
* first item.
Second paragraph of first item.
And a third one.
* second item
* second item v1
Another paragraph in second item v1
* second item v2
]]
<Tables> are not very different from lists. We use double pipes for headers `||`, and a single pipe `|` for regular rows:
\OurBigBookExample[[
|| City
|| Sales
| Salt Lake City
| 124,00
| New York
| 1,000,000
]]
To add a title we need to use an explicit `\Table` macro as in:
\OurBigBookExample[[
See <table Sales per city> for more information.
\Table
{title=Sales per city}
[
|| City
|| Sales
| Salt Lake City
| 124,00
| New York
| 1,000,000
]
]]
= Macro
{parent=OurBigBook Markup}
This section documents all OurBigBook macros.
Macros are magic commands that do cool stuff, e.g. `\Image` to create an image.
The most common macros also have <insane macro shortcuts> to keep the syntax shorter.
The general macro syntax is described at <OurBigBook Markup syntax>{full}.
= Link
{parent=Macro}
{tag=Macro with insane shortcut}
= `\a`
{synonym}
{title2}
= `\a` macro
{synonym}
<Insane> autolink, i.e. the link text is the same as the link address:
\OurBigBookExample[[
The website http://example.com is cool. See also:
\Q[http://example.com/2]
]]
Exact parsing rules described at: <insane link parsing rules>{full}.
Note that the prefixes `http://` and `https://` are automatically removed from the displayed link, since they are so common that they woudly simply add noise.
Equivalent sane version:
\OurBigBookExample[[[
The website \a[http://example.com] is cool.
\Q[\a[http://example.com/2]]
]]]
Insane link with custom text:
\OurBigBookExample[[The website http://example.com[example.com] is cool.]]
Equivalent sane version:
\OurBigBookExample[[The website \a[http://example.com][example.com] is cool.]]
If the custom text is empty, an autolink is generated. This is often useful if you want your link to be followed by punctuation:
\OurBigBookExample[[The website is really cool: http://example.com[].]]
This could also be achieved with the sane syntax of course, but this pattern saves a tiny bit of typing.
Link with multiple paragraphs inside it:
\OurBigBookExample[[
\a[http://example.com][Multiple
paragraphs]
]]
Link to a file in the current repository:
\OurBigBookExample[[
The file \a[index.js] is cool.
]]
This links to a raw view of that file.
Link to a directory in the current repository:
\OurBigBookExample[[
The directory \a[file_demo] is cooler.
]]
This links to an output file that contains a generated directory listing of that directory.
= `\a` `href` argument
{parent=Link}
The link target, e.g. in:
``
\a[http://example.com]
``
`href` equals `http://example.com`.
Important behaviours associated with this property for local links are detailed at <\a external argument>{full}:
* they are checked for existence in the local filesystem
* they are modified to account for <scopes> with <split headers>
= `\a` `ref` argument
{parent=Link}
{tag=Boolean argument}
Analogous to the <\x ref argument>, e.g.:
\OurBigBookExample[[Trump said this and that.https://en.wikipedia.org/wiki/Donald_Trump_Access_Hollywood_tape#Trump's_responses{ref}https://web.archive.org/web/20161007210105/https://www.donaldjtrump.com/press-releases/statement-from-donald-j.-trump{ref} Then he said that and this.https://en.wikipedia.org/wiki/Donald_Trump_Access_Hollywood_tape#Trump's_responses{ref}https://web.archive.org/web/20161007210105/https://www.donaldjtrump.com/press-releases/statement-from-donald-j.-trump{ref}]]
= `\a` `external` argument
{parent=Link}
{tag=Boolean argument}
If given and true, forces a the link to be an <external link>.
Otherwise, the external is automatically guessed based on the address given as explained at <external link>{full}.
Common use cases for the `external` argument is to link to non OurBigBook content in the curent domain, e.g.:
* <link to the domain root path> for <Subdirectory deployments>
* link non OurBigBook subdirectories. E.g., https://github.com/cirosantilli/cirosantilli.github.io/blob/master/README.bigb was rendered at https://cirosantilli.com[], and contains links `\a[markdown-style-guide]{external}` to https://cirosantilli.com/markdown-style-guide[], whose source lives in a separate non-OurBigBook repository: https://github.com/cirosantilli/markdown-style-guide/
= Internal links are smart
{parent=`\a` `external` argument}
= Link to the domain root path
{synonym}
{title2}
The <`\a` `external` argument> can be used to refer to the root of the domain. E.g. suppose that we have a <subdirectory deployment> under `https://mydomain.com/subdir/`. Then:
* `\a[/somepath]` refers to the directory `/subdir/somepath`
* `\a[/somepath]{external}` refers t othe directory `/somepath`
= Subdirectory deployment
{parent=a `external` argument}
TODO test if it works. But we want it to be possible to deploy <OurBigBook CLI> static websites on subdirectories, e.g.:
``
https://mydomain.com/subdir/
https://mydomain.com/subdir/mathematics
``
If it doesn't work, it should be easy to make it work, as we use relative links almost everywhere already. Likely there would only be some minor fixes to the <--template> arguments.
= External link
{parent=a `external` argument}
= Internal link
{synonym}
An external link is a link that points to a resource that is not present in the curent OurBigBook project sources.
By default, most links are internal links, e.g. it is often the case in computer programming tutorials that we want to refer to source files in the current directory. So from our `README.bigb`, we could want to write something like:
\OurBigBookExample[[Have a look at this amazing source file: \a[index.js].]]
and here `\a[ourbigbook]` is a internal link.
A typicial external link is something like:
\OurBigBookExample[[This is great website: https://cirosantilli.com]]
which points to an absolute URL.
OurBigBook considers a link relative by default if:
* it is not a <URL with protocol>
Therefore, the following links are external by default:
* `http://cirosantilli.com`
* `https://cirosantilli.com`
* `file:///etc/fstab`
* `ftp://cirosantilli.com`
and the following are internal by default:
* `index.js`
* `../index.js`
* `path/to/index.js`
* `/path/to/index.js`. Note that paths starting with `/` refer to the root of the <OurBigBook CLI> deployment, not the root of the domain, see: <link to the domain root path>.
* `//example.com/path/to/index.js`
A link being internal has the following effects
* the correct relative path to the file is used when using nested <scopes> with <split headers>. For example, if we have:
``
= h1
== h2
{scope}
=== h3
\a[index.js]
``
then in split header mode, `h3` will be rendered to `h2/h3.html`.
Therefore, if we didn't do anything about it, the link to `index.js` would render as `href="index.js"` and thus point to `h2/index.js` instead of the correct `index.js`.
Instead, OurBigBook automatically converts it to the correct `href="../index.js"`
* the <`_raw` directory> prefix is added to the link
* existence of the file is checked on compilation. If it does not exist, an error is given.
Implemented at: https://github.com/ourbigbook/ourbigbook/issues/87[] as `relative`, and subsequently modified to the more accurate/useful `external`.
= `_dir` directory
{parent=a `external` argument}
The `_dir` directory tree contains file listings of files in the `_raw` directory.
We originally wanted to place these listings under `_raw` itself, but this leads to unsolvable conflicts when there are files called `index.html` present vs the index.
= `_file` output directory
{parent=a `external` argument}
= `_file` directory
{synonym}
Analogous to the <`_raw` directory>, but for the <`\H` `file` argument>.
= `_raw` directory
{parent=a `external` argument}
<OurBigBook> places output files that are not the output of `.bigb` to `.html` conversion (i.e. `.html` output files) under the `_raw/` prefix of the output.
<Internal links> then automatically add the `_raw/` prefix to every link.
For example, consider an input directory that contains:
notindex.bigb
``
= Hello
Check out \a[myfile.c].
The source code for this file is at: \a[notindex.bigb].
\Image[myimg.png]
``
myfile.c
``
int i = 1;
``
myimg.png
``
Binary!
``
After conversion with:
``
ourbigbook .
``
the following files would exist in the output directory:
* `notindex.html`: converted output of `notindex.bigb`
* `_raw/notindex.bigb`: a copy of the input source code `notindex.bigb`
* `_raw/myfile.c`: a copy of the input file `myfile.c`
* `_raw/myimg.png`: a copy of the input file `myimg.c`
and all links/image references would work and automtically point to the correct locations under `_raw`.
Some live examples:
* link to a file: