Migrate spec and "By Example" to new docs system

[adrianholovaty]

This GitHub issue serves as a general holding place for remaining to-dos in migrating our docs.

As of 2021-01-21, the doc system is still in progress but is already capable of:

• Displaying list and detail pages for elements, examples, data types and concepts
• Allowing editing of all that data, via the Django admin
• Serializing/deserializing the raw database to a JSON file
• Generating a static HTML version of the docs

See the docgenerator README for more details. You can browse the new docs here.

Remaining to-dos are:

• Migrate all stable parts of the spec into the doc system. This involves creating "XML Elements" in the DB, adding their attributes/relationships/datatypes and adding descriptions.
• Figure out what to do with the MusicXML examples of "MNX By Example." All MNX examples from that older document have already been migrated into the new system; only the MusicXML examples and the short textual description of each example remain.
• Improve the new docs system to deal with "complex types." This is especially crucial in order to be able to support MusicXML.
• Improve the docs design. I'm planning to ask a colleague to help with this.
• Improve the new docs system to support one-off HTML pages, such as tutorials.
• Improve the new docs system to generate XSD from its database.

[cecilios]

@adrianholovaty Thanks for working on this. Regarding the MusicXML examples on the previous page "MNX By Example", they were very illustrative and useful to compare and assess the changes. I would vote to keep them on the new pages when convenient, side by side as they were on the old pages, if it is possible and their maintenance does not require excessive work.

[adrianholovaty]

@cecilios Thanks for the thoughts. Yes, I agree the comparisons with MusicXML are very useful!

[notator]

Here's my TODO list for the MNX by Example examples, starting from the current (readonly) examples:

1. There are 10 examples that need to be updated to use a '#' character at the start of each reference to a local ID.
2. Add a (graphics only) MNX by Example for grouped parts (see How to group parts? #185 (comment)).
3. Finalise Actions #214 (Actions) and Texts #215 (Texts), so as to reach agreement on the <actions> and <text> elements (or some other solution).
4. The 5 Repeat examples need versions containing temporal information (<actions>, <goto> etc.) as in the Jumps examples (MNX by Example: Jumps 3 (dal segno al Coda) #218, MNX by Example: Jumps 2 (dal Segno al Fine) #220, MNX by Example: Jumps1 (dal Segno) #221). One of them also needs <text>. These examples depend on the result of Actions #214 and Texts #215.
5. Agree and add the examples in MNX by Example: Jumps 3 (dal segno al Coda) #218, MNX by Example: Grand Staff with cross-staff beams #219, MNX by Example: Jumps 2 (dal Segno al Fine) #220, MNX by Example: Jumps1 (dal Segno) #221, MNX by Example: Mid-measure repeats #222, MNX by Example: Beamed grace notes #224, MNX by Example: Beams (with inner grace note) #225. These also depend on the result of Actions #214 and Texts #215.
6. The following additional issues need to be finalized, resulting in general agreement on the definitions of new elements. When agreement has been reached, corresponding examples need to be added to the new MNX by Example examples.
   • §6.4.1 The note element (accidentals) #184 (Unicode accidentals)
   • How to group parts? #185 (<staffGroup> and other layout elements)
   • MIDI #216 MIDI
   • Tick-based timing #217 Tick-based timing

[notator]

@adrianholovaty Your first TODO says

Migrate all stable parts of the spec into the doc system.

Merging PR #213 into the master branch now would stabilize the spec, and probably save some future work.

[adrianholovaty]

Status update: As of 028c8b9, the new docs system now has a Comparing MNX and MusicXML page, which is essentially a replacement for the old MNX By Example page.

The new page has some new features:

• In the MNX examples, each element and attribute is linked to the appropriate docs page.
• Each example document (see list) now deep-links to the appropriate part of the "Comparing MNX and MusicXML page" if the latter page contains that example.

It may be slightly confusing for each example to live in two places ("Comparing MNX and MusicXML" plus the examples section), but I think there's benefit in having all of the MusicXML comparisons on a single page, for easy, tutorial-style consumption by somebody new to MNX. I expect we'll tweak the way this works at some point, especially as the number of examples grows.

Remaining todos for this line of work:

• Add a way to specify the ordering of the examples on /comparisons/musicxml/. At the moment ordering is undefined.
• Implement the 'Show relevant section' vs 'Show full document'.

Once those last todos are done, I'll remove the old MNX By Example page and redirect it to the new page.

[adrianholovaty]

Status update: As of 8a02d35, the new docs system's Comparing MNX and MusicXML page now has the "Show relevant section" vs. "Show full document" logic. This is created dynamically; the system looks for a magic <metadiff> element within the examples to know which sections should be treated as "relevant." See the commit message for technical details.

Just one thing remaining before I can redirect the old By Example page to this new page: adding a way to specify ordering of the examples.

[samuelbradshaw]

This is really cool. Nice work!

[notator]

The following examples are either missing from the above list or need to be updated:

• Beamed grace notes (MNX by Example: Beamed grace notes #224)
• Mid-measure repeats (MNX by Example: Mid-measure repeats #222)
• Grand Staff with cross-staff beams (MNX by Example: Grand Staff with cross-staff beams #219)
• Jumps1 (dal segno) (MNX by Example: Jumps1 (dal Segno) #221)
• Jumps2 (dal segno al fine) (MNX by Example: Jumps 2 (dal Segno al Fine) #220 )
• Jumps3 (dal segno al coda) (MNX by Example: Jumps 3 (dal segno al Coda) #218)

As per my TODO list above, I have now created a new example showing how to group parts. According to this example, all the MNX examples need to be updated. See #185 (comment).

[adrianholovaty]

Status update: As of 7cbab68, "MNX By Example" has been fully replaced by the (essentially identical) version in the new docs system. See the commit message for more details. The old URL has been redirected to the new one.

[adrianholovaty]

This work was done a little while ago (see also #253).