Better example files for development

[mkoskim]

It could be a good idea to get some copyright free stories and convert them to MaweJS format. This way you could use actual stories as a test bench when developing features.

There is at least Project Gutenberg: https://www.gutenberg.org/

The best possibility would find these stories as plain ASCII or very close to it (Project Gutenberg offers them as plain UTF-8). Currently, "importing" stories to MaweJS works best by opening a text with suitable editor, press Ctrl-A to select all, and copy-paste it to MaweJS. Splitting the pasted text to scenes and such is manual work.

One thing that these examples would miss, are metadata: synopses, missing and comment blocks. We might need artificially add these, if we like to use these for testing metadata features.

Update: Need to take some closer look what Project Gutenberg has. As a speculative fiction writer, there might be some classics like Frankenstein, Dracula or something.

[mkoskim]

Done this. examples/ folder now includes Frankenstein as .mawe file. Some other classics could be also converted, although before that, it would be good to take a look for issues raised when converting Frankenstein.

[mkoskim]

Yes, I really should make better example file.

As far as I can see, a document is made up of parts and scenes, with scenes also functioning as chapters?

Generally, yes, but I use name chapter in exporting, and there you can choose, if scenes or parts are exported as chapters. Short stories can use scenes as chapters, longer ones use parts.

Part is bad name for what it really is. It is a group of scenes, so I should rename it at some point. It just sounds too funny to write e.g. body.parts.map(...) so I haven't have any hurry to do renaming :D

General document structure at the moment is following:

[story]
  [head] ... header elements like title, subtitle and so on ... [/head]
  [export] ... export settings ... [/export]
  [body]
    [part name="XXX"]
      [scene name="YYY"]
        [p] ... paragraphs: p, br, comment, synopsis, missing, fill, tag [/p]
        [p] ... [/p]
        [p] ... [/p]
        [br/] // empty paragraph: for CSS purposes
      [/scene]
      [scene] ... [/scene]
      [scene] ... [/scene]
      [scene] ... [/scene]
      ...
    [/part]
    [part] ... [/part]
    [part] ... [/part]
    ...
  [/body]
  [notes]
    // Similar to body: body and notes - which are list of parts - are called sections internally.
    // Future: maybe multi-section editor.
  [/notes]
  [ui] ... UI settings ... [/ui]
  [history]
    // "History" entries to track progression (new feature)
    [words date="YYYY-MM-DD" text: XX missing: XX chars: XX[/words]
  [/history]
[/story]

Many names are such as they are just for historical reasons.

[mkoskim]

BTW, one file that close always contains all the editor features, is "User's Guide":

https://github.com/mkoskim/mawejs/blob/master/examples/UserGuide.mawe

[peter88213]

Well, understood. Anyway, if the document structure depends on the application's export settings, it cannot be reflected properly in the file format, can it?

Concerning conversion of file formats, fixed conventions are advantageous. This is the document hierarchy as defined for novelibre:
https://peter88213.github.io/nvhelp-en/basic_concepts.html#the-book-hierarchy

This can be easily mapped to the novelWriter structure, for example:
https://novelwriter.io/docs/project_structure.html

• A novelWriter "scene" (3th level) is converted to a novelibre section.
• A novelWriter "section" (4th level) is converted to an "appended" novelibre section.

body and notes - which are list of parts - are called sections internally.

If the file format is to be open, it is always advisable to use the wording of the application.
Have you considered versioning your file format? This can help when renaming elements during the development phase.

[mkoskim]

Well, understood. Anyway, if the document structure depends on the application's export settings, it cannot be reflected properly in the file format, can it?

Oh, no, no, the file format remains the same :D I just talked why I don't call scene groups chapters, and that's because I use word "chapter" meaning the separated (separator like "* * *", numbered or named) elements in the exported draft :) When exporting, you can choose if you want to make groups or scenes to draft chapters.

Concerning conversion of file formats, fixed conventions are advantageous. This is the document hierarchy as defined for novelibre: https://peter88213.github.io/nvhelp-en/basic_concepts.html#the-book-hierarchy

Yes, they are. Like said, many internal names are such because of history. Parts became parts, because I didn't want to call them groups, because I called arbitrary deep tree elements to groups in my first editor, moe :)

I am not sure, if I like to use book hierarchy as internal names. My basic idea has been, that a story is just well organized sequence of scenes, where a scene is defined so that it is text block that makes no sense, if it is splitted and separated. Everything above scenes are just helping the writer - like me - to craft that scene sequence.

Then, when exporting draft, you start to think about chapter breaks and such. The reason for this choice is that some writers write really large scenes and some write really small scenes. Exporting a draft is different if the story is a novellette to a writing competition, or if it is novel manuscript you like to send to a publisher.

This can be easily mapped to the novelWriter structure, for example: https://novelwriter.io/docs/project_structure.html

• A novelWriter "scene" (3th level) is converted to a novelibre section.
• A novelWriter "section" (4th level) is converted to an "appended" novelibre section.

Yes, I need to take a look to those.

body and notes - which are list of parts - are called sections internally.

If the file format is to be open, it is always advisable to use the wording of the application. Have you considered versioning your file format? This can help when renaming elements during the development phase.

Yes, I have considered versioning file format, instead I have already two versions. Version 1 is old, and do not support part elements (just scenes), and version 2 supports them.

But TBH the editor project is still somewhat early phase at least if we look it from file format point of view, and I need some time to sketch how to implement certain things so that they fit nicely to overall paradigm.

---

The following are more processing my thoughts than an answer, just freeform monology :)

The definitely most difficult thing for me is that most structured editors really evolve around somewhat fixed chapter-scene -tree. What that means is that they like to store metadata to the scene and chapter elements. My old Moe worked just like that.

Now, in MaweJS scenes and chapters (groups) are anything but fixed, because you can merge scenes together by removing the scene break. Scene and group breaks operate the same way as headers in Word / GoogleDocs / LibreOffice, and it just do not make any sense to put too much metadata into them. In practise, scene & group breaks are just breaks, they are used to split text at certain points to make them freely movable.

That's why all the metadata in current MaweJS is describing the text itself, not a scene. When I add a synopsis, it tells what happens in the next few paragraphs below the line. I can't store that info to scene node: e.g. synopsis line needs to have its "anchor" in the text. But I can collect all the blocks in the scene together, to get information about the scene.

Example screenshot (from User's Guide):

In this screenshot, you can see comment blocks (green), scene breaks (grey) and a filler element (magenta). Also, I enabled them in the index, so there you can see them there, too.

What I wanted is that I could give scene a "target word count". What I ended up, was filler elements. Fillers remain in the text even if you merge or split scenes above. So, to make me that "target word count", I need to make an editor view to edit fillers (I have planned to do that in Story Arc diagram view). So, not all things are as straight-forward as they are with more conventional structured editors, at least not now.

One thing to keep in mind is that if I want something to be editable, it needs to be a text block because I use SlateJS. I have tried its "void" element types (which are mainly used to insert images to text), but it can be really messy, if you want it to have editable content.

[mkoskim]

Hi!

After this discussion, I made some changes. First, "part" is now "chapter", so the story body format is list of chapters, which are list of scenes. See:

#233

I also made changes to export settings according to this change. Earlier, you chose what element is regarded as a chapter, and how to format it. Now you choose formatting to chapters & scenes.

I updated file format page: #105

Files really have formats, and for that purpose I have a migration module:

https://github.com/mkoskim/mawejs/blob/master/src/document/xmljs/migration.js

I also looked more closely novelWriter, and implemented unnumbered chapters:

#234

So, big thanks for your input, they lead to several improvements!