Proposal to use a document loader

[OptimisticPeach]

Description

Writing the dom code for a document is long and kind of hard to follow. It is nicely formatted, but it is rather cumbersome to write. And so I created a document loader which does the following:

• Load the text
• Tokenize text
• Parse the tokens
• Provide texts, fonts and sizes needed to be cached
• Create the Dom<T> from the abstract syntax tree emitted by the parser

Describe your solution

A document loader. Kind of like an HTML document loader, but with a less involved syntax (We don't need to be able to use javascript or css in the file).
Here an example of the syntax my loader currently uses:

{this is a comment}
div:wrapper[
    label:red ("Hello123");
    div:sub_wrapper[
        div:yellow;
        div:grey;
    ]
    text:pink ("Lorem Ipsum dolor sit amet" "Roboto" 10);
    image:cat ("Cat01");
]

Are there alternatives or drawbacks?

An alternative solution is to create a different form to define documents. My loader may have some drawbacks in the future for supporting widgets/new features of Azul, not to mention I'm no language specialist, but it may be a start.

Is this a breaking change

All of the current code would still work. This could be a separate module or project completely.

Additional notes

The repo containing the code is available here

[fschutt]

Well, it's essentially a new markup language - I already thought about this, but intended to do this with XML, not implement my custom non-standard parser. Azul already understands "classes" and "ids", just like HTML, which many people are already familiar with. The CSS can already be hot-reloaded, so the document shouldn't contain any styling information, just content. Then I wanted to write a XML-to-Rust translator that can take the XML and spit out the equivalent Rust code, so you can "compile" these XML documents to standard Rust when you're done with the quick iteration.

The thing is that an XML document maps 1:1 to a DOM tree. I want to avoid making my own markup language if possible, because it's yet another learning overhead - and for what, XML parsers are plenty, and XML is widely used and people are familiar with it.

So the code in your example would become:

<!-- This is a comment -->
<div id="wrapper">
    <p id="red">"Hello123"</p>
    <div id="sub_wrapper">
        <div id="yellow"></div>
        <div id="grey"></div>
    </div>
    <p id="pink">"Lorem Ipsum dolor sit amet"</div>
    <img src="Cat01" />
</div>

I've also thought about how you'd render custom components, like the spreadsheet. For this I've thought about a trait that "reacts" to a hashmap of key-value pairs, then outputs a Dom<T>. Ex.:

impl LoadFromXml for Spreadsheet {

    fn component_name() -> &'static str { 
        "spreadsheet"
    }
    
    fn react_to_key_values(kv: &HashMap<String, String>) -> Result<Dom<T>, SyntaxError> {
         let columns = kv.get("cols").and_then(|v| v.parse::<usize>().ok()).unwrap_or(0);
         let rows = kv.get("rows").and_then(|v| v.parse::<usize>().ok()).unwrap_or(0);
         Ok(Spreadsheet::new(columns, rows).dom())
    }

   // Called by the XML-to-Rust compiler
   fn compile_to_rust_code(kv: &HashMap<String, String>) -> String {
        let columns = kv.get("cols").and_then(|v| v.parse::<usize>().ok()).unwrap_or(0);
        let rows = kv.get("rows").and_then(|v| v.parse::<usize>().ok()).unwrap_or(0);
        format!("Spreadsheet::new({}, {}).dom()", columns, rows)
   }
}

Then you could "inject" this component as a Box<LoadFromXml> into the App<T> and then hot-reload it from XML as well as compile it to Rust code in the end:

<div id="my_wrapper">
    <spreadsheet rows="4" cols="10" />
</div>

... which then could be compile to Rust when you're done with this component:

$ azul_to_rust my_ui.xml > my_ui.rs
$ cat my_ui.rs

Dom::new(NodeType::Div).with_id("my_wrapper")
    .with_child(Spreadsheet::new(4, 10).dom())

I see little reason why a custom markup language is needed for this instead of just using XML / HTML. The styling information has to be in a seperate CSS file anyways (ok, you could "merge" them together with your custom document loader, but eh...). Your project is certainly cool and I don't want to stop you from developing it, but I wouldn't include this in Azul itself, rather keep it as a third-party plugin (if it turns out that people really do want to use your markup language over XML).

[OptimisticPeach]

Oh! Okay then, perhaps I should move my attention to making a plugin that could use an xml parser and emit a dom tree.
Thanks for the review and amazing attention to detail!

[rask]

With HTML parsing I could take my desktop GUI and with some small changes make a web application layout from it, while sharing the same stylesheet. Is this HTML/XML parsing already available or still under consideration/implementation? Did not see any references in the documentation.

[fschutt]

No, it's not implemented yet. It's just an idea, because there is already an XML build dependency anyways (necessary for generating opengl bindings). And yes, the plan was to be HTML-friendly, so that people can port existing HTML applications easily. I'll reopen this as a tracking issue so it doesn't get lost.

[rask]

Neat, will be following for news on this. :)

[GunpowderGuy]

@fschutt is a xml to rust compiler necessary ? wouldnt it be possible to parse xml at compile time in release builds ?

[fschutt]

@diegor8 Cargo does not allow you to modify the source directory in a build.rs file. So yes, you'd need to compile it manually in a separate step. Besides, how would you do variables, callbacks, for loops in XML? Again, XML would just be for prototyping.

[GunpowderGuy]

@fschutt i was talking about compile time function execution , compile time serialization of xml documents , not plugging a transpiler into rustc

[fschutt]

@diegor8 Yes, and? You cannot generate compilable Rust code from XML without a separate compliation step. As to your question, no it's not possible. A const fn can only call other const fns, not regular functions. Building a DOM tree needs dynamic allocation (since integer generics are missing from Rust), and dynamic allocation isn't allowed inside of const fns.

And you'd still not be able to do for loops in XML, even with const fn, you need to translate the XML to Rust first and then you'd often need make manual adjustments. That means you have to modify the build directory before the compilation actually starts - which isn't allowed by a build script. The goal of a document loader is for enabling hot-reloading, not ease-of-use.

[GunpowderGuy]

which then could be compile to Rust when you're done with this component:

made me think users could be expected to use the results of this without modifying them . Are there some parts of the ui that could be used as is after being generated by the document loader ?

[fschutt]

Closing this because this is more or less what the XML loader now does: see the /examples/ui.xml for an example of the syntax:

<component name="Invoice" args="dueDate: String">
    <p>Your invoice is due at {dueDate}</p>
</component>

<app>
    <Invoice dueDate="02.03.2019" />
</app>

In the future, you will be able to compile that XML code to Rust functions (to have both a performant and type-safe prototyping API), but for now you can use Dom::from_str() and Dom::from_file("my_file.ui.xml") to load DOM components from a file.