Convert backend to TypeScript

[eliandoran]

Discussed in https://github.com/orgs/TriliumNext/discussions/4

^{Originally posted by eliandoran February 14, 2024}
This is a bit of a reply to some of the replies in #4646 regarding type management, but it's actually something I've thought for some time now.

I don't think rewriting the server to Rust is currently a good candidate due to the sheer difficulty of implementing such a change. Nevertheless, we can consider for the future an approach I saw in NextCloud: rewrite in Rust (or other high-performance language) only the most performance-sensitive stuff. In our case it would be most likely the search and tree functionality.

To me, adding types to Trilium is almost mandatory. And the sole reason to that is that types makes it easier to navigate through a repository where you don't know everything about it. I've worked with JavaScript projects before and as soon as the application grows in complexity to anymore more than trivial, it becomes hard to navigate through it once you start forgetting all the details about the implementation.

When it comes to adding types, there are really only two possibilities that would make sense: JSDoc and TypeScript. For me, I'm not very fond of JSDoc since I found it to be a bit unwieldy compared to TypeScript. Also important is that TypeScript introduces some safety features that are really helpful: for example checks for nullish values, and of course any type inconsistencies.

Feel free to pitch in if you have any ideas or if you agree or disagree to this.

cc. @rom1dep @rauenzi @meichthys

[deajan]

@eliandoran So I've a got a (perhaps silly) question about this.
Why not go use webassembly, so the app can run near native speed in most browsers and most platforms ?
Forgive me if the question is out of scope, since I'm not a frontend dev, it's more curiosity than anything else that pushes me to ask ;)

[zerebos]

I believe to this day WebAssembly does not have full direct DOM access as it's really a sandboxed virtual environment and not a new way of running JS or an API. Though some DOM access does exist the main way is to have the WASM call out to the JS to perform DOM operations which are generally the heaviest tasks for a simple frontend anyways.

And even if WebAssembly did gain full DOM access since I last checked, I doubt any of the frontend frameworks are ready to compile to it just yet 😄

[deajan]

@rauenzi Thanks ^^

[rom1dep]

Does the TypeScript compiler even compile to WASM yet?

[eliandoran]

@rauenzi , @deajan ,

First we have to consider that we are currently converting the backend, not the frontend. WebAssembly for front-end is kind of a no go exactly due to what @rauenzi mentioned regarding DOM access.

For the backend we don't need DOM access, so theoretically it should be OK. But I have no experience with WASM so I'm not sure if this is achievable.

Regarding the performance of the server, we need to identify exactly where the performance issues lie. I'm pretty convinced that TypeScript running on Node (using V8) is performant enough, we just need to tweak the algorithms, add some caching etc. If this is not the case, we can rewrite some portions of it to use a native language such as Rust, however this could cause some issues.

[ProfDoof]

One benefit I do see of RIIR (yes, it's a meme) is that we could distribute a straight binary for the server that people could just run. Obviously making it friendly with Mac and all that isn't ideal but that's already not exactly ideal. Then again there's probably more solutions for that I don't know about for electron style apps.

[eliandoran]

@ProfDoof , are you proficient in Rust? Maybe you could try doing a proof of concept to see how it would fit our usecase.

[zerebos]

I think a Rust or Go (I'm more proficient in Go than Rust) backend even if it's just as a sync-only alternative for servers would be cool to have, but not super high priority imo.

[ProfDoof]

I would be willing to try to work on it over the summer depending on how busy my dissertation work has me. Or if you have a particular PoC you want to see, I could try to spin up a smaller PoC as I have free time over the next month or two.

[sigaloid]

I am also interested in working on it but similarly have unstable availability.

I think the hardest part is finding exactly what parts of the code handle the syncing - so much of it calls into other parts of the code base, which makes it very hard to find a place to start.

[ProfDoof]

Just reading this, I'm starting to get a feel that the very first step should likely be going through and adding types and extra documentation to what already exists to make it easier to explore and discuss next steps. If we make a branch for that, I'm willing to throw some time at it when I have free time to get to the point we can break things up more cleanly.