Early Development discussions

[FBumann]

This serves as documentation of decisions made during early development

[FBumann]

@brynpickering Any motivation on your site to work on this? Or is it not relevant for you atm?

[FBumann]

After a lot of back and forth, benchmarking and thinking about the importance of memory usage vs speed, I decided to go for polars as the engine instead of duckdb (#189). benchmarks.html

I think the memory usage of polars is low enough, and will probably shrink as polars is actively pushing boundaries. Duckdb has the ability to write >200 Mio. Variable Models at <2 GB, which is impressive, but probably not needed ba anyone. Models of this size will probably always be solved on machines with >32 GB of Memory, or solved incrementally.
In both cases, polars wins in speed & Readability.

Ill continue with this wrking hypothesis, but #189 is contained enough to revert this decision if needed.