Build everything in mods/ and include it with the .app

[Habbie]

For the out-of-box experience.

[Habbie]

Hydra did not have .so files for the included modules, instead it linked the .m bits right into the Hydra binary. This provides some virtual memory pressure (the executable pages become demand paged) which is uninteresting; it causes some static allocation/initialisation (on the order of kilobytes I bet); and in theory it could cause code to get run on startup instead of on require, but this is something we can control when accepting modules.

If we are strict enough about the quality of modules we import, linking them all into our binary is an option. However, I don't see many benefits to this.

I also see no downsides to shipping .so files with all modules that have .m files, keeping memory entirely free of unused modules.

Opinions?

[cmsj]

Hi

I say let's let the data decide, by measuring the difference between implicitly loading all the modules and not.

Cheers,

Chris Jones

On 10 Oct 2014, at 18:03, Peter van Dijk notifications@github.com wrote:

Hydra did not have .so files for the included modules, instead it linked the .m bits right into the Hydra binary. This provides some virtual memory pressure (the executable pages become demand paged) which is uninteresting; it causes some static allocation/initialisation (on the order of kilobytes I bet); and in theory it could cause code to get run on startup instead of on require, but this is something we can control when accepting modules.

If we are strict enough about the quality of modules we import, linking them all into our binary is an option. However, I don't see many benefits to this.

I also see no downsides to shipping .so files with all modules that have .m files, keeping memory entirely free of unused modules.

Opinions?

—
Reply to this email directly or view it on GitHub.

[Habbie]

👍

[Habbie]

TLDR static linking all .m files will cause 0.5M of VM pressure and allocate a few hundred bytes.

With all modules available as rocks installed:

tesla:mjolnir peter$ pwd
/usr/local/lib/lua/5.2/mjolnir
tesla:mjolnir peter$ du -hs .
644K    .
tesla:mjolnir peter$ find . -type f | xargs cat | wc -c
  577072
tesla:mjolnir peter$ find . -type f -name '*.so' | xargs -n 1 size -m | grep bss | cut -f2 -d: | awk '{x=x+$1} END {print x}'
132

i.e. loading every .so will cause around 600kbyte of virtual mapping, and 132 bytes of static allocation.

tesla:mjolnir peter$ pwd
/usr/local/Cellar/luarocks/2.1.2_1/share/lua/5.2/mjolnir
tesla:mjolnir peter$ du -hs .
232K    .
tesla:mjolnir peter$ find . -type f | xargs cat | wc -c
  131243

furthermore, there is around 130kbyte of Lua source in all of the rocks combined (82kb of which is from _asm ;). A very generous estimate would be that on load, due to compilation, this turns into one megabyte -- but note that the topic is not 'preload all .lua?' but 'prelink all .so'.

[Habbie]

However, static linking makes it harder for third party devs/hackers to override things from within their custom versions of modules, forcing them to put their author prefix not just in the package name but also the name of their meta table, in theory. I think this would go alright in practice, the way the modules currently look, but I feel .so will give more flexibility.