Making Nix-Portable's runner script POSIX

[xplshn]

Hi! I like this project very much but it has a heavy dependency on Bash, and GNU coreutils, I have made this: https://github.com/xplshn/Handyscripts/blob/master/pelf

It packages ELFs + LDD visible libraries + Additional Libaries + Additional binaries + Additional arbitrary files as needed into a single POSIX SH script, the base64 encoding could be ditched and use m_ascii85 which doesn't bloat the file size up to 37% and just around 13%. Also one could simply ditch any encoding too. It works similarly to how it Nix-Portable works, but this one also adds two important things:

1. Bundles will remove from their dir the libraries which are found on the system. Reducing RAM usage. (happens at runtime)
2. Concurrent, if you have a program, let's say: Wezterm, packaged as a blob, from that terminal, you can call another instance (not the blob but the real terminal binary), without adding any overhead, also if you call the blob, it realizes that another instance is running and simply runs without unpacking new files nor modifying the ENV.
3. Very simple.
4. [experimental] If a user executes multiple different bundled programs, they can share some common libraries between them
5. Since its POSIX, one can use Dash, which makes execution a bit faster (with hyperfine it yielded good results in my +8 years old machine)

Testing?

Yes, I have done lots; For example, my window manager is a single file which contains Cava, Dmenu, MPV, ani-cli, yt-dlp, ytfzf, st, etc. All into a single file, all programs running inside the WM session will inherit the access to these programs.

]@ HELP_PAGE_LIST_PACKEDFILES=1 spectrwm.blob --pbundle_help
Description: Pack an ELF
Usage:
 <--pbundle_link <binary>|--pbundle_help> <args...>
EnvVars:
 USE_BULKLIBS=[0,1]
 USE_SYSTEM_LIBRARIES=[1,0]
 SHOW_DISCARDPROCESS=[0,1]
 HELP_PAGE_LIST_PACKEDFILES=[0,1]
/tmp/.pelfbundles/pbundle_spectrwm17146840120652_26952/bin:
ani-cli              dmenu_run            picom                screenshot.sh        yt-dlp
cava                 feh                  picom-trans          scrot                ytfzf
dmenu                i3lock               rofi                 spectrwm
dmenu_path           mpv                  rofi-theme-selector  stest

/tmp/.pelfbundles/pbundle_spectrwm17146840120652_26952/libs:
libImlib2.so.1                  libhwy.so.1                     libssh.so.4
libSDL2-2.0.so.0                libiniparser.so.1               libstartup-notification-1.so.0
libSPIRV-Tools-opt.so           libjack.so.0                    libswmhack.so.0.0
libSPIRV-Tools.so               libjxl.so.0.9                   libswresample.so.4
libSPIRV.so.12                  libjxl_cms.so.0.9               libswscale.so.7
libSvtAv1Enc.so.2               libjxl_threads.so.0.9           libtheoradec.so.1
libXcomposite.so.1              liblcms2.so.2                   libtheoraenc.so.1
libXft.so.2                     liblilv-0.so.0                  libuchardet.so.0
libXpresent.so.1                libluajit-5.1.so.2              libunibreak.so.6
libXv.so.1                      libmagic.so.1                   libv4l2.so.0
libaom.so.3                     libmbedcrypto.so.7              libv4lconvert.so.0
libass.so.9                     libmp3lame.so.0                 libva-drm.so.2
libasyncns.so.0                 libmpg123.so.0                  libva-wayland.so.2
libavcodec.so.60                libnuma.so.1                    libva-x11.so.2
libavdevice.so.60               libopenmpt.so.0                 libva.so.2
libavfilter.so.9                libpam.so.0                     libvdpau.so.1
libavformat.so.60               libpango-1.0.so.0               libvidstab.so.1.2
libavutil.so.58                 libpangocairo-1.0.so.0          libvpl.so.2
libbluray.so.2                  libpangoft2-1.0.so.0            libvpx.so.9
libcaca.so.0                    libpcre.so.1                    libvulkan.so.1
libcairo.so.2                   libplacebo.so.338               libwayland-cursor.so.0
libcdio.so.19                   libpostproc.so.57               libwayland-egl.so.1
libcdio_cdda.so.2               libpulse-simple.so.0            libwebpmux.so.3
libcdio_paranoia.so.2           libpulse.so.0                   libx264.so.164
libcjson.so.1                   libpulsecommon-17.0.so          libx265.so.199
libconfig.so.11                 librav1e.so.0.7                 libxcb-cursor.so.0
libdav1d.so.7                   librist.so.4                    libxcb-ewmh.so.2
libdovi.so.3                    librubberband.so.2              libxcb-icccm.so.4
libdvdcss.so.2                  libsamplerate.so.0              libxcb-image.so.0
libdvdnav.so.4                  libserd-0.so.0                  libxcb-keysyms.so.1
libdvdread.so.8                 libshaderc_shared.so.1          libxcb-render-util.so.0
libev.so.4                      libsndio.so.7                   libxcb-xrm.so.0
libfftw3.so.3                   libsodium.so.26                 libxkbcommon-x11.so.0
libfribidi.so.0                 libsord-0.so.0                  libxvidcore.so.4
libgomp.so.1                    libsoxr.so.0                    libzimg.so.2
libgraphite2.so.3               libsratom-0.so.0                libzix-0.so.0
libharfbuzz.so.0                libsrt.so.1.5                   libzmq.so.5
]@

This is how my env looks inside said WM, using Wezterm.blob

spectrwm_libs=/tmp/.pelfbundles/pbundle_spectrwm17146840120652_26952/libs
wezterm_libs=/tmp/.pelfbundles/pbundle_wezterm17146913370857_31914/libs
spectrwm_bindir=/tmp/.pelfbundles/pbundle_spectrwm17146840120652_26952/bin
wezterm_bindir=/tmp/.pelfbundles/pbundle_wezterm17146913370857_31914/bin
PELF_LIBDIRS=/tmp/.pelfbundles/pbundle_spectrwm17146840120652_26952/libs:/tmp/.pelfbundles/pbundle_wezterm17146913370857_31914/libs
PELF_BINDIRS=/tmp/.pelfbundles/pbundle_spectrwm17146840120652_26952/bin:/tmp/.pelfbundles/pbundle_wezterm17146913370857_31914/bin

This is how the PATH looks inside said programs:

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/anto/.local/bin:/home/anto/.local/share/flatpak/exports/bin:/usr/local/go/bin:/usr/local/nim/bin:/usr/local/rust/bin:/usr/local/zig/bin:/tmp/.pelfbundles/pbundle_spectrwm17146840120652_26952/bin:/tmp/.pelfbundles/pbundle_wezterm17146913370857_31914/bin

There is also a way to call a program that is outside a blob/bundle, but giving it access to all libraries and programs in a program.blob:
So for example, someone may pack FFMPEG and call a script which uses ffplay with:
ffmpeg.blob --pbundle_link script_that_uses_ffmpegPrograms.sh

And I have made this other separate program/script which allows for further customization and also having other bundles/programs inherit access to programs inside other's:

]@ pelf_linker
PELF_BINDIRS="/tmp/.pelfbundles/pbundle_spectrwm17146840120652_26952/bin:/tmp/.pelfbundles/pbundle_wezterm17146913370857_31914/bin"
PELF_LIBDIRS="/tmp/.pelfbundles/pbundle_spectrwm17146840120652_26952/libs:/tmp/.pelfbundles/pbundle_wezterm17146913370857_31914/libs"
]@

If any of you good people want a demo of some graphical programs bundled, like for example; chocolate-doom with an included freedoom wad, I can provide those. It is also very straightforward to make wrapper scripts to do things like: A development environment packed in a single file, which for example, I made a Go + Make + some other programs bundle that included all of Go's files necessary for development, you executed the script and it simply called SH after setting a new PATH, LD_LIBRARY_PATH and the variables which PELF uses for concurrency.

Thanks for reading, I hope to see this here, since it would make nix-portable be even more portable.

NOTE: Bundles can't ship the LD loader nor the C library. They can be modified however and use patchelf to self-modify everything that depends on libraries which can't be put in another LD_LIBRARY_PATH

[DavHau]

I won't invest my time into this, but feel free to open a PR if you want to remove the dependency on bash.