Running flatpak bundle-build on a bundle that is very large ( > 2GB) generates the following error:
(/usr/bin/flatpak build-bundle:28630): GLib-ERROR **: /usr/src/packages/BUILD/glib2.0-2.50.2+dev22.0cb2266/./glib/gmem.c:165: failed to allocate 4294967296 bytes
Running the same command under gdb produces this backtrace:
#0 0x00007ffff6d243c1 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#1 0x00007ffff6d253ed in g_log_default_handler ()
#2 0x00007ffff6d256d1 in g_logv () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#3 0x00007ffff6d2591f in g_log () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#4 0x00007ffff6d24046 in g_realloc ()
#5 0x00007ffff6d40247 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0
#6 0x00007ffff6d4048f in g_string_set_size ()
#7 0x00007ffff78f7436 in ?? () from /lib/x86_64-linux-gnu/libostree-1.so.1
#8 0x00007ffff78f7a35 in ?? () from /lib/x86_64-linux-gnu/libostree-1.so.1
#9 0x00007ffff78f9e91 in ostree_repo_static_delta_generate ()
#10 0x000055555557848f in build_bundle (repo=0x1, repo@entry=0x555555805000,
name=0x1 <error: Cannot access memory at address 0x1>,
full_branch=0x5555557fef50 "PQ", cancellable=0x7fffe0000ae0,
#11 0x00005555555799a2 in flatpak_builtin_build_bundle (argc=5,
argv=0x7fffffffdf98, cancellable=0x0, error=0x7fffffffde28)
#12 0x000055555556a772 in flatpak_run (argc=5, argv=0x7fffffffdf98,
res_error=0x7fffffffde78) at app/flatpak-main.c:366
#13 0x0000555555569e92 in main (argc=6, argv=0x7fffffffdf98)
Its unclear why you can't allocate a 4 gigabyte chunk, how much memory do you have?
However, its clearly a problem that we're building up the entire bundle in memory before saving it.
@cgwalters: Did you ever look at doing streaming builds of static deltas?
I was running on a VM, to which I had assigned 4GB of memory. I just upped that to 8GB and tried again, and got the same error (with same backtrace), though, curiously, this time it fails trying to allocate 2GB.
(/usr/bin/flatpak build-bundle:1573): GLib-ERROR **: /usr/src/packages/BUILD/glib2.0-2.50.2+dev22.0cb2266/./glib/gmem.c:165: failed to allocate 2147483648 bytes
Potentially, it's a bit tricky as to stream we'd have to move away from GVariant, or at least the standard GVariant APIs.