mksquashfs is killed by the system's oomkiller

[lalten]

We're building AppImages with Bazel in our CI (via rules_appimage). When the appimage'd program is large and the CI worker is under high memory pressure, the appimagekit invocation fails because mksquashfs is killed by the system's oomkiller.

ERROR: /code/path/to/package/BUILD:4:11: AppImage path/to/package/program.appimage failed: (Killed): tool failed: error executing command

(Killed) is an oom killer symptom. The reason is that mksquashfs seems to try and use all the available memory. See also https://gitlab.tails.boum.org/tails/tails/-/issues/16177.
mksquashfs supports a -mem SIZE argument: https://www.mankier.com/1/mksquashfs#-mem, which would be a great solution to this, but this isn't currently exposed in AppImageKit in

AppImageKit/src/appimagetoolnoglib.c

Line 158 in 8bbf694

execlp("mksquashfs", "mksquashfs", source, destination, "-root-owned", "-noappend", (char *)0);

or

AppImageKit/src/appimagetool.c

Lines 150 to 214 in 672b3d0

	args[i++] = "mksquashfs";
	#else
	args[i++] = pathToMksquashfs;
	#endif
	args[i++] = source;
	args[i++] = destination;
	args[i++] = "-offset";
	args[i++] = offset_string;
	args[i++] = "-comp";
	if (use_xz)
	args[i++] = "xz";
	else
	args[i++] = sqfs_comp;
	args[i++] = "-root-owned";
	args[i++] = "-noappend";
	if (use_xz) {
	// https://jonathancarter.org/2015/04/06/squashfs-performance-testing/ says:
	// improved performance by using a 16384 block size with a sacrifice of around 3% more squashfs image space
	args[i++] = "-Xdict-size";
	args[i++] = "100%";
	args[i++] = "-b";
	args[i++] = "16384";
	}
	// check if ignore file exists and use it if possible
	if (access(APPIMAGEIGNORE, F_OK) >= 0) {
	printf("Including %s", APPIMAGEIGNORE);
	args[i++] = "-wildcards";
	args[i++] = "-ef";
	// avoid warning: assignment discards ‘const’ qualifier
	char* buf = strdup(APPIMAGEIGNORE);
	args[i++] = buf;
	}
	// if an exclude file has been passed on the command line, should be used, too
	if (exclude_file != 0 && strlen(exclude_file) > 0) {
	if (access(exclude_file, F_OK) < 0) {
	printf("WARNING: exclude file %s not found!", exclude_file);
	return -1;
	}
	args[i++] = "-wildcards";
	args[i++] = "-ef";
	args[i++] = exclude_file;
	}
	args[i++] = "-mkfs-time";
	args[i++] = "0";
	args[i++] = 0;
	if (verbose) {
	printf("mksquashfs commandline: ");
	for (char** t = args; *t != 0; t++) {
	printf("%s ", *t);
	}
	printf("\n");
	}
	#ifndef AUXILIARY_FILES_DESTINATION
	execvp("mksquashfs", args);

It would be awesome if a new AppImageKit parameter could be made to pass this through to mksquashfs. What do you think?

[TheAssassin]

A better solution is to just allow users to add mksquashfs parameters, e.g., --mksquashfs-param -mem --mksquashfs-param 1024.