Unable to build various complex project on-device due to various Operation not permitted errors.

[deatondg]

I am not savvy enough to understand exactly what is going wrong here, so please let me know if I should submit this issue to the Odyssey repo or some other repo instead.

I am unable to build various complex projects on-device (iPad Pro 4 iOS 13.5 Odyssey 1.2.2) due to various Operation not permitted errors. I had the same issue on Unc0ver using Sam Binger’s tools, and it was acknowledged as a bug on StackOverflow, but never fixed. I’ve recently switched to Odyssey and your tools.

The errors are all something like this: building make using

git clone https://git.savannah.gnu.org/git/make.git
cd make
./bootstrap

fails with

autoreconf: running: aclocal -I m4 --force -I m4
Can't exec "aclocal": Operation not permitted at /usr/share/autoconf/Autom4te/FileUtils.pm line 326.
autoreconf: failed to run aclocal: Operation not permitted
./bootstrap: autoreconf failed

Building texlive using

git clone https://github.com/TeX-Live/texlive-source
cd texlive-source

fails with

/home/mobile/Developer/sources/texlive-source/libs/freetype2/freetype-src/builds/unix/configure  '--disable-shared' '--without-bzip2' '--without-zlib' '--without-png' '--without-harfbuzz' '--prefix=/home/mobile/Developer/sources/texlive-source/Work/libs/freetype2/ft-install' '--libdir=/home/mobile/Developer/sources/texlive-source/Work/libs/freetype2' '--includedir=/home/mobile/Developer/sources/texlive-source/Work/libs/freetype2'
make[4]: /home/mobile/Developer/sources/texlive-source/libs/freetype2/freetype-src/builds/unix/configure: Operation not permitted
make[4]: *** [/home/mobile/Developer/sources/texlive-source/libs/freetype2/freetype-src/builds/unix/detect.mk:89: setup] Error 127make[4]: Leaving directory '/home/mobile/Developer/sources/texlive-source/Work/libs/freetype2/ft-build'make[3]: *** [Makefile:606: ft-config] Error 2
make[3]: Leaving directory '/home/mobile/Developer/sources/texlive-source/Work/libs/freetype2'make[2]: *** [Makefile:918: recurse] Error 1
make[2]: Leaving directory '/home/mobile/Developer/sources/texlive-source/Work/libs'
make[1]: *** [Makefile:488: all-recursive] Error 1make[1]: Leaving directory '/home/mobile/Developer/sources/texlive-source/Work/libs'make: *** [Makefile:582: all-recursive] Error 1

Importantly though, changing like 89 of detect.mk from

                $(TOP_DIR)/builds/unix/configure $(value CFG)

to

                bash $(TOP_DIR)/builds/unix/configure $(value CFG)

fixes this error, and the build progresses to a new one:

make[4]: /home/mobile/Developer/sources/texlive-source/Work/libs/freetype2/ft-build/libtool: Operation not permitted
make[4]: *** [config.mk:55: /home/mobile/Developer/sources/texlive-source/Work/libs/freetype2/ft-build/libfreetype.la] Error 127
make[4]: Leaving directory '/home/mobile/Developer/sources/texlive-source/Work/libs/freetype2/ft-build'

which can be alleviated in a similar way. Every time there is an error, the solution is just to prefix the line which errors with bash.

I can just keep doing this until the build succeeds, but I’d rather not do that.

I have installed all the required build tools, I am logged in a mobile, I am not building from /var/mobile/, and my environment variables are configured like so.

export SDK_ROOT="/usr/share/SDKs/iPhoneOS.sdk"
export ARCH=arm64

export CC="clang-10"
export CXX="clang++-10"
export CFLAGS="-isysroot $SDK_ROOT"
export CXXFLAGS=$CFLAGS
export CPPFLAGS=$CFLAGS
export LDFLAGS="-isysroot $SDK_ROOT"
export CCASFLAGS="-isysroot $SDK_ROOT"

export CCexe_CFLAGS=$CFLAGS
export CCexe_LDFLAGS=$LDFLAGS

Let me know if there’s anything I can do to help fix this, or if there’s any other info you need from me. I greatly appreciate your help.

[Diatrus]

Currently the easiest fix I can offer is as follows. Do you have access to a checkra1n-able device? If so, use Odysseyra1n with that. This is due to a limitation in kppless jailbreaks (most app-based jailbreaks) and would likely need a major workaround to fix. This isn’t necessarily a problem with Procursus, but a problem with kppless jailbreaks in general.

[deatondg]

Gotcha. Unfortunately, I do not have a checkra1n-able device.

I’d love to get native development working on my iPad, so I will pursue other avenues like iSH or UTM or something where this issue can’t get in my way. It really is a shame Apple hinders their devices like this.

Should I file a bug on the Odyssey repo so that the other Odyssey dev’s know at least one user is hitting this issue, or should I not bother?

[Diatrus]

The Odyssey dev knows. It’s been a pain in everyone’s ass for a very, very long time. I’m looking into a workaround, just slowly. I have too much on my plate as a 19 year old.

[deatondg]

Good to know. Thanks for your help!

I hope you’re safe and healthy and that your workload diminishes with age. :)

[Diatrus]

Heh, thank you. I’m absolutely safe and healthy, just do too much. Staying in shape, learning an instrument, Procursus, university, and money.

[demhademha]

Could you join Hayden's discord so that I can try to help you?

[mwoolweaver]

Could you join Hayden's discord so that I can try to help you?

what's the link?

[CRKatri]

https://diatr.us/discord

[deatondg]

This morning I decided to try again on a fix for this. After reading Apple’s source for posix_spawn, I ended up with the %hookf that I’ve attached at the end of this. Because the exec syscalls are implemented in terms of posix_spawn, this fixes all sorts of exec-related issues.

The tweak somewhat solves my build problems, but to use this tweak for command line apps, I have to export DYLD_INSERT_LIBRARIES=/usr/lib/TweakInject.dylib. After I do that, this line still doesn’t show up in env which I imagine is related to the dyld-environment-variable-ignoring that is present on macOS.

Unfortunately though, this environment variable seems not to stick: even if make gets the hook, the programs that make spawns don’t have this environment variable and so if they go out and spawn a script, it still fails.

Any advice here? What should I do to make sure this hook/other tweaks are successfully loaded for command line apps?

#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <spawn.h>

// Copied from bsd/kern/kern_exec.c
#define IS_WHITESPACE(ch) ((ch == ' ') || (ch == '\t'))
#define IS_EOL(ch) ((ch == '#') || (ch == '\n'))
// Copied from bsd/sys/imgact.h
#define IMG_SHSIZE 512

// Here, we provide an alternate implementation of posix_spawn which correctly handles #!.
// This is based on the implementation of posix_spawn in bsd/kern/kern_exec.c from Apple's xnu source.
// Thus, I am fairly confident that this posix_spawn has correct behavior relative to macOS.
%hookf(int, posix_spawn, pid_t *pid, const char *orig_path, const posix_spawn_file_actions_t *file_actions, const posix_spawnattr_t *attrp, char *const orig_argv[], char *const envp[]) {
	
	// Call orig before checking for anything.
	// This mirrors the standard implementation of posix_spawn because it first checks if we are spawning a binary.
	int err = %orig;
	
	// %orig returns EPERM when spawning a script.
	// Thus, if err is anything other than EPERM, we can just return like normal.
	if (err != EPERM) 
		return err;
	
	// At this point, we do not need to check for exec permissions or anything like that.
	// 	because posix_spawn would have returned that error instead of EPERM.

	// Now we open the file for reading so that we can check if it's a script.
	// If it turns out not to be a script, the EPERM must be from something else
	//	so we just return err.

	FILE *file = fopen(orig_path, "r");
	if (file == NULL) {
		return err;
	}
	if (fseek(file, 0, SEEK_SET)) {
		return err;
	}	

	// In exec_activate_image, the data buffer is filled with the first PAGE_SIZE bytes of the file.
	// However, in exec_shell_imgact, only the first IMG_SHSIZE bytes are used.
	// Thus, we read IMG_SHSIZE bytes out of our file.
	// The buffer is filled with newlines so that if the file is not IMG_SHSIZE bytes,
	// 	the logic reads an IS_EOL.
	char vdata[IMG_SHSIZE] = {'\n'};
	if (fread(vdata, 1, IMG_SHSIZE, file) < 2) { // If we couldn't read at least two bytes, it's not a script.
		fclose(file);
		return err;
	}

	// Now that we've filled the buffer, we don't need the file anymore.
	fclose(file);
	
	// Now we follow exec_shell_imgact.
	// The point of this is to confirm we have a script 
	//	and extract the usable part of the interpreter+arg string.
	// Where they return -1, we don't have a shell script, so we return err.
	// Where they return an error, we return that same error.
	// We don't bother doing any SUID stuff because SUID scripts should be disabled anyway.
        char *ihp;
        char *line_startp, *line_endp;
        
	// Make sure we have a shell script.
        if (vdata[0] != '#' || vdata[1] != '!') {
		return err;
	}
        
        // Try to find the first non-whitespace character
        for (ihp = &vdata[2]; ihp < &vdata[IMG_SHSIZE]; ihp++) {
                if (IS_EOL(*ihp)) {
                        // Did not find interpreter, "#!\n"
                        return ENOEXEC;
                } else if (IS_WHITESPACE(*ihp)) {
                        // Whitespace, like "#!    /bin/sh\n", keep going.
                } else {
                        // Found start of interpreter
                        break;
                }
        }
        
        if (ihp == &vdata[IMG_SHSIZE]) {
                /* All whitespace, like "#!           " */
                return ENOEXEC;
        }

        line_startp = ihp;
        
        // Try to find the end of the interpreter+args string
        for (; ihp < &vdata[IMG_SHSIZE]; ihp++) {
                if (IS_EOL(*ihp)) {
                        // Got it
                        break;
                } else {
                        // Still part of interpreter or args
                }
        }
        
        if (ihp == &vdata[IMG_SHSIZE]) {
                // A long line, like "#! blah blah blah" without end
                return ENOEXEC;
        }
        
        // Backtrack until we find the last non-whitespace
        while (IS_EOL(*ihp) || IS_WHITESPACE(*ihp)) {
                ihp--;
        }
        
        // The character after the last non-whitespace is our logical end of line
        line_endp = ihp + 1;
        
        /*
         * Now we have pointers to the usable part of:
         *
         * "#!  /usr/bin/int first    second   third    \n"
         *      ^ line_startp                       ^ line_endp
         */

	// Now, exec_shell_imgact copies the interpreter into another buffer and then null-terminates it.
	// Then, it copies the entire interpreter+args into another buffer and null-terminates it for later processing into argv.
	// This processing is done in exec_extract_strings, which goes through and null-terminates each argument.
	// We will just do this all at once since that's much easier.
	
	// Keep track of how many arguments we have.
	int i_argc = 0;

	ihp = line_startp;
	while (true) {
		// ihp is on the start of an argument.
		i_argc++;
		// Scan to the end of the argument.
		for (; ihp < line_endp; ihp++) {
			if (IS_WHITESPACE(*ihp)) {
				// Found the end of the argument
				break;
			} else {
				// Keep going
			} 
		}
		// Null terminate the argument
		*ihp = '\0';
		// Scan to the beginning of the next argument.
		for (; ihp < line_endp; ihp++) {
			if (!IS_WHITESPACE(*ihp)) {
				// Found the next argument
				break;
			} else {
				// Keep going
			}
		}
		if (ihp == line_endp) {
			// We've reached the end of the arg string
			break;
		}
		// If we are here, ihp is the start of an argument.
	}
	// Now line_startp is a bunch of null-terminated arguments possibly padded by whitespace.
	// i_argc is now the count of the interpreter arguments.
       
	// Our new argv should look like i_argv[0], i_argv[1], i_argv[2], ..., orig_path, orig_argv[1], orig_argv[2], ..., NULL
	//	where i_argv is the arguments to be extracted from line_startp;
	// To allocate our new argv, we need to know orig_argc.
	int orig_argc = 0;
	while (orig_argv[orig_argc] != NULL) {
		orig_argc++;
	}
	
	// We need space for i_argc + 1 + (orig_argc - 1) + 1 char*'s
	char *argv[i_argc + orig_argc + 1];
	
	// Copy i_argv into argv
	int i = 0;
	ihp = line_startp;
	for (; i < i_argc; i++) {
		// ihp is on the start of an argument
		argv[i] = ihp;
		// Scan to the next null-terminator
		for (; ihp < line_endp; ihp++) {
			if (*ihp == '\0') {
				// Found it
				break;
			} else {
				// Keep going
			}
		}
		// Go to the next character
		ihp++;
		// Then scan to the next argument. 
		// There must be another argument because we already counted i_argc.
		for (; ihp < line_endp; ihp++) {
			if (!IS_WHITESPACE(*ihp)) {
				// Found it
				break;
			} else {
				// Keep going
			}
		}
		// ihp is on the start of an argument.
	}

	// Then, copy orig_path into into argv.
	// We need to make a copy of orig_path to avoid issues with const.
	char orig_path_copy[strlen(orig_path)+1];
	strcpy(orig_path_copy, orig_path);	
	argv[i] = orig_path_copy;
	i++;

	// Now, copy orig_argv[1...] into argv.
	for (int j = 1; j < orig_argc; i++, j++) {
		argv[i] = orig_argv[j];
	}
	// Finally, add the null.
	argv[i] = NULL;
	// Now, our argv is setup correctly. 

	// Now, we can call out to posix_spawn again.
	// The interpeter is in argv[0], so we use that for the path.
	return %orig(pid, argv[0], file_actions, attrp, argv, envp);
}

[deatondg]

Update: Apparently it’s something with injecting TweakInjector. If I export DYLD_INSERT_LIBRARIES=/usr/lib/TweakInject/shebang_fix.dylib to directly inject my tweak, everything appears to work as I’d expect.

What’s the problem with injecting TweakInjector?

[Diatrus]

@deatondg Wow! Amazing work on this. It was entirely unexpected but absolutely welcome. Regarding the injecting TweakInjector directly, I'm not entirely sure why that would cause it not to get injected through successive spawned programs, I do know it could cause some conflicting hooks if you have another tweak that hooks something you don't want it to.

[coolstar]

Regarding the injecting TweakInjector directly, I'm not entirely sure why that would cause it not to get injected through successive spawned programs

TweakInject.dylib by design unsets DYLD_INSERT_LIBRARIES so that random tweaks don't get injected into CLI processes spawned by stuff GUI apps like Sileo / NewTerm / etc.

[deatondg]

@Diatrus I’m just glad to have got something working! :)

@coolstar Makes sense, seems reasonable. I’ll just keep injecting my tweak manually then.

[Diatrus]

@deatondg this should now be fixed without the need for a tweak after reinstalling perl. Give it a a go, if it’s still not, I’ll reopen. Cheers.

[deatondg]

I'm excited to try this out soon. Thanks for your hard work!