Merge pull request #7 from steven-michaud/master

Deal with breakage caused by macOS 11.4 (fixes issue steven-michaud#28)

0-whats-new.md

@@ -1,3 +1,67 @@
+# What's New in Version 5.0.5
+
+macOS 11.4 broke HookCase, just like macOS 11.3 did. macOS 11.4 made
+further changes to `struct thread`, of a kind that normally only takes
+place in a new major release. These changes caused a kernel panic
+every time you tried to load a hook library into an application. The
+problem is fixed by HookCase 5.0.5. `struct thread` is one of several
+kernel structures that HookCase needs to access directly. For more
+information see
+[Issue #28](https://github.com/steven-michaud/HookCase/issues/28).
+
+# What's New in Version 5.0.4
+
+This version of HookCase fixes a bug that caused intermittent
+instability, though not kernel panics. I fixed it by tweaking the
+[code at the heart of HookCase's watchpoint support](HookCase/HookCase/HookCase.cpp#L11620).
+See [Issue #26](https://github.com/steven-michaud/HookCase/issues/26)
+for more information.
+
+HookCase's watchpoint code is quite complex. So if you see any sort of
+instability short of kernel panics, especially if it resembles what's
+reported at Issue #26, you should try
+[disabling watchpoint support](HookCase/HookCase/HookCase.cpp#L12531)
+
+# What's New in Version 5.0.3
+
+This release deals with changes in macOS 11.3 that broke HookCase. The
+11.3 update changed two kernel structures whose fields HookCase needs
+to access directly. Major changes were made to `struct task`, and
+`struct thread` seems to have been completely redesigned. This kind of
+change normally only takes place in a new major release, so HookCase
+wasn't "expecting" it. HookCase now does separate version checks for
+macOS 11 and macOS 11.3. This fixes
+[Issue #27](https://github.com/steven-michaud/HookCase/issues/27).
+
+# What's New in Version 5.0.2
+
+This version of HookCase fixes a bug that caused some interpose hooks
+to be skipped on macOS 11 (Big Sur)
+([Issue #24](https://github.com/steven-michaud/HookCase/issues/24)).
+HookCase uses a structure called the lazy pointer table to implement
+interpose hooks. In the past it was always located in the `__DATA`
+segment. But in Big Sur it's sometimes located in the `__DATA_CONST`
+segment. HookCase now looks for it in both places.
+
+# What's New in Version 5.0.1
+
+This version of HookCase fixes a bug that caused intermittent kernel
+panics in `set_interpose_hooks_for_module()`
+([Issue #22](https://github.com/steven-michaud/HookCase/issues/22)).
+They seem to have been particularly likely to occur with hook
+libraries containing lots of interpose hooks, particularly ones that
+are invoked both before and after the CoreFoundation framework is
+initialized.
+
+# What's New in Version 5.0
+
+HookCase now supports macOS 11 (Big Sur).
+
+Note that, on macOS 11, HookCase now requires the `keepsyms=1` boot
+arg. To set this you'll need to turn off SIP at least temporarily.
+
+`sudo nvram boot-args="keepsyms=1"`
+
 # What's New in Version 4.1.1
 
 This version of HookCase contains several tweaks to its watchpoint
@@ -16,8 +80,8 @@ This version of HookCase supports watchpoints. You can now set a
 watchpoint on a location in memory and gather information (including a
 stack trace) about the code that writes to that location.  For more
 information see
-[config_watcher() in the hook library template](HookLibraryTemplate/hook.mm#L793),
-[Hooked_watcher_example() in the hook library template](HookLibraryTemplate/hook.mm#L932)
+[config_watcher() in the hook library template](HookLibraryTemplate/hook.mm#L983),
+[Hooked_watcher_example() in the hook library template](HookLibraryTemplate/hook.mm#L1122)
 and [the watchpoints example](examples-watchpoints.md).
 
 # What's New in Version 4.0.5
@@ -80,7 +144,7 @@ HookCase now supports dynamically adding patch hooks for raw function
 pointers. This is useful in hooks for methods that use callbacks --
 for example CFMachPortCreate() and CFRunLoopObserverCreate(). For more
 information see
-[dynamic_patch_example() in the hook library template](HookLibraryTemplate/hook.mm#L876)
+[dynamic_patch_example() in the hook library template](HookLibraryTemplate/hook.mm#L1066)
 and [the dynamic patch hooks example](examples-dynamic-hooking.md).
 
 # What's New in Version 3.2.1
@@ -132,7 +196,7 @@ HookCase now supports macOS Mojave (10.14).
 
 But Mojave's Debug kernel is currently very flaky -- lots of panics,
 with and without HookCase.  So support for the Debug kernel
-[has been disabled](HookCase/HookCase/HookCase.cpp#L371), at least
+[has been disabled](HookCase/HookCase/HookCase.cpp#L397), at least
 temporarily.
 
 # What's New in Version 2.1
@@ -160,16 +224,16 @@ instead of `int 0x22`, as follows:
 at a particular address in a given module.  This means that HookCase
 can now hook methods that aren't in their module's symbol table.  For
 more information see
-[Hooked_sub_123abc() in the hook library template](HookLibraryTemplate/hook.mm#L915).
+[Hooked_sub_123abc() in the hook library template](HookLibraryTemplate/hook.mm#L1105).
 
-* Version 2.0 [fixes a bug](HookCase/HookCase/HookCase.cpp#L9381) that
+* Version 2.0 [fixes a bug](HookCase/HookCase/HookCase.cpp#L9914) that
 prevented interpose hooks from working outside the shared cache of
 system modules.
 
 * Version 2.0
-[fixes a previously undiscovered edge case](HookCase/HookCase/HookCase.cpp#L10871)
+[fixes a previously undiscovered edge case](HookCase/HookCase/HookCase.cpp#L11405)
 of an Apple kernel panic bug that was partially fixed in version 1.
 
 * Version 2.0
-[fixes a premature-release bug](Examples/events/hook.mm#L1335)
+[fixes a premature-release bug](Examples/events/hook.mm#L1459)
 in the "System Events" example's hook library.

1-more-about.md

@@ -64,7 +64,7 @@ on the same method, or try to step through code that contains a patch
 hook.
 
 Apple's support for `DYLD_INSERT_LIBRARIES` is implemented in
-[`/usr/lib/dyld`](https://opensource.apple.com/source/dyld/dyld-655.1.1/).
+[`/usr/lib/dyld`](https://opensource.apple.com/source/dyld/dyld-750.6/).
 A (shared) copy of this module gets loaded into the image of every new
 process before it starts running.  `dyld`'s `man` page calls it the
 "dynamic link editor", and it's what runs first (starting from
@@ -88,5 +88,5 @@ allow the original `dyld::InitializeMainExecutable()` method to run
 (which, among other things, runs the process's C++ initializers).
 
 For more information, the best place to start is the
-[long series of comments](HookCase/HookCase/HookCase.cpp#L6626)
+[long series of comments](HookCase/HookCase/HookCase.cpp#L7031)
 in `HookCase.cpp` before the definition of `C_64_REDZONE_LEN`.

2-building.md

@@ -1,7 +1,7 @@
 # Building
 
 HookCase requires a compatible version of OS X -- OS X 10.9
-(Mavericks) through macOS 10.15 (Mojave).  Building it also requires a
+(Mavericks) through macOS 11 (Big Sur).  Building it also requires a
 relatively recent version of XCode.  I recommend building on the
 version of OS X where you'll be using HookCase, and using the most
 recent version of XCode available for that version.  But the version

3-installing.md

@@ -47,13 +47,15 @@ libraries.
 
 3. Quit Terminal and reboot your computer.
 
-Now copy `HookCase.kext` to the `/usr/local/sbin/` directory.  One way
-to do this is with the following command:
+Now copy `HookCase.kext` to the `/usr/local/sbin/` directory.  You may
+need to create this directory.  Make sure it's owned by `root:wheel`.
 
 `sudo cp -R HookCase.kext /usr/local/sbin`
 
-To load `HookCase.kext` into the kernel, do the following on the
-command line:
+## On macOS 10.15 and below:
+
+On macOS 10.15 (Catalina) and below, you need only a single command to
+load `HookCase.kext` into the kernel:
 
 `sudo kextutil /usr/local/sbin/HookCase.kext`
 
@@ -73,10 +75,78 @@ To unload `HookCase.kext` from the kernel, run the following command:
 
 `sudo kextunload -b org.smichaud.HookCase`
 
+## On macOS 11 and above:
+
+Things are more complicated on macOS 11 (Big Sur) and above.  As of
+this version of macOS, HookCase requires the `keepsyms=1` boot arg.
+And third party kernel extensions must be loaded into the "auxiliary
+kext collection" before they can be loaded into the kernel.  Along the
+way you'll need to explicitly give permission for `HookCase.kext` to
+be loaded, then reboot your computer.  macOS 11 also uses a new set of
+command line utilties to load and unload kernel extensions.
+
+1. Change your boot args as follows. To do this, you will need to
+disable SIP at least temporarily. Reboot your computer to make the
+change take effect.
+
+`sudo nvram boot-args="keepsyms=1"`
+
+2. Run the following command at a Terminal prompt:
+
+`sudo kmutil load -p /usr/local/sbin/HookCase.kext`
+
+3. After a few seconds, a "System Extension Updated" dialog will
+appear telling you that the HookCase system extension has been
+updated.  Click on the "Open Security Preferences" button.
+
+4. In the "Security & Privacy" preference panel, first "click the lock
+to make changes", then click on the "Allow" button next to HookCase.
+Another dialog will appear telling you that "a restart is required
+before new system extensions can be used".  The default choice is "Not
+Now", and it's best to choose that.  Wierdness can happen if you
+restart immediately.  I usually close all open applications and then
+restart.
+
+5. After your computer has restarted, open a Terminal prompt and once
+again enter the following command.  It should immediately load
+`HookCase.kext` into the kernel.
+
+`sudo kmutil load -p /usr/local/sbin/HookCase.kext`
+
+Run `kextstat` to see that it did load.
+
+Run one of the following commands to unload `HookCase.kext` from the
+kernel:
+
+`sudo kmutil unload -p /usr/local/sbin/HookCase.kext`
+
+`sudo kumtil unload -b org.smichaud.HookCase`
+
+`HookCase.kext` will not be loaded automatically when you once again
+restart your computer.  You will need to load it (and unload it)
+manually as per step 4.
+
+## Increasing the kernel stack size
+
+For some reason, HookCase 5.0 and above sometimes require that you
+increase the kernel stack size.  I've seen kernel stack underflows
+hooking methods in 32-bit applications (like TextWrangler) on older
+versions of macOS (which still support them).  The symptom of a kernel
+stack underflow is a double-fault kernel panic with CR2 set to an
+address on the stack.  One way to increase the kernel stack size is as
+follows.  `kernel_stack_pages` default to `4`.  You will need to
+disable SIP at least temporarily to make changes to your "kernel boot
+args".
+
+1. `sudo nvram boot-args="keepsyms=1 kernel_stack_pages=6"`
+
+2. Reboot your computer.
+
+## Using the "development" and "debug" kernels
+
 HookCase supports the release, development and debug kernels.  But if
 you use it with the debug kernel, we recommend increasing the kernel
-stack size.  One way to do this is as follows.  `kernel_stack_pages`
-defaults to 4.
+stack size.  One way to do this is as follows.
 
 1. Copy `kernel.debug` (from the appropriate Kernel Debug Kit) to
    `/System/Library/Kernels`.
@@ -85,6 +155,3 @@ defaults to 4.
 
 2. Reboot your computer.
 
-Without this change, you sometimes get kernel panics using the debug
-kernel.  These are usually double-faults with `CR2` set to an address
-on the stack (indicating a stack underflow).

4-using.md

@@ -37,7 +37,7 @@ on OS X 10.10 (Yosemite) or 10.9 (Mavericks)).
 Recent versions of HookCase support creating a patch hook for an
 (un-named) method at a particular address in a given module.  (For
 more information see
-[Hooked_sub_123abc() in the hook library template](HookLibraryTemplate/hook.mm#L915).)
+[Hooked_sub_123abc() in the hook library template](HookLibraryTemplate/hook.mm#L1105).)
 So, for example, creating a patch hook for a function named
 "sub_123abc" would (by default) specify that the hook should be
 inserted at offset 0x123abc (hexadecimal notation) in the module.  But
@@ -53,7 +53,7 @@ best to patch callbacks in their "create" methods, before they start
 being used. Otherwise there's some danger of a race condition,
 especially if the callback can be used on different threads from the
 one that calls add_patch_hook(). For more information see
-[dynamic_patch_example() in the hook library template](HookLibraryTemplate/hook.mm#L876)
+[dynamic_patch_example() in the hook library template](HookLibraryTemplate/hook.mm#L1066)
 and [the dynamic patch hooks example](examples-dynamic-hooking.md).
 
 HookCase now supports watchpoints. You can set a watchpoint on a range
@@ -64,6 +64,6 @@ certain buffers in memory, for example the "sideband buffer" that's
 used by OpenGL accelerated graphics. Watchpoints are per page of
 memory. So to avoid confusion, it's best to set them in buffers that
 are page-aligned. For more information see
-[config_watcher() in the hook library template](HookLibraryTemplate/hook.mm#L793),
-[Hooked_watcher_example() in the hook library template](HookLibraryTemplate/hook.mm#L932)
+[config_watcher() in the hook library template](HookLibraryTemplate/hook.mm#L983),
+[Hooked_watcher_example() in the hook library template](HookLibraryTemplate/hook.mm#L1122)
 and [the watchpoints example](examples-watchpoints.md).

6-examples.md

@@ -20,6 +20,13 @@ partition that is by default mounted read-only. The effect of the
 `sudo mount -uw /` command is temporary, and only lasts until you
 reboot your computer.
 
+The [secinit](examples-secinit.md) and [Kernel logging](examples-kernel-logging.md)
+examples currently don't work at all on macOS 11 (Big Sur).  System
+files are in a separate partition, as on macOS 10.15.  But in addition
+to the previous protections, this partition is now a
+["Signed System Volume"](https://developer.apple.com/news/?id=3xpv8r2m)
+I don't yet know of a safe, easily undoable workaround for this.
+
 * [Dynamic patch hooks](examples-dynamic-hooking.md)
 * [Watchpoints](examples-watchpoints.md)
 * [xpcproxy trampoline](examples-xpcproxy.md)

Examples/dynamic-hooking/hook.mm

@@ -104,6 +104,7 @@ bool CanUseCF()
 #define MAC_OS_X_VERSION_10_13_HEX 0x00000AD0
 #define MAC_OS_X_VERSION_10_14_HEX 0x00000AE0
 #define MAC_OS_X_VERSION_10_15_HEX 0x00000AF0
+#define MAC_OS_X_VERSION_10_16_HEX 0x00000B00
 
 char gOSVersionString[PATH_MAX] = {0};
 
@@ -189,6 +190,11 @@ bool macOS_Catalina()
   return ((OSX_Version() & 0xFFF0) == MAC_OS_X_VERSION_10_15_HEX);
 }
 
+bool macOS_BigSur()
+{
+  return ((OSX_Version() & 0xFFF0) == MAC_OS_X_VERSION_10_16_HEX);
+}
+
 class nsAutoreleasePool {
 public:
     nsAutoreleasePool()
@@ -510,9 +516,74 @@ void GetModuleHeaderAndSlide(const char *moduleName,
         strncpy(moduleName_local, moduleName, sizeof(moduleName_local));
       }
       close(fd);
+#if __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ < 110000
     } else {
       strncpy(moduleName_local, moduleName, sizeof(moduleName_local));
     }
+#else // __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ < 110000
+    // On macOS 11 (Big Sur), open() generally doesn't work on moduleName,
+    // because it generally isn't in the file system (only in the dyld shared
+    // cache). As best I can tell, there's no general workaround for this
+    // design flaw. But because all (or almost all) frameworks have a
+    // 'Resources' soft link in the same directory where there used to be a
+    // soft link to the framework binary, we can hack together a workaround
+    // for frameworks.
+    } else {
+      char holder[PATH_MAX];
+      strncpy(holder, moduleName, sizeof(holder));
+      size_t fixed_to = 0;
+      bool done = false;
+
+      while (!done) {
+        char proxy_path[PATH_MAX];
+        strncpy(proxy_path, holder, sizeof(proxy_path));
+        const char *subpath_tag = ".framework/";
+        char *subpath_ptr =
+          strnstr(proxy_path + fixed_to,
+                  subpath_tag, sizeof(proxy_path) - fixed_to);
+
+        if (subpath_ptr) {
+          subpath_ptr += strlen(subpath_tag);
+          char subpath[PATH_MAX];
+          strncpy(subpath, subpath_ptr, sizeof(subpath));
+          subpath_ptr[0] = 0;
+
+          const char *proxy_name = "Resources";
+          size_t proxy_name_len = strlen(proxy_name);
+          strncat(proxy_path, proxy_name,
+                  sizeof(proxy_path) - strlen(proxy_path) - 1);
+
+          fd = open(proxy_path, O_RDONLY);
+          if (fd > 0) {
+            if (fcntl(fd, F_GETPATH, holder) != -1) {
+              fixed_to = strlen(holder) - proxy_name_len;
+              holder[fixed_to] = 0;
+              strncat(holder, subpath, sizeof(holder) - fixed_to);
+
+              const char *frameworks_tag = "Frameworks";
+              if (strncmp(holder + fixed_to, frameworks_tag,
+                          strlen(frameworks_tag)) != 0)
+              {
+                strncpy(moduleName_local, holder, sizeof(moduleName_local));
+                done = true;
+              }
+            } else {
+              done = true;
+            }
+            close(fd);
+          } else {
+            done = true;
+          }
+        } else {
+          done = true;
+        }
+      }
+
+      if (!moduleName_local[0]) {
+        strncpy(moduleName_local, moduleName, sizeof(moduleName_local));
+      }
+    }
+#endif // __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ < 110000
   }
 
   for (uint32_t i = 0; i < _dyld_image_count(); ++i) {