Skip to content

Commit

Permalink
Update documentations
Browse files Browse the repository at this point in the history
  • Loading branch information
@topjohnwu
topjohnwu committed Sep 19, 2019
1 parent b44f512 commit 9329094
Show file tree
Hide file tree
Showing 7 changed files with 123 additions and 145 deletions.
22 changes: 1 addition & 21 deletions docs/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,34 +1,14 @@
# Magisk Documentations
(Updated on 2019.5.1)
(Updated on 2019.9.19)

- [Installation](install.md)
- [Prerequisite](install.md#prerequisite)
- [Custom Recovery](install.md#custom-recovery)
- [Boot Image Patching](install.md#boot-image-patching)
- [Samsung (System-as-root)](install.md#samsung-system-as-root)
- [Huawei](install.md#huawei)
- [Tutorials](tutorials.md)
- [OTA Installation](tutorials.md#ota-installation)
- [Best Practices for MagiskHide](tutorials.md#best-practices-for-magiskhide)

The followings are for developers

- [Magisk Details](details.md)
- [File Structure](details.md#file-structure)
- [Magisk Booting Process](details.md#magisk-booting-process)
- [Resetprop](details.md#resetprop)
- [Magic Mount](details.md#magic-mount)
- [Miscellaneous](details.md#miscellaneous)
- [Magisk Tools](tools.md)
- [Developer Guides](guides.md)
- [Magisk Modules](guides.md#magisk-modules)
- [Magisk Module Installer](guides.md#magisk-module-installer)
- [Submit Modules](guides.md#submit-modules)
- [Notes on Partitions](guides.md#notes-on-partitions)
- [Boot Scripts](guides.md#boot-scripts)
- [Remove Files](guides.md#remove-files)
- [Remove Folders](guides.md#remove-folders)
- [Deployment](deploy.md)
- [Systemless](deploy.md#systemless)
- [System Only](deploy.md#system-only)
- [Exploits](deploy.md#exploits)
61 changes: 8 additions & 53 deletions docs/deploy.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,24 +16,24 @@ Here are the bare minimum commands to install Magisk into a stock boot/recovery
BOOTIMAGE=<path to boot>
# First unpack the image
./magiskboot --unpack $BOOTIMAGE
./magiskboot unpack $BOOTIMAGE
# In normal cases, after unpacking you should get at least kernel and ramdisk.cpio
# Patch ramdisk
./magiskboot --cpio ramdisk.cpio \
./magiskboot cpio ramdisk.cpio \
"mkdir 000 .backup" \ # create a folder to store our init backup
"mv init .backup/init" \ # backup the original init
"add 750 init magiskinit" # replace init with magiskinit
# Patch kernel to always use ramdisk as rootfs
# You only need to do this on system-as-root devices
./magiskboot --hexpatch kernel \
./magiskboot hexpatch kernel \
736B69705F696E697472616D6673 \
77616E745F696E697472616D6673
# Repack the boot image
./magiskboot --repack $BOOTIMAGE
./magiskboot repack $BOOTIMAGE
# The patched image should be located in new-boot.img
```
Expand All @@ -45,6 +45,9 @@ WIP
# Currently not available
```

## Emulators (Official AVB Only)
The script `scripts/emulator.sh` allows you to establish a minimal Magisk environment within the official Android Virtual Device included along with Android Studio / SDK. Please check the comments in the script for further information.

## Exploits
**(Note: Magisk could only be used as root)**

Expand All @@ -53,52 +56,4 @@ Occasionally, there would be exploits in certain devices that could lead to full
- Effective UID should be privileged (root, or `euid=0`)
- Have the ability to reload `sepolicy` (which 99.9% of the time means SELinux permissive)

Once you got a proper root shell, you should have `magiskinit` somewhere on the device. The basic idea is try to live patch `sepolicy` with `magiskpolicy`, and start `magiskd` with `magisk --daemon`. Here are some examples you could use as a reference.

If dm-verity is enforced (no system r/w allowed)

```
# Assume magiskinit is in current directory
# All commands are required to run on each reboot
# Live patch selinux
ln -s ./magiskinit magiskpolicy
./magiskpolicy --live --magisk "allow magisk * * *"
# Mount tmpfs to /sbin
mount -t tmpfs tmpfs /sbin
chmod 755 /sbin
chcon u:object_r:magisk_file:s0 /sbin
# Add files to /sbin
./magiskinit -x magisk /sbin/magisk
cp -a magiskpolicy /sbin
/sbin/magisk --install /sbin
# Launch magisk daemon
/sbin/magisk --daemon
# (Optional) switch back to enforced
setenforce 1
```

If dm-verity is not enforced (can modify system)

```
# Assume magiskinit is in current directory
# The following commands should only need to run once
# Mount system rw
mount -o rw,remount /system
# Add files to system
./magiskinit -x magisk /system/xbin/magisk
cp -a magiskinit /system/xbin
ln -s /system/xbin/magiskinit /system/xbin/magiskpolicy
/system/xbin/magisk --install /system/xbin
# The following commands should run on each reboot
/system/xbin/magiskpolicy --live --magisk "allow magisk * * *"
/system/xbin/magisk --daemon
```
You can check out `scripts/emulator.sh` as a reference for bringing up Magisk with a root shell. Note that these changes are not persistent, and you will need to find ways to rerun the whole process every boot.
44 changes: 22 additions & 22 deletions docs/details.md
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Magisk Details
## File Structure
### Paths in "sbin tmpfs overlay"
sbin tmpfs overlay is the key to hiding Magisk from detection. All Magisk binaries, applets, mirrors, and other trivial stuffs are all located in the `tmpfs` mounted on `/sbin`. MagiskHide can just simply unmount `/sbin` and the bind mounts to hide all modifications easily.
One of Magisk's breakthrough designs is sbin tmpfs overlay. It is required to support system-as-root devices, and also is the key to hiding Magisk from detection. All Magisk binaries, applets, mirrors, and other trivial stuffs are all located in the `tmpfs` mounted on `/sbin`. MagiskHide can just simply unmount `/sbin` and the bind mounts to hide all modifications easily.

```
# Binaries like magisk, magiskinit, and all symlinks to
Expand All @@ -20,17 +20,21 @@ $MAGISKTMP/modules
# The configuration used in last installation
$MAGISKTMP/config
MIRRORDIR=$MAGISKTMP/mirror
# Partition mirrors.
# There would be system, vendor, data, and possibly product
# in this directory, each is the mirror to the name of the partition
$MAGISKTMP/mirror
# System mirror
$MIRRORDIR/system
# Root directory patch files
# On system-as-root devices, / is not writable.
# All patched files are stored here and bind mounted at boot.
$MAGISKTMP/rootdir
# Vendor mirror, could be a symlink to $SYSTEMMIR/vendor
# if vendor is not a separate partition
$MIRRORDIR/vendor
# The patched sepolicy file on system-as-root devices.
# This is required as /sepolicy does not exist
# on those devices and / is not writable.
/sbin/.se
# Data mirror to workaround nosuid flag
$MIRRORDIR/data
```

### Paths in `/data`
Expand Down Expand Up @@ -70,19 +74,15 @@ DATABIN=$SECURE_DIR/magisk
```

### Final Words
The file structure of Magisk is designed in a weird and overly complicated way. But all of these quirks are done to properly support hiding modifications from detection. These design choices are mostly what makes Magisk difficult to implement properly and maintain.

## Magisk Booting Process
### Pre-Init
`magiskinit` will replace `init` as the first program to run. It is responsible for constructing rootfs on system-as-root devices: it parses kernel cmdline, sysfs, device tree fstabs, uevents etc., recreating **early-mount** and clones rootfs files from the system. On traditional devices, it will simply revert `init` to the original one and continue on to the following steps.
`magiskinit` will replace `init` as the first program to run.

- Early mount required partitions. On system-as-root devices, we will switch root to system
- Inject magisk services into `init.rc`
- Load sepolicy either from `/sepolicy`, precompiled sepolicy in vendor, or compile split sepolicy
- Patch sepolicy rules and dump to `/sepolicy` and patch `init` to always load `/sepolicy`
- Fork a new daemon and wait for early-init trigger
- Patch sepolicy rules and dump to `/sepolicy` or `/sbin/.se` and patch `init` or `libselinux.so` to load the patched policies
- Execute the original `init` to start the ordinary boot process
- The early-init daemon will construct `/sbin` `tmpfs` overlay and remove all traces of Magisk in ramdisk

### post-fs-data
This triggers on `post-fs-data` when `/data` is properly decrypted (if required) and mounted. The daemon `magiskd` will be launched, post-fs-data scripts are executed, and module files are magic mounted.
Expand All @@ -91,21 +91,21 @@ This triggers on `post-fs-data` when `/data` is properly decrypted (if required)
Later in the booting process, the class `late_start` will be triggered, and Magisk "service" mode will be started. In this mode, service scripts are executed, and it will try to install Magisk Manager if it doesn't exist.

## Resetprop
Usually, system properties are designed to only be updated by a single `init` process and read-only to non-root processes. With root you can change properties by sending requests via `property_service` using commands such as `setprop`, but you are still prohibited from changing read-only props (props that start with `ro.` like `ro.build.product`) and deleting properties.
Usually, system properties are designed to only be updated by `init` and read-only to non-root processes. With root you can change properties by sending requests to `property_service` (hosted by `init`) using commands such as `setprop`, but changing read-only props (props that start with `ro.` like `ro.build.product`) and deleting properties are still prohibited.

`resetprop` is implemented by distilling out the source code related to system properties from AOSP with modifications to map the property area, or `prop_area`, r/w and some clever hacks to modify the trie structure in ways it wasn't intended, like detaching nodes. In a nut shell, it directly do modifications to `prop_area`, bypassing the need to go through `property_service`. Since we are bypassing `property_service`, there are a few caveats:
`resetprop` is implemented by distilling out the source code related to system properties from AOSP and patched to allow direct modification to property area, or `prop_area`, bypassing the need to go through `property_service`. Since we are bypassing `property_service`, there are a few caveats:

- `on property:foo=bar` actions registered in `*.rc` scripts will not be triggered if property changes does not go through `property_service`. The default set property behavior of `resetprop` matches `setprop`, which **WILL** trigger events (implemented by first deleting the property then set it via `property_service`), but there is a flag `-n` to disable it if you need this special behavior.
- `on property:foo=bar` actions registered in `*.rc` scripts will not be triggered if property changes does not go through `property_service`. The default set property behavior of `resetprop` matches `setprop`, which **WILL** trigger events (implemented by first deleting the property then set it via `property_service`). There is a flag `-n` to disable it if you need this special behavior.
- persist properties (props that starts with `persist.`, like `persist.sys.usb.config`) are stored in both `prop_area` and `/data/property`. By default, deleting props will **NOT** remove it from persistent storage, meaning the property will be restored after the next reboot; reading props will **NOT** read from persistent storage, as this is the behavior of `getprop`. With the flag `-p`, deleting props will remove the prop in **BOTH** `prop_area` and `/data/property`, and reading props will be read from **BOTH** `prop_area` and persistent storage.

## Magic Mount
I will skip the details in the actual implementation of how Magic Mount works as it will become a lecture, but you can always directly dive into the source code if interested. (`bootstages.c`)
I will skip the details in the actual implementation and algorithm of Magic Mount, but you can always directly dive into the source code if interested. (`bootstages.cpp`)

Even though the mounting logic and traversal algorithm is pretty complicated, the final result of Magic Mount is actually pretty simple. For each module, the folder `$MODPATH/system` will be recursively merged into the real `/system`; that is: existing files in the real system will be replaced by the one in modules' system, and new files in modules' system will be added to the real system.
Even though the mounting logic is pretty complicated, the final result of Magic Mount is actually pretty simple. For each module, the folder `$MODPATH/system` will be recursively merged into the real `/system`; that is: existing files in the real system will be replaced by the one in modules' system, and new files in modules' system will be added to the real system.

There is one additional trick you can use: if you place an empty file named `.replace` in any of the folders in a module's system, instead of merging the contents, that folder will directly replace the one in the real system. This will be very handy in some cases, for example swapping out a system app.

If you want to replace files in `/vendor`, please place it under `$MODPATH/system/vendor`. Magisk will transparently handle both cases, whether vendor is a separate partition or not.
If you want to replace files in `/vendor` or `/product`, please place them under `$MODPATH/system/vendor` or `$MODPATH/system/product`. Magisk will transparently handle both cases, whether vendor or product is a separate partition or not.

## Miscellaneous
Here are some tidbits in Magisk but unable to be categorized into any sections:
Expand Down
25 changes: 18 additions & 7 deletions docs/guides.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Developer Guides

Please read through [Magisk Details](details.md) before reading the following guides. If you are developing a module, pay extra attention to the [Magic Mount](details.md#magic-mount) section.

## Magisk Modules
A Magisk module is a folder placed in `/data/adb/modules` with a structure below:

Expand Down Expand Up @@ -36,7 +38,11 @@ A Magisk module is a folder placed in `/data/adb/modules` with a structure below
│   │   ├── .
│   │   ├── .
│   │   └── .
│   ├── vendor <--- Auto generated. A symlink to $MODID/system/vendor
│   │
│ │ *** Auto Generated by Magisk ***
│   │
│   ├── vendor <--- A symlink to $MODID/system/vendor
│   ├── product <--- A symlink to $MODID/system/product
│ │
│ │ *** Others ***
│ │
Expand Down Expand Up @@ -84,12 +90,6 @@ You can submit a module to **Magisk-Module-Repo** so users can download your mod
- When your module is downloaded with Magisk Manager, `META-INF/com/google/android/update-binary` will be **forcefully** replaced with the latest [`module_installer.sh`](https://github.com/topjohnwu/Magisk/blob/master/scripts/module_installer.sh) to make sure all installation uses the latest scripts.
- Since `update-binary` will be replaced, this means that all modules in the repo are expected to follow how the installation framework works: the installation framework will load your `install.sh` script and run the corresponding callbacks.
- This also means that you should NOT add custom logic in `update-binary` as they will simply be ignored.
- **Existing module devs please read!!** For devs migrating from the old template based modules to the new installer format, one thing you might overlook is the change in configuration flags: it no longer uses `AUTO_MOUNT`, but instead uses `SKIP_MOUNT`. In a nutshell, `AUTO_MOUNT=true` behaves exactly the same as `SKIP_MOUNT=false`, and 99% of the time you should NOT touch this flag.

## Notes on Partitions
On modern Android, `/system/vendor` is moved out from the system partition into its own separate `vendor` partition. For module developers, Magisk will handle these different configurations transparently so you do not need to worry anything about it. If you want to modify files in `vendor`, place the modified files under `/system/vendor` and you're good to go!

Starting in Android Q and some devices on older Android versions, a separate partition `platform` is available. Support for `platform` will come soon in upcoming Magisk versions, please stay tuned!

## Boot Scripts
In Magisk, you can run boot scripts in 2 different modes: **post-fs-data** and **late_start service** mode.
Expand Down Expand Up @@ -120,6 +120,17 @@ In Magisk, there are also 2 kinds of scripts: **general scripts** and **module s

Magisk's internal busybox's path `$BBPATH` is always prepended in `PATH`. This means all commands you call in scripts are always using busybox unless the applet is not included. This makes sure that your script always run in a predictable environment and always have the full suite of commands regardless of which Android version it is running on.

## Root Directory Overlay System
Since `/` is read-only in system-as-root devices, Magisk provides an overlay system, allowing developers to patch files / add new rc scripts. Additional files shall be placed in the `overlay.d` folder in the ramdisk, and they will have the following restrictions:

- All `*.rc` files in `overlay.d` will be read and concatenated *AFTER* `init.rc`
- Replacing existing files are allowed.<br>
e.g. you can replace `/res/random.png` by adding the file `overlay.d/res/random.png`
- Non-existing files will be ignored (with exceptions detailed in the next point).<br>
e.g. `overlay.d/new_file` will be ignored if `/new_file` does not exist
- Additional files in `overlay.d/sbin` is allowed as they will be copied into Magisk's sbin overlay.<br>
e.g. `overlay.d/sbin/libfoo.ko` will be copied to `/sbin/libfoo.ko`.

## Remove Files
How to remove a file systemless-ly? To actually make the file *disappear* is complicated (possible, not worth the effort). Replacing it with a dummy file should be good enough! Create an empty file with the same name and place it in the same path within a module, it shall replace your target file with a dummy file.

Expand Down
Loading

0 comments on commit 9329094

Please sign in to comment.