Nothing installed in Lineage /system/addon.d/ via Magisk app Direct Install + fix_env

[mdoggydog]

I installed Magisk by using the "patch boot.img and flash with fastboot" method, since the installation instructions say that using the installer zip via a custom recovery is no longer recommended. On the next LineageOS OTA update, Magisk was not preserved, and I had to manually patch the boot.img and flash with fastboot again.

I discovered that /system/addon.d/99-magisk.sh was never installed, thus the OTA update would have no way to preserve Magisk.

Digging through the code, it appears that /system/addon.d/99-magisk.h is only installed when a Magisk installer zip is used via a custom recovery.

A related problem: even if Magisk is initially installed via installer-zip/custom-recovery, it appears that updating Magisk via the MagiskManager app will never update an existing /system/addon.d/99-magisk.sh either.

This issue is basically a duplicate of #3782, which I think got prematurely closed because I did not explain the problem well enough the first time. Please see my last comment in #3782 for more analysis of this problem.

[osm0sis]

It's supposed to install at some point via the app Direct Install too:

Magisk/app/src/main/java/com/topjohnwu/magisk/core/tasks/MagiskInstaller.kt

Lines 133 to 134 in cd96454

	// Extract scripts
	for (script in listOf("util_functions.sh", "boot_patch.sh", "addon.d.sh")) {

There was a plan to remove the unneeded installer code duplication and have everything run and maintained in one place via the shell scripts, but I'm not sure where that's at now.

[mdoggydog]

Yep... all the bits are there. This probably needs just a one- or two-line fix MagiskInstaller.kt, along the lines of cp /data/adb/addon.d.sh /system/addon.d/99-magisk.sh spelled the correct way (well, after testing if the /system/addon.d/ directory exists). But maybe there is some tricky subtlety involving security contexts or file ownership or something; this android system-level stuff is all new to me.

Half of the lines of code in

Magisk/scripts/flash_script.sh

Lines 78 to 85 in b6643b7

	if [ -d /system/addon.d ]; then
	ui_print "- Adding addon.d survival script"
	blockdev --setrw /dev/block/mapper/system$SLOT 2>/dev/null
	mount -o rw,remount /system || mount -o rw,remount /
	ADDOND=/system/addon.d/99-magisk.sh
	cp -af $COMMONDIR/addon.d.sh $ADDOND
	chmod 755 $ADDOND
	fi

are just ensuring that the target directory exists and is writable; I don't know if the writeability is an issue outside of the recovery environment.

[osm0sis]

So looks like this was intentional. John said "I do not want to involve system rw at anytime during boot. This just opens up a whole can of worms..."

I won't close this as #wontfix since he may still change his mind. It would just need to use Magisk's own system mirror and rw it if the addon.d directory exists. 🤞

[mdoggydog]

I'm sorry if I'm missing something, but I just don't understand that comment in the context of this bug report, which is about install-time not boot-time. The essential core of this issue is:

• If your device's OS has a /system/addon.d/ directory, then:
  • If Magisk is installed via installer .zip in recovery, then 99-magisk.sh is installed.
  • If Magisk is installed via Magisk app (MagiskInstaller.kt), then 99-magisk.sh is not installed.

That means that some devices out there are getting 99-magisk.sh and some are not, and it only depends on how the user decided to install Magisk (and in neither case does this installation happen at boot time).

How does "system rw at anytime during boot" come into this?

[mdoggydog]

(To rephrase again: logically, either 99-magisk.sh should be installed, or it shouldn't, so either MagiskInstaller.kt has a bug, or the .zip installer has a bug. And people installing via the .zip do not seem to be complaining....)

[osm0sis]

He means "while booted", hopefully that clears up the context for you.

[mdoggydog]

Ohhhhhhh! Yes, that clears it up a lot! Thank you! I now understand why it is the way it is right now.

It would be really nice if this difference between the two installation methods was mentioned in the installation documentation (https://topjohnwu.github.io/Magisk/install.html), because it is a pretty significant difference in behavior for a ROM that supports addon.d. One way, Magisk magically survives future OTA updates; the other way, the user has to manually intervene before the next reboot....

(In any case, at least the next confused schmuck like me that is trying to figure out what is going on will have this bug report to stumble upon. :^))

[osm0sis]

addon.d-v2 is only guaranteed to work from a booted OTA. Lineage intentionally didn't want it working in recovery and TWRP only has it working in the _9 branch with some hacks I suggested currently.

[osm0sis]

For Magisk addon.d script to install it must currently be installed from recovery, then if you have an A/B device (i.e. addon.d-v2) then the only way to be sure the addon.d script runs is to do the OTA through the system settings updater app.

[osm0sis]

Hmm I believe either recovery should work. It can still install the addon.d script to /system in Lineage Recovery last time I checked.

But the Magisk apk must be flashed in recovery for the addon.d script to be installed. If it's not working after that then there's something else weird going on, separate from this issue.

[osm0sis]

Not sure why regular addon.d (i.e. A-only devices) wouldn't work in LOS Recovery, but, again, that's a different issue from this.

Edit: Oh wait, I know why, LOS Recovery doesn't support decrypting /data. 😛

Well known if you check the logs:
https://github.com/topjohnwu/Magisk/blob/master/scripts/addon.d.sh#L39-L40

So yeah probably TWRP only for A-only too.

[osm0sis]

Sure, fuck it, Merry Christmas.

[osm0sis]

That said, Magisk still makes system rw on custom ROMs to remove any existing system su (see remove_system_su), so unless you have a way to systemlessly do that as part of injecting Magisk's su wherever it needs to go, then system rw is still a necessary "evil" and so the addon.d script might as well also remain part of the package.

[yujincheng08]

remove_system_su only uses on recovery mode, so not "while booted": 17efdff

Further, module that adds addon support for magisk: https://github.com/yujincheng08/Magisk-Addon. This also adds support if magisk is installed via boot patch.

[osm0sis]

You're incorrect. remove_system_su is run booted as part of addon.d-v2 as well.

[yujincheng08]

remove_system_su is run booted as part of addon.d-v2 as well

But we are now removing addon support. So when addon support removes, no more remount,rw

[osm0sis]

Then I guess you should add remove_system_su to your module.

[yujincheng08]

Since magisk has one (tho for recovery only), we can reuse it. If magisk removes this function, we can add that too. I am just proving the possibility of the existence of such a module. So the implementation is somehow trivial. Of course, we can add more features.

[vvb2060]

@osm0sis

And there it is. Let's let Lineage know you brazenly don't consider their ROM Android. Maybe you should check in with topjohnwu before you alienate the entire Android customization community.

It looks like LineageOS won't pass CTS and I'm surprised by the news. LineageOS, being the most popular custom ROM, should avoid splitting the Android community and make the ROM compliant with CDD to reduce the pressure on developers to adapt it.

[vvb2060]

https://github.com/programminghoch10/Lygisk/blob/ci-management/README.md?plain=1#L9-L11

Lygisk fixes the needed /data access in addon.d,
which fails to reinstall Magisk during OTA
if the device does not support FBE decryption in recovery.

I saw the previous link (programminghoch10/Lygisk#20), this feature has never worked on FBE devices. #3037

This feature requires writing to system partitions and unencrypted data partitions, which would be unthinkable in 2022.

[christophehenry]

That is a pretty heated conversation here. Very technical too, but I haven't seen anyone consider the problem from the user perspective. It excessively simple: I don't want to be required to reinstall Magisk any time I do a Lineage update. This is a fairly legitimate request, I think. I don't care how ugly addons.d is or how hypothetically it may break things in the future.

It works.

And that's all a user asks for. So, until a better solution is found, there's no good reason to deny a feature parity based on that one.

[yujincheng08]

@christophehenry I absolutely know users' requests. And my point is to avoid magisk doing such a dirty job (since addon.d itself is quite a dirty solution). Instead, a better solution is, no doubt, to build a dedicated magisk module for it, and we have basically developed a very initial version of such a module: https://github.com/yujincheng08/Magisk-Addon. Moreover, other developers are now attracted to making this module not depend on an unencrypted data partition.

[christophehenry]

@yujincheng08 Sounds promising.

Thank you all for your work.

[mdoggydog]

Hi, it's me, 🤩 @mdoggydog, the original poster here at good ol' #3820 --- "the thread that keeps on giving". 🎅🎁

As a conscientious but naive user, my perspective is:

• I don't follow Magisk conversations all day on XDA/Twitter/Reddit/etc, but I do my best to RTFM (read the documentation) before I go pestering anyone.

  • The Magisk README literally says

    "Github is the only source where you can get official Magisk information and downloads."

    so I don't think I should even try to look anywhere else.

• "Surviving OTA updates" is an essential feature for Magisk, and should be supported on any system where it is possible.
  • I am using a ROM that features OTA updates, and I chose that ROM in part because I want those easy OTA updates; I bothered to install Magisk, because I need Magisk; and thus, if Magisk does not survive an OTA update (especially if it could and it used to), I will be frustrated.
• "Via .zip sideload in Recovery" is the best way to install Magisk, and I have no idea why it would be deprecated.
  • Why the best? Because it is so simple! Look:

    $ adb reboot recovery
    $ adb sideload Magisk-vXXX.zip
    

    That's it! And if I just installed a ROM via Recovery, then I only need the second line! No craziness with extracting a boot.img from somewhere, transfer to phone, patch it, transfer it back, boot to bootloader, flash via fastboot... Who wants to do all that?
  • Seriously, "Install via .zip in Recovery" is awesome.
• The documentation says nothing about what happens to Magisk after a system update (OTA or not), so if Magisk disappears after an update, I become worried; and if Magisk had survived earlier updates and then disappears, I am confused; and when I finally discover an answer by digging through two-year-old bug reports in github, and I realize that I am just one of many users suffering the same experience, I am frustrated and angry.
  • Yes, there is that docs/ota.md page, but it is not linked by any of the documentation index pages. I only find it after desperately rooting around in the Magisk source tree. (Even I keep forgetting it exists, even though this is my favorite topic on my favorite piece of software of all time.) Also, it doesn't mention anything about the addon.d mechanism.
  • To this day, the documentation still does not mention, anywhere, that "Install via .zip in Recovery" is the magic that provides the addon.d installation.
• I know nothing about Modules and the Magisk documentation does not help me. I do not know where to find modules, how to install them, or (very important) how to assess the trustworthiness of any modules I do find.

As a security-conscious software engineer, my perspective is:

• Moving OTA-survival into a module seems reasonable because:
  1. As I understand it, OTA-survival will never work automagically on StockROM's, because stock Android has no native mechanism for "specify extra stuff to do at the end of an OTA update", so it cannot be a "universal" feature anyway.
  2. Implementation as a module would make it easier to support different/new/additional OTA-survival mechanisms in the future.
  3. Set up via installing a module makes it easier to have a single user-flow for enabling OTA-survival, when it is desired and available, no matter how Magisk was installed. (I.e., "Finally, if you want OTA-survival, install one of these modules...")
• However, I am concerned that if an essential feature is moved into a module, it will become more fragile because Magisk-core will take the attitude of "not my problem anymore!"
  • The basic machinery for OTA-survival needs to be part of the core, and needs to have a stable API. (I assume that there is a lot of overlap between "bits needed to install Magisk" and "bits needed to survive OTA".)
  • The focus of survival modules should be on how/when that core machinery is activated.
  • The Magisk documentation should, at the very least, inform users that these modules exist and where to find them.
• I would not want to see an essential feature, OTA-survival, removed from Magisk-core until there was a well-tested, widely-available, well-documented alternative available.
  • I appreciate that @yujincheng08 has a proof-of-concept module already, but I wouldn't expunge addon.d support from Magisk-core yet without the testing, documenting, and informing users.
• "Deprecation" means (1) informing other people, through documentation, that a change will be happening; (2) providing a timeline for that change; (3) providing the alternative that should be used to replace the deprecated element. Deprecation is a process to ensure feature-continuity without surprises.
  • Mentioning that something is deprecated in a sentence buried in 87 comments in a bug report is, at best, "wishful thinking", at worst, "preparing for failure".

Happy Holidays to all the residents of #3820!

😷 🥳 🎉 🍾 🍿

[osm0sis]

Okay, so the best way to handle this would be that addon.d script becomes opt-in only as a button in the app settings, similar to Systemless Hosts, but only visible on a custom ROM with /system/addon.d. That way it can be used regardless of Magisk install method, and a custom ROM user can choose not to use it too.

[breversa]

I just found out about this issue while doing yet another Magisk reinstall after a LineageOS-based /e/OS update.

Is there any roadmap to implement @osm0sis’ proposed solution?

@mdoggydog : best user feedback/POV I’ve ever read!!! :D

[brianjmurrell]

Looking on my device in /system/addon.d/ I don't see any 99-magisk.sh.

So assuming I have Magisk 26.3 installed on my device, if I (i.e. manually, with adb) copy https://github.com/topjohnwu/Magisk/blob/v26.3/scripts/addon.d.sh to /system/addon.d/99-magisk.sh on my device, can I stop having to do:

open magisk manager before starting lineage os updater (important), minimize magisk manager and run lineage os updater do not reboot when prompted, go back to magisk manager click install magisk and click install after OTA and reboot when prompted (magisk should be present along with your modules and settings)

and just let the normal post-OTA-update reboot happen as usual and have Magisk survive the OTA update? My device is A/B if that makes any difference.

[Anderhar]

Further, module that adds addon support for magisk: https://github.com/yujincheng08/Magisk-Addon. This also adds support if magisk is installed via boot patch.

Found a module that works around the issue: https://github.com/mModule/mSurvival

[brianjmurrell]

Further, module that adds addon support for magisk: https://github.com/yujincheng08/Magisk-Addon.

This one supplies it's own script for /data/addon.d. What happens if a newer Magisk updates it's addon.d script?

Found a module that works around the issue: https://github.com/mModule/mSurvival

This one installs the addon.d script that is supplied by the Magisk that is installed. This seems more resilient than the above one supplying it's own. But it still begs the question of what happens if Magisk is updated to a newer version and that newer version of Magisk supplies an updated addon.d script.

[Anderhar]

what happens if Magisk is updated to a newer version and that newer version of Magisk supplies an updated addon.d script.

Just my assumption: if this module only puts the same files in the same place as the original Magisk installation via recovery, then why shouldn't Magisk update these files like its own?

[Joni-Ahmed-Dev]

I think It's ok.

[duven87]

Hi, I noticed that some apps (tk.de) checks if magisk.sh is in addon.d and does not let you use warning that the phone is rooted.
Would it be possible that with the hide Magisk app option also edit that script name?