opam deleted home directory

[nerpderp83]

This issue was previously logged @ #3231 , I am re-submitting here to raise visibility incase it hits anyone else.

If your ocaml compiler version changes, opam will do an upgrade and I think if you don't have the opam vars set in the current shell, it will do an rm -rf from the root of your system.

See #3231 (comment)

[nerpderp83]

Given how much is controlled by env vars, here is the offending logic "patched".

camlp5/camlp5@d561693#diff-b67911656ef5d18c4ae36cb6741b7965

$(RM) -rf "$(DESTDIR)$(LIBDIR)/$(CAMLP5N)"

If the above env vars are not set, that command decays into rm -rf /.

Maybe opam shouldn't proceed with any action unless the proper env vars are set, including in subshells?

[dra27]

Sorry you've been hit by this :(

A few things to clarify, though:

• opam does indeed recompile a system switch if your system compiler is upgraded
• this issue only occurs if you have a faulty version of camlp5 installed
• I think that running opam update to update your package repository first mitigates this (is that right, @AltGr, or does opam use the old definition before uninstalling?)

opam's starting to use sand-boxing where available to prevent this - there's not a lot opam can do with environment variables here.

[nerpderp83]

I was in a new shell that afaik didn't have the opam env vars set, I had just done an upgrade of ocaml, opam detected the change and offered to upgrade opam for me.

I think if the env vars are not set and/or they are not set in a fresh subshell (for what ever reason), then opam should exit with an informative error. There are only a couple cases where opam should still function with out proper environment variables.

[dra27]

What are you regarding as "proper" environment variables? The fault here is that camlp5's configure script fails, leaving LIBDIR and DESTDIR unset, as you note above - it's not opam which sets those.

[AltGr]

NOTE TO ANY USERS IN DOUBT

The safe things to do

• don't run any opam commands
• either rm -rf ~/.opam (it's then safe to init again)
• or wait for more detailed instructions once we triple-check them, for a less destructive approach
• or update to the pre-release of opam 2.0.0

I am very sorry about this. Due to the issues you mentionned, we have been running to integrate sandboxing as quickly as possible in opam 2.0, and this is the main reason for the rc2 delay (we want to make sure it will work properly for everyone). Support is now in opam's master branch, but not yet in the release.

I think that running opam update to update your package repository first mitigates this (is that right, @AltGr, or does opam use the old definition before uninstalling?)

Unfortunately, to ensure consistency, opam 2 will use the definition it has been using for installation when removing, so opam update will not help here.

I was in a new shell that afaik didn't have the opam env vars set

it's not related to opam (would actually happen if compiling camlp5 from the command-line too!), these variables are expected to be set in the package's own Makefile.configure through its ./configure script.

Again, sorry, I was hoping we could ship the sandboxing update quickly enough to avoid such disasters.

[thierry-martinez]

The faulty rm -rf / will only occur if opam tries to build camlp5 version <= 7.04 with OCaml 4.06.1:
./configure will fail and then, opam will still try to execute make uninstall whereas the configuration files have been left blank.

Therefore:

• yes, opam update will help indeed, because the faulty removing only happens for new builds (that has even not been installed yet!), and opam update will make opam use camlp5 version >= 7.05 that has been fixed;
• it would be certainly a good idea to fix opam behaviour so that opam does not try to remove a package that has not been installed (because of failure during the building phase). It will be even better if opam 1.2 and 1.3 could be patched, but I am not sure how to make the patch myself.

[pmetzger]

What are you regarding as "proper" environment variables? The fault here is that camlp5's configure script fails, leaving LIBDIR and DESTDIR unset, as you note above - it's not opam which sets those.

True enough, but cold comfort to someone who has just had their system blown away. Perhaps if OPAM notices that an older version of camlp5 is about to be used in the offending manner, even though this is a special case, it should refuse to proceed.

[pmetzger]

@AltGr, you say:

The safe thing to do is to REMOVE any opam switch based on OCaml 4.06+ as soon as possible, using opam switch remove, AND run opam update.

Can one then restore 4.06, or does one have to leave it disabled to be safe? If the former, how can one check if one is already safe or not?

[dbuenzli]

Perhaps if OPAM notices that an older version of camlp5 is about to be used in the offending manner, even though this is a special case, it should refuse to proceed.

AFAIU. This problem is horrible because basically we can't do anything, if you opam update you'll never run into the issue as the package has been fixed. What you propose here is to update opam which is totally equivalent. Basically we can only pray that none will try to use opam with an outdated opam-repository that has the faulty version of the offending package. This will decoy with time but for now it seems people still run into the issue which is truly gruesome but I'm afraid there's no real fix here for that particular instance of the problem.

[pmetzger]

This will decoy with time but for now it seems people still run into the issue which is truly gruesome but I'm afraid there's no real fix here for that particular instance of the problem.

Perhaps an announcement on the caml mailing list and on discuss.ocaml.org with precise diagnosis and mitigation instructions would help then? I only found out about this by accident.

[dra27]

@pmetzger wrote:

True enough, but cold comfort to someone who has just had their system blown away.

Indeed, my first comment noted my sympathy to the situation. Context matters, please.

I was investigating this in detail with @AltGr on Friday. If you downgrade your system compiler back to 4.06.0, do an opam update and then upgrade the system compiler back to 4.06.1 you would be fine, yes. The only other way at the moment is manually editing things in ~/.opam very carefully (you have to remove camlp5 and any packages which depend on it from ~/.opam/system/installed and then run opam update).

So, we reckon that the only reliable way to fix this is going to be to push an update to opam 1.2.2 - possibly expressly for macOS (so via homebrew and, if it's also affected, MacPorts). The problem in opam 1.2.2 is that the question to upgrade the system compiler is asked before anything else can possibly process. The first fix will be to defer the "upgrade your system compiler" question when you run opam update and the second fix will be to add a special case which either detects the broken camlp5 package being present and requires an opam update to have been run or just detects any camlp5 and emits a strong warning.

My opinion is that the risk of announcing this even more widely before that patch is available is that there will be users who will fail to follow the instructions properly and answer "yes" to the "upgrade your system compiler" question so, especially at the weekend, I think it would be better not to post this to Discuss - what do you think? (and @AltGr, @avsm?)

I don't use macOS, but if I did, I'd really want GNU coreutils overriding rm:

dra@bionic:~$ sudo rm -rf /
rm: it is dangerous to operate recursively on '/'
rm: use --no-preserve-root to override this failsafe

(added in GNU coreutils 5.1.0 in November 2003)

[pmetzger]

Suggestion: would a shell script to let one check if one is vulnerable, possibly followed by saving the list of installed packages and forcing the removal of the bad switch, and then reinstallation, be feasible?

[pmetzger]

Indeed, my first comment noted my sympathy to the situation. Context matters, please.

Oh, that came through. It may not have been obvious in what I then wrote, written language doesn't convey tone that well. Mostly I'm just encouraging some sort of action to be taken, though I don't know what exactly would work.

[dra27]

@pmetzger - the script is a good idea (which I'm now working on!). The problem is that if you're affected (so you have the old metadata version of camlp5.7.03 installed in a system switch and have upgraded to 4.06.1) then any opam command will offer to upgrade the system switch. So any script must do it by inspecting the files manually.