Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failed to recover LUKS version 2 #2204

Closed
adatum opened this issue Aug 11, 2019 · 24 comments
Closed

Failed to recover LUKS version 2 #2204

adatum opened this issue Aug 11, 2019 · 24 comments
Labels
enhancement Adaptions and new features fixed / solved / done
Milestone

Comments

@adatum
Copy link

adatum commented Aug 11, 2019

  • ReaR version ("/usr/sbin/rear -V"): 2.4 and 2.5

  • OS version ("cat /etc/rear/os.conf" or "lsb_release -a" or "cat /etc/os-release"): Fedora 30

  • ReaR configuration files ("cat /etc/rear/site.conf" and/or "cat /etc/rear/local.conf"):

PROGS=( "${PROGS[@]}" /home/test/Downloads/borg-linux64 )
  • Hardware (PC or PowerNV BareMetal or ARM) or virtual machine (KVM guest or PoverVM LPAR): Virtualbox VM

  • System architecture (x86 compatible or PPC64/PPC64LE or what exact ARM device): x86_64

  • Firmware (BIOS or UEFI or Open Firmware) and bootloader (GRUB or ELILO or Petitboot): UEFI

  • Storage (local disk or SSD) and/or SAN (FC or iSCSI or FCoE) and/or multipath (DM or NVMe): local SSD

  • Description of the issue (ideally so that others can reproduce it):
    The disk layout recreation script fails after printing a message asking for the LUKS password, but never giving a chance to enter it. I tried rerunning it, but each time it skips to exactly the same point asking the options seen in the screenshot.

rear_luks_fail

@gozora
Copy link
Member

gozora commented Aug 11, 2019

Can you please test if you have same behavior with ReaR 2.2 ?

V.

@adatum
Copy link
Author

adatum commented Aug 11, 2019

Here's the log after it failed with rear-2.2:

rear_luks_error_log

@gozora
Copy link
Member

gozora commented Aug 11, 2019

heh, this is even worse ... Never mind it was worth a try ;-).

Could you provide debug log from session executed with your most recent ReaR (guess it was 2.4)

V.

@adatum
Copy link
Author

adatum commented Aug 11, 2019

Here is the log file from rear -vD recover with RearR 2.5

rear-localhost.log

This time I noticed a couple of message about UserInput. I'm not sure what they mean:
rear_UserInput
Additional notes:

  • Is it normal to be asked for a login? When ReaR finishes booting, it stops at the line in the screenshot with "[OK] Reached target Multi-User." Then if I press enter I see "localhost login:", which accepts any input (notice I typed "blah"). Is this expected behavior? It has been consistent between ReaR 2.2, 2.4, and 2.5.

  • To get rear mkrescue to succeed in creating an ISO, as stated in another issue, I must always remove "multiboot" from I believe line 49 of /usr/share/rear/lib/uefi-functions.sh for ReaR versions 2.2, 2.4, and 2.5, and also "linuxefi" for ReaR 2.2 and 2.4, but not 2.5.

@gozora
Copy link
Member

gozora commented Aug 11, 2019

@adatum
Your problem is on line 3750 of your rear-localhost.log.
Executed code from 160_include_luks_code.sh, in your particular case: cryptsetup luksFormat --batch-mode --cipher - --key-size --hash --uuid 974b19f8-021a-46b6-a089-a46e06e6e746 --iter-time 2000 --use-random /dev/sda3 does not have any "safety net", meaning if it fails you will get exactly this behavior (prompt for password will be ignored)

Your command cryptsetup luksFormat --batch-mode --cipher - --key-size --hash --uuid 974b19f8-021a-46b6-a089-a46e06e6e746 --iter-time 2000 --use-random /dev/sda3 failed most probably due: --hash: invalid numeric value. Why you don't get correct hash from disklayout.conf, I really don't know.

According change history of 160_include_luks_code.sh, @OliverO2 was the last one doing changes in this file so he might be the most suitable one to help you ...

V.

@adatum
Copy link
Author

adatum commented Aug 11, 2019

Thanks for noticing that @gozora If there's anything you'd like me to try, let me know. Since the system being backed up and recovered is just a test VM, I can try anything even if it's destructive.

Where is a good place to ask questions about ReaR that aren't necessarily issues? Besides the two "additional notes" above, I have a few other questions about ReaR's behavior.

@OliverO2
Copy link
Contributor

Good evening @adatum, having been notified of this issue I took a short look. As @gozora noted correctly there is a missing hash parameter value. The root cause thus seems to be that the hash value is not properly detected by

hash=$(cryptsetup luksDump $source_device | grep "Hash spec" | sed -r 's/^.+:\s*(.+)$/\1/')

That code exists since its initial implementation on 2011, so its not something I was directly involved in. But maybe you could shed some light on it by running this command on the original system:

sudo cryptsetup luksDump /dev/sda3

The output is expected to look similar to this, in particular the line starting with Hash spec::

LUKS header information for /dev/sda3

Version:       	1
Cipher name:   	aes
Cipher mode:   	xts-plain64
Hash spec:     	sha256
Payload offset:	4096
MK bits:       	512
[...]

@adatum
Copy link
Author

adatum commented Aug 11, 2019

Thank you for the quick reply @OliverO2. Here is the luksDump. The hash spec is sha256 but notice that this is using LUKS version 2, which is now the default on recent Fedora releases. Could that have anything to do with the problem?

$ sudo cryptsetup luksDump /dev/sda3
LUKS header information
Version:       	2
Epoch:         	3
Metadata area: 	16384 [bytes]
Keyslots area: 	16744448 [bytes]
UUID:          	974b19f8-021a-46b6-a089-a46e06e6e746
Label:         	(no label)
Subsystem:     	(no subsystem)
Flags:       	(no flags)

Data segments:
  0: crypt
	offset: 16777216 [bytes]
	length: (whole device)
	cipher: aes-xts-plain64
	sector: 512 [bytes]

Keyslots:
  0: luks2
	Key:        512 bits
	Priority:   normal
	Cipher:     aes-xts-plain64
	Cipher key: 512 bits
	PBKDF:      argon2i
	Time cost:  4
	Memory:     973984
	Threads:    4
	Salt:       af 33 7e 3b 6c bb 55 dc e3 dc 2b 07 c5 9e c3 6d 
	            f2 c9 08 be 2f 1d 8b 78 8a 33 65 90 41 e3 05 10 
	AF stripes: 4000
	AF hash:    sha256
	Area offset:32768 [bytes]
	Area length:258048 [bytes]
	Digest ID:  0
Tokens:
Digests:
  0: pbkdf2
	Hash:       sha256
	Iterations: 100361
	Salt:       d9 30 b6 7f 60 d0 e0 19 39 f6 a2 38 ae 22 88 43 
	            1e 5c 74 75 e6 b5 dd db a9 e7 29 1a 74 64 9c 0f 
	Digest:     ae 06 29 5f 71 49 bd c8 75 de 53 e8 95 94 d3 38 
	            57 43 5f 0e 1e ac 6d 59 fb 34 a3 97 e4 5a 94 0c 

@OliverO2
Copy link
Contributor

Yes, it seems like LUKS version 2 requires its own port within ReaR: The output format of cryptsetup luksDump differs significantly between the two versions, and that might be just the tip of the iceberg. One would have to analyze the implications of the version changes step by step to make ReaR save and reproduce a LUKS 2 setup correctly.

Maybe you have the option of just staying with LUKS version 1 for now until version 2 is more established and someone would come up with a port for ReaR.

@adatum
Copy link
Author

adatum commented Aug 11, 2019

Thanks for confirming.

This complicates the situation. Of two systems I want to use with ReaR, one is using LUKS2.

LUKS2 has been supported on Fedora since 29, and is the default since Fedora 30.

LUKS allows converting between versions 1 and 2 under certain conditions:
https://www.saout.de/pipermail/dm-crypt/2017-December/005771.html

* In-place conversion form LUKS1

  To allow easy testing and transition to the new LUKS2 format, there is a new
  convert command that allows in-place conversion from the LUKS1 format and,
  if there are no incompatible options, also conversion back from LUKS2
  to LUKS1 format.

  Note this command can be used only on some LUKS1 devices (some device header
  sizes are not supported).
  This command is dangerous, never run it without header backup!
  If something fails in the middle of conversion (IO error), the header
  is destroyed. (Note that conversion requires move of keyslot data area to
  a different offset.)

  To convert header in-place to LUKS2 format, use
  $ cryptsetup convert <device> --type luks2

  To convert it back to LUKS1 format, use
  $ cryptsetup convert <device> --type luks1

  You can verify LUKS version with luksDump command.
  $ cryptsetup luksDump <device>

  Note that some LUKS2 features will make header incompatible with LUKS1 and
  conversion will be rejected (for example using new Argon2 PBKDF or integrity
  extensions). Some minor attributes can be lost in conversion.

Unfortunately that last point combined with Fedora's defaults prevents converting back to LUKS1. Here is the output from my conversion attempt on the VM with the luksDump from above:

$ sudo cryptsetup convert /dev/sda3 --type luks1

WARNING!
========
This operation will convert /dev/sda3 to LUKS1 format.


Are you sure? (Type uppercase yes): YES
Cannot convert to LUKS1 format - keyslot 0 is not LUKS1 compatible.

@gozora
Copy link
Member

gozora commented Aug 11, 2019

@OliverO2 thanks for finding culprit!

Since fixing this issue would require some additional work, I'm assigning "need sponsorship" label and changing title of this issue.

If anyone wants to sponsor fix for this issue, please drop message to this thread to avoid double work.

V.

@gozora gozora added bug The code does not do what it is meant to do needs sponsorship This issue will not get solved on a voluntary base by ReaR upstream. labels Aug 11, 2019
@gozora gozora changed the title Recovery script does not prompt for LUKS password and fails Failed to recover LUKS version 2 Aug 11, 2019
@gozora gozora added this to the ReaR future milestone Aug 11, 2019
@jsmeix jsmeix added enhancement Adaptions and new features and removed bug The code does not do what it is meant to do labels Aug 12, 2019
@jsmeix
Copy link
Member

jsmeix commented Aug 12, 2019

I think in general missing support for a newer and incompatible version
of some software cannot be a bug in ReaR but is an enhancement.

@adatum
Copy link
Author

adatum commented Aug 18, 2019

In the meantime, a workaround is to convert LUKS2 to LUKS1, which is by no means guaranteed to succeed, as "it depends" on many factors such as what features of LUKS2 are being used.

However this worked for me:

  • Boot from an environment other than the LUKS2 volume that is to be converted
  • Do cryptsetup luksConvertKey --pbkdf=pbkdf2 [/device/with/luks2] if for example argon2i is used (default on Fedora), which LUKS1 does not support.
  • Do cryptsetup convert [/device/with/luks2] --type luks1

https://unix.stackexchange.com/questions/535077/convert-luks2-back-to-luks-version-1/535081#535081

jsmeix added a commit that referenced this issue Apr 27, 2020
Error out during "rear mkrescue/mkbackup" when LUKS version 2 is used.
LUKS version 2 is not suppported, see #2204
When LUKS version 2 is used it fails at least to determine the hash value
so we use an empty hash value as a simple test if gathering crypt information
was successful and error out if not.
@jsmeix
Copy link
Member

jsmeix commented Apr 27, 2020

With #2381
we detect at least when gathering crypt information
was not successful because LUKS 2 is used
instead of let the code blindly proceed resulting
wrong (incomplete) entries in disklayout.conf
that let later (when it is too late) "rear recover" fail.

The whole LUKS code needs to be overhauled to be more fail safe.
Currently it blindly proceeds basically everywhere, e.g. also
layout/prepare/GNU/Linux/160_include_luks_code.sh
blindly adds cryptsetup_options independent of whether or not
the resulting cryptsetup command is at least syntactically correct.

In
https://github.com/rear/rear/blob/master/doc/user-guide/06-layout-configuration.adoc
the LUKS Devices syntax description in disklayout.conf
indicates that most parameters are optional but it seems this is not the case.

jsmeix added a commit that referenced this issue Apr 27, 2020
Error out during "rear mkrescue/mkbackup" when LUKS version 2 is used.
LUKS version 2 is not suppported, see #2204
When LUKS version 2 is used it fails at least to determine the hash value
so we use an empty hash value as a simple test if gathering crypt information
was successful and error out if not.
@github-actions
Copy link

Stale issue message

jsmeix added a commit that referenced this issue Jun 30, 2020
Added "--type luks1" to the default LUKS_CRYPTSETUP_OPTIONS.
Because LUKS2 is not supported by ReaR (cf. #2204)
the option '--type luks1' is needed to enforce the LUKS1 header format
because the default header format is LUKS1 with cryptsetup < 2.1.0
but LUKS2 with cryptsetup ≥ 2.1.0 (cf. #2432)
to ensure LUKS1 gets recreated as LUKS1 also with with newer cryptsetup versions.
@github-actions github-actions bot closed this as completed Jul 4, 2020
@mannp
Copy link

mannp commented Jul 4, 2020

Reading through the numerous updates to the thread, it seems LUKS2 is not supported, but the issue has been closed?

Is LUKS2 now supported?

Thanks for any clarifications 👍

@gdha
Copy link
Member

gdha commented Jul 4, 2020

@mannp LUKS2 is not (yet) supported so far. Cold issues are getting automatically closed by GitHub Bot as it makes no sense if nothing happens (after 90 days) to keep them open. I f you feel it is so important you cannot live without it then sponsoring an issue is a valid option to keep it alive. Or, even better preparing a pull request or adding feedback to the issue which could be useful to get it fixed by someone.

@mannp
Copy link

mannp commented Jul 4, 2020

@mannp LUKS2 is not (yet) supported so far. Cold issues are getting automatically closed by GitHub Bot as it makes no sense if nothing happens (after 90 days) to keep them open. I f you feel it is so important you cannot live without it then sponsoring an issue is a valid option to keep it alive. Or, even better preparing a pull request or adding feedback to the issue which could be useful to get it fixed by someone.

Thanks @gdha I assumed as much, but as it had been open for almost a year I wondered why the bot closed now....updated 12 days ago....

Anyway, I have not been using rear as all my machines are luks2, just using alternatives :)

All the best.

@adatum
Copy link
Author

adatum commented Jul 4, 2020

I have not been using rear as all my machines are luks2, just using alternatives

@mannp What alternatives have you found?

One way is to convert LUKS2 to LUKS1 #2204 (comment) to be able to use ReaR.

I find it unfortunate to close valid issues simply because they haven't been addressed in X days.

@mannp
Copy link

mannp commented Jul 4, 2020

I have not been using rear as all my machines are luks2, just using alternatives

@mannp What alternatives have you found?

One way is to convert LUKS2 to LUKS1 #2204 (comment) to be able to use ReaR.

I find it unfortunate to close valid issues simply because they haven't been addressed in X days.

Hi @adatum, not wanting to take anything away from ReaR, as I think it is a great idea

I use timeshift [https://github.com/teejee2008/timeshift] which is not perhaps a direct comparison, but is does support LUKS2 and has bailed me out on one occasion. Using it with arch it takes care of snapshots in between pacman updates.

That said I did subscribe to this thread as I would have liked another backup of my backup and I do use borg backup [https://github.com/borgbackup/borg] but only for user files, not bare metal restores.

That's where ReaR would be an great fit being able to use borg.

Hope that helps :)

@kalos
Copy link

kalos commented Sep 12, 2020

With the patch #2381, I get an error even if I exclude the device with LUKS2.

In this way, if I have mounted an (even external) LUKS2 device, which I have no intention to backing up, the creation of the layout fails:
ERROR: No hash value for LUKS device 'XXX_crypt' at '/dev/sdb1' (only LUKS version 1 is supported)

@jsmeix
Copy link
Member

jsmeix commented Sep 15, 2020

@kalos
please report your case as a new issue via
https://github.com/rear/rear/issues/new
and provide the information we need as asked for there.
In particular I cannot guess what you had specified in
your etc/rear/local.conf to exclude the not needed thing
and how lsblk looks like on your particular system.
Additionally provide your var/lib/rear/layout/disklayout.conf
and var/lib/rear/layout/disktodo.conf files.

@jsmeix
Copy link
Member

jsmeix commented Sep 15, 2020

#2204 (comment)
is LUKS2 error even if the device is excluded reported via
#2491

jsmeix added a commit that referenced this issue Oct 28, 2020
Added initial LUKS2 support, see #2204
Added new parameter 'type' to 'crypt' keyword used in disklayout.conf.
Using this parameter allows to recreate the same version of LUKS as on the original system.
Added LUKS version detection, parsing depending on version and usage of 'type' parameter.
@jsmeix jsmeix added fixed / solved / done and removed needs sponsorship This issue will not get solved on a voluntary base by ReaR upstream. no-issue-activity labels Oct 28, 2020
@jsmeix jsmeix modified the milestones: ReaR future, ReaR v2.7 Oct 28, 2020
@jsmeix
Copy link
Member

jsmeix commented Oct 28, 2020

With #2504 merged
initial LUKS2 support was added tom ReaR.

@vcrhonek
thank you for your valuable contribution to ReaR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Adaptions and new features fixed / solved / done
Projects
None yet
Development

No branches or pull requests

7 participants