New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ZFS + GELI is broken on 10-STABLE #168

Closed
Scarletts opened this Issue Oct 27, 2015 · 16 comments

Comments

Projects
None yet
5 participants
@Scarletts

A ZFS encrypted filesystem created by the installer can't be mounted, regardless of the correctness of the passphrase, it gets rejected as invalid. The 10-STABLE installer never finishes initializing the partition.

I also couldn't get UFS encryption to work on 11-CURRENT because of similar unmountable with incorrect passphrase problems, but I don't have that VM any more and I could have been doing something wrong during the setup. :|

This applies to both versions of HardenedBSD I tried, 11-CURRENT and 10-STABLE. I can reproduce it.

I'm a bit new to the FreeBSD way of doing things, so I'm sorry that this bug report sucks. If you need any more information I'm happy to provide it as long as I'm told how to get that information. :)

@opntr

This comment has been minimized.

Show comment
Hide comment
@opntr

opntr Oct 27, 2015

Member

Could you please test the same scenario with FreeBSD's installer?
We don't changed anything ZFS related code in the HardenedBSD.

Member

opntr commented Oct 27, 2015

Could you please test the same scenario with FreeBSD's installer?
We don't changed anything ZFS related code in the HardenedBSD.

@opntr opntr added the regression label Oct 27, 2015

@Scarletts

This comment has been minimized.

Show comment
Hide comment
@Scarletts

Scarletts Oct 28, 2015

I've tested everything again with the newest 11-CURRENT build (1) and the problem has either magically gone away or I'm guilty of user error.

However, the newest 10-STABLE (2) definitely has problems. These aren't present in the equivalent vanilla FreeBSD build - the ZFS encryption initialization finishes in several seconds, as expected. Maybe relevant: I'm using VirtualBox with 2048mb memory.

I'll close this issue and move it to the hardenedbsd-stable repository.

(1) SHA256 eb17a41da96069572f524aca81497546e64a7f4e26cd6b493ed3b246e61aa840

(2) SHA256 4be32b363e54e21c2ebb0c8c675c6ecf92f7ad23bdbd90f1491d3fbd5c921e64

I've tested everything again with the newest 11-CURRENT build (1) and the problem has either magically gone away or I'm guilty of user error.

However, the newest 10-STABLE (2) definitely has problems. These aren't present in the equivalent vanilla FreeBSD build - the ZFS encryption initialization finishes in several seconds, as expected. Maybe relevant: I'm using VirtualBox with 2048mb memory.

I'll close this issue and move it to the hardenedbsd-stable repository.

(1) SHA256 eb17a41da96069572f524aca81497546e64a7f4e26cd6b493ed3b246e61aa840

(2) SHA256 4be32b363e54e21c2ebb0c8c675c6ecf92f7ad23bdbd90f1491d3fbd5c921e64

@Scarletts Scarletts closed this Oct 28, 2015

@opntr

This comment has been minimized.

Show comment
Hide comment
@opntr

opntr Nov 20, 2015

Member

10-STABLE has the issue, so reopen

Member

opntr commented Nov 20, 2015

10-STABLE has the issue, so reopen

@opntr opntr reopened this Nov 20, 2015

@eagle1maledetto

This comment has been minimized.

Show comment
Hide comment
@eagle1maledetto

eagle1maledetto Nov 29, 2015

I've got the same trouble with ZFS on HardenedBSD 10-STABLE, even without encryption. I've tested it 4 times, and I've managed to install vanilla fBSD on the same guest (with ZFS) without a problem. The installer of HardenedBSD hangs after creating the partitions (again, it doesn't work even without encryption). It hangs here.

schermata 2015-11-29 alle 17 41 45

I've got the same trouble with ZFS on HardenedBSD 10-STABLE, even without encryption. I've tested it 4 times, and I've managed to install vanilla fBSD on the same guest (with ZFS) without a problem. The installer of HardenedBSD hangs after creating the partitions (again, it doesn't work even without encryption). It hangs here.

schermata 2015-11-29 alle 17 41 45

@lattera

This comment has been minimized.

Show comment
Hide comment
@lattera

lattera Nov 29, 2015

Member

I've been able to reproduce this myself. I'll try within the next couple weeks to debug.

Member

lattera commented Nov 29, 2015

I've been able to reproduce this myself. I'll try within the next couple weeks to debug.

@lattera lattera self-assigned this Nov 29, 2015

@opntr opntr added this to the 11-STABLE milestone Dec 5, 2015

@lattera

This comment has been minimized.

Show comment
Hide comment
@lattera

lattera Dec 6, 2015

Member

@eagle1maledetto, can you download a new version of HardenedBSD 11-CURRENT and give it a try? Here's a handy link: http://jenkins.hardenedbsd.org/builds/HardenedBSD-master-amd64-LATEST/ISO-IMAGES/HardenedBSD-11-CURRENT_hardenedbsd-master-amd64-disc1.iso

I've successfully installed it in bhyve:

root@hbsd-dev-throwaway-01:~ # uname -a
FreeBSD hbsd-dev-throwaway-01 11.0-CURRENT-HBSD FreeBSD 11.0-CURRENT-HBSD #288 9fb5343(HEAD): Sat Dec  5 02:34:12 EST 2015     jenkins@nyi-01.build.hardenedbsd.org:/usr/obj/jenkins/workspace/HardenedBSD-master-amd64/sys/HARDENEDBSD  amd64
root@hbsd-dev-throwaway-01:~ # zpool status
  pool: bootpool
 state: ONLINE
  scan: none requested
config:

    NAME        STATE     READ WRITE CKSUM
    bootpool    ONLINE       0     0     0
      vtbd0p2   ONLINE       0     0     0

errors: No known data errors

  pool: rpool
 state: ONLINE
  scan: none requested
config:

    NAME           STATE     READ WRITE CKSUM
    rpool          ONLINE       0     0     0
      vtbd0p4.eli  ONLINE       0     0     0

errors: No known data errors
Member

lattera commented Dec 6, 2015

@eagle1maledetto, can you download a new version of HardenedBSD 11-CURRENT and give it a try? Here's a handy link: http://jenkins.hardenedbsd.org/builds/HardenedBSD-master-amd64-LATEST/ISO-IMAGES/HardenedBSD-11-CURRENT_hardenedbsd-master-amd64-disc1.iso

I've successfully installed it in bhyve:

root@hbsd-dev-throwaway-01:~ # uname -a
FreeBSD hbsd-dev-throwaway-01 11.0-CURRENT-HBSD FreeBSD 11.0-CURRENT-HBSD #288 9fb5343(HEAD): Sat Dec  5 02:34:12 EST 2015     jenkins@nyi-01.build.hardenedbsd.org:/usr/obj/jenkins/workspace/HardenedBSD-master-amd64/sys/HARDENEDBSD  amd64
root@hbsd-dev-throwaway-01:~ # zpool status
  pool: bootpool
 state: ONLINE
  scan: none requested
config:

    NAME        STATE     READ WRITE CKSUM
    bootpool    ONLINE       0     0     0
      vtbd0p2   ONLINE       0     0     0

errors: No known data errors

  pool: rpool
 state: ONLINE
  scan: none requested
config:

    NAME           STATE     READ WRITE CKSUM
    rpool          ONLINE       0     0     0
      vtbd0p4.eli  ONLINE       0     0     0

errors: No known data errors
@eagle1maledetto

This comment has been minimized.

Show comment
Hide comment
@eagle1maledetto

eagle1maledetto Dec 8, 2015

Now it works @lattera . The 11-CURRENT will install on vm (KVM) with no problem at all. We will wait for the patch to be backported to the 10.2-Stable to test it again.

Now it works @lattera . The 11-CURRENT will install on vm (KVM) with no problem at all. We will wait for the patch to be backported to the 10.2-Stable to test it again.

@opntr

This comment has been minimized.

Show comment
Hide comment
@opntr

opntr Jan 17, 2016

Member

10-STABLE is still broken.

Member

opntr commented Jan 17, 2016

10-STABLE is still broken.

@lattera

This comment has been minimized.

Show comment
Hide comment
@lattera

lattera Jan 17, 2016

Member

I hope to start working again on this issue this week.

Member

lattera commented Jan 17, 2016

I hope to start working again on this issue this week.

@opntr opntr modified the milestones: 10-STABLE, 11-STABLE Mar 3, 2016

@opntr opntr changed the title from Disk encryption is broken to ZFS + GELI is broken on 10-STABLE May 2, 2016

@ciaby

This comment has been minimized.

Show comment
Hide comment
@ciaby

ciaby May 8, 2016

Any update on this one? I installed 11-CURRENT with no problems, but I'm running into issues with bhyve, I would like to try 10-STABLE but this looks like a deal-breaker... Is it still broken?

ciaby commented May 8, 2016

Any update on this one? I installed 11-CURRENT with no problems, but I'm running into issues with bhyve, I would like to try 10-STABLE but this looks like a deal-breaker... Is it still broken?

@lattera

This comment has been minimized.

Show comment
Hide comment
@lattera

lattera May 8, 2016

Member

I'll see if I have time this week or next week. When I originally looked into this, I couldn't get kernel debugging with gdb working for some reason. What issues with bhyve are you running into on 11-CURRENT? If it's with grub-bhyve, that will need a secadm rule to disable pageexec/mprotect restrictions.

Member

lattera commented May 8, 2016

I'll see if I have time this week or next week. When I originally looked into this, I couldn't get kernel debugging with gdb working for some reason. What issues with bhyve are you running into on 11-CURRENT? If it's with grub-bhyve, that will need a secadm rule to disable pageexec/mprotect restrictions.

@ciaby

This comment has been minimized.

Show comment
Hide comment
@ciaby

ciaby May 9, 2016

Indeed, that was the issue, grub-bhyve was crashing. I was suspecting something similar, but I didn't manage to get secadm installed/configured. If you know how to do it, I'll be happy to test and document it.

ciaby commented May 9, 2016

Indeed, that was the issue, grub-bhyve was crashing. I was suspecting something similar, but I didn't manage to get secadm installed/configured. If you know how to do it, I'll be happy to test and document it.

@lattera

This comment has been minimized.

Show comment
Hide comment
@lattera

lattera May 9, 2016

Member

As root: pkg install secadm && kldload secadm :)

Then you can take a look at the secadm(8) and secadm.rules(5) manpages. If you have any questions, please let me know.

Member

lattera commented May 9, 2016

As root: pkg install secadm && kldload secadm :)

Then you can take a look at the secadm(8) and secadm.rules(5) manpages. If you have any questions, please let me know.

@lattera

This comment has been minimized.

Show comment
Hide comment
@lattera

lattera May 16, 2016

Member

I found the problem! We build 10-STABLE with INVARIANTS. The invariants support in the kernel is catching something in the ZFS code, causing a deadlock. I'm going to double-check with a vanilla FreeBSD build (but with INVARIANTS enabled) to see if it's a problem specific to HardenedBSD. If the bug exists upstream, I'll go ahead and file a bug report upstream and close this ticket.

Member

lattera commented May 16, 2016

I found the problem! We build 10-STABLE with INVARIANTS. The invariants support in the kernel is catching something in the ZFS code, causing a deadlock. I'm going to double-check with a vanilla FreeBSD build (but with INVARIANTS enabled) to see if it's a problem specific to HardenedBSD. If the bug exists upstream, I'll go ahead and file a bug report upstream and close this ticket.

@lattera

This comment has been minimized.

Show comment
Hide comment
@lattera

lattera May 17, 2016

Member

I just confirmed this is a bug in FreeBSD, not HardenedBSD. I'll open a ticket upstream. This is due to having the INVARIANTS kernel option enabled.

Member

lattera commented May 17, 2016

I just confirmed this is a bug in FreeBSD, not HardenedBSD. I'll open a ticket upstream. This is due to having the INVARIANTS kernel option enabled.

@lattera lattera closed this May 17, 2016

@opntr

This comment has been minimized.

Show comment
Hide comment
Member

opntr commented Oct 8, 2016

@opntr opntr added the done label Oct 10, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment