sgx-psw: init package and module #148593

veehaitch · 2021-12-04T14:27:20Z

Motivation for this change

sgx-sdk: create sgx dir and move
sgx-psw: init at 2.14.100.2
sgx-sdk, sgx-psw: add debug argument
nixos/sgx: add module
nixosTests.aesmd: init

Supersedes #129432

Things done

nixos/modules/services/security/aesmd.nix

veehaitch · 2021-12-04T22:46:02Z

# nix develop "github:yaxitech/nix-linux-sgx/sgxs-tools" --command "sgx-detect"
SGX_SDK         = /nix/store/m3sna34srcj8s2q98kcz43aly57wxynk-sgx-sdk-2.14.100.2
SGX_PSW         = /nix/store/17zbdsmmc1w11maxqbhff4i6wc7bn5ck-sgx-psw-2.14.100.2
LD_LIBRARY_PATH = /nix/store/17zbdsmmc1w11maxqbhff4i6wc7bn5ck-sgx-psw-2.14.100.2/lib:/nix/store/m3sna34srcj8s2q98kcz43aly57wxynk-sgx-sdk-2.14.100.2/lib
Detecting SGX, this may take a minute...
✔  SGX instruction set
  ✔  CPU support
  ✔  CPU configuration
  ✔  Enclave attributes
  ✔  Enclave Page Cache
  SGX features
    ✘  SGX2  ✘  EXINFO  ✘  ENCLV  ✘  OVERSUB  ✘  KSS
    Total EPC size: 93.5MiB
✔  Flexible launch control
  ✔  CPU support
  ？ CPU configuration
  ✔  Able to launch production mode enclave
✔  SGX system software
  ✔  SGX kernel device (/dev/sgx/enclave)
  ✔  libsgx_enclave_common
  ✔  AESM service
  ✔  Able to launch enclaves
    ✔  Debug mode
    ✔  Production mode
    ✔  Production mode (Intel whitelisted)

You're all set to start running SGX programs!

pkgs/os-specific/linux/sgx/psw/default.nix

pkgs/os-specific/linux/sgx/sdk/default.nix

blitz

This looks very nice! Thanks for bringing SGX forward on NixOS. 👍

RealityAnomaly

LGTM

tomberek · 2021-12-06T16:17:58Z

nixos/modules/services/security/aesmd.nix

+
+    users.groups = {
+      "${cfg.group}" = { };
+      "sgx_prv" = { };


I'm still curious about https://github.com/systemd/systemd-stable/blob/main/rules.d/50-udev-default.rules.in#L43 and if we need the "sgx" group or how "sgx_prv" is different.

The sgx group is for managing access to the application enclave device, i.e., /dev/sgx_enclave. The sgx_prv group is for the provisioning device, i.e., /dev/sgx_provision. Using the group sgx for sgx_enclave and the group sgx_prv for sgx_provision is a convention which we should follow.

Access to the provisioning device implies access to the CPU-unique provisioning certificate key. Therefore, a user which is allowed to create application enclaves usually should not be allowed to create provisioning enclaves. The AESM service is a privileged exception as it provides remote attestation.

What is the right module in which to create the sgx group? I keep bringing this up because the existence or nonexistence of this group caused some problems in systemd startup in stage2 when there were differing bios settings.

Before #148649, we did not create the sgx group for the systemd-bundled udev rule you mentioned above. This led to a warning during boot which is now resolved. We currently take care of the sgx_prv group in this PR only: create the group (the position you annotated) and the AESM-bundled udev rules assign permissions and create symlinks.

As an alternative, we could take an approach similar to #148649 and create the sgx_prv and the respective udev rule (change group to sgx_prv, assign permissions 0660) independent of having the AESM service enabled or not. I think that would make sense and would leave us with the following permissions:

# stat -c '%U %G %a %n' /dev/sgx_* root sgx 660 /dev/sgx_enclave root sgx_prv 660 /dev/sgx_provision

Anyone who should be allowed to create application enclaves joins the sgx group. Anyone who should be allowed to create provisioning enclaves (for example the AESM service of this PR) joins the sgx_prv group.

~~Maybe something like veehaitch@0666d1b?~~

See latest commits.

blitz · 2021-12-06T22:54:24Z

Note for other reviewers: This needs the changes from #147693 to work. It has already been merged, but still sits in staging.

veehaitch · 2021-12-06T23:27:25Z

Note for other reviewers: This needs the changes from #147693 to work. It has already been merged, but still sits in staging.

If you expect a fully working setup on your SGX-capable bare-metal hardware by just setting services.aesmd.enable = true in your NixOS configuration, you need #147693, that's right. I, for example, also run the AESM service within a systemd-nspawn container on a non-NixOS host which uses the DCAP driver. The host binds /dev/sgx_{enclave,provision} to the container and, therefore, does not need kernel support for it.

This PR is certainly more useful with #147693 being part of the master branch but I wouldn’t say it's blocked by it.

blitz · 2021-12-07T08:20:53Z

@veehaitch True, given that everything comes together anyhow when staging is merged, there is no reason to keep this PR sitting around. :)

Meanwhile, I've been able to test this on a Intel Celeron J4005 and everything seems to be working just fine! 🥳

Detecting SGX, this may take a minute...
✔  SGX instruction set
  ✔  CPU support
  ✔  CPU configuration
  ✔  Enclave attributes
  ✔  Enclave Page Cache
  SGX features
    ✔  SGX2  ✔  EXINFO  ✘  ENCLV  ✘  OVERSUB  ✘  KSS  
    Total EPC size: 94.0MiB
✔  Flexible launch control
  ✔  CPU support
  ？ CPU configuration
  ✔  Able to launch production mode enclave
✔  SGX system software
  ✔  SGX kernel device (/dev/sgx/enclave)
  ✔  libsgx_enclave_common
  ✔  AESM service
  ✔  Able to launch enclaves
    ✔  Debug mode
    ✔  Production mode
    ✔  Production mode (Intel whitelisted)

You're all set to start running SGX programs!

veehaitch · 2021-12-08T13:07:22Z

Following a discussion with @tomberek, I decided to take a different approach regarding the users and groups involved. I considered opening another PR following the merge of this one but I think it makes sense to discuss and ideally incorporate the changes in this PR already:

Add a new config options attribute hardware.cpu.intel.sgx.provision. If enabled = true, it installs a udev rule to set the user, group and mode for the SGX provisioning device (/dev/sgx_provision). The user defaults to root, the group to sgx_prv and the mode to 0660. As a result, a user has to join the configured group to access the provisioning device. I didn't deem a dedicated release note entry necessary for the module. I'm happy to create one if you want me to.
Cherry-picked nixos: add sgx group with gid 304 #148649 (currently not part of the PR's base) which creates and assigns the group sgx to the sgx_enclave device with permissions 0660. That means, a user has to join the sgx group to create SGX application enclaves.
To reflect these changes, the AESM service now runs as a dynamically allocated systemd user (DynamicUser=) with a primary group of sgx and the supplementary group sgx_prv. The runtime directory is set to 0750 to limit access to the AESM UNIX socket to members of the sgx group. Set the state directory permissions to 0700 to only allow the service user to access its contents.
As a result, we don't need the AESM-bundled udev rules anymore. Those rules set the sgx_enclave device mode to 0666 allowing anybody to create SGX application enclaves. This is actually a bit more liberal than I'd like and also overrides the systemd mode of 0660. The recent changes keep the systemd udev rule for sgx_enclave and adapt a similar approach for sgx_provision.

I'm going to rebase those commits as soon as I heard back from you. I didn't do that yet to make it easier to review the latest changes.

blitz · 2021-12-08T13:56:09Z

@veehaitch Sounds reasonable to me. 👍

Co-authored-by: Alex Zero <joseph@marsden.space>

veehaitch · 2021-12-10T09:46:44Z

I'm going to rebase those commits as soon as I heard back from you. I didn't do that yet to make it easier to review the latest changes.

Rebased.

blitz

👍

veehaitch · 2021-12-11T10:23:26Z

I think we're good to go. Any thoughts, @Mic92?

Mic92 · 2021-12-13T14:16:27Z

Thanks. Looks solid!

github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: changelog 8.has: documentation 8.has: module (update) This PR changes an existing module in `nixos/` labels Dec 4, 2021

veehaitch force-pushed the sgx-psw branch from 933e19c to 6fce010 Compare December 4, 2021 14:30

veehaitch commented Dec 4, 2021

View reviewed changes

nixos/modules/services/security/aesmd.nix Outdated Show resolved Hide resolved

ofborg bot added the 8.has: package (new) This PR adds a new package label Dec 4, 2021

ofborg bot requested a review from RealityAnomaly December 4, 2021 14:50

ofborg bot added 11.by: package-maintainer This PR was created by the maintainer of the package it changes 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin 10.rebuild-linux: 1-10 labels Dec 4, 2021

veehaitch force-pushed the sgx-psw branch from 6fce010 to 82f1b3f Compare December 4, 2021 19:16

veehaitch requested a review from Mic92 December 4, 2021 22:46

SuperSandro2000 reviewed Dec 5, 2021

View reviewed changes

pkgs/os-specific/linux/sgx/psw/default.nix Outdated Show resolved Hide resolved

pkgs/os-specific/linux/sgx/psw/default.nix Outdated Show resolved Hide resolved

pkgs/os-specific/linux/sgx/psw/default.nix Outdated Show resolved Hide resolved

veehaitch force-pushed the sgx-psw branch from 82f1b3f to 91de9df Compare December 5, 2021 11:23

veehaitch commented Dec 5, 2021

View reviewed changes

pkgs/os-specific/linux/sgx/psw/default.nix Outdated Show resolved Hide resolved

veehaitch commented Dec 5, 2021

View reviewed changes

pkgs/os-specific/linux/sgx/sdk/default.nix Outdated Show resolved Hide resolved

veehaitch force-pushed the sgx-psw branch from 91de9df to 9490bbb Compare December 5, 2021 12:15

veehaitch mentioned this pull request Dec 6, 2021

nixos/security/sgx: init #129432

Closed

11 tasks

blitz approved these changes Dec 6, 2021

View reviewed changes

RealityAnomaly approved these changes Dec 6, 2021

View reviewed changes

tomberek reviewed Dec 6, 2021

View reviewed changes

blitz requested a review from SuperSandro2000 December 8, 2021 11:22

ofborg bot requested a review from RealityAnomaly December 8, 2021 13:19

veehaitch requested a review from tomberek December 8, 2021 13:20

veehaitch requested a review from blitz December 8, 2021 13:20

veehaitch mentioned this pull request Dec 10, 2021

sgx-sdk, sgx-psw: 2.14 -> 2.15.1 #149948

Merged

13 tasks

veehaitch and others added 6 commits December 10, 2021 10:04

sgx-sdk: create sgx dir and move

f5fcb87

sgx-psw: init at 2.14.100.2

dd79220

Co-authored-by: Alex Zero <joseph@marsden.space>

sgx-sdk, sgx-psw: add debug argument

92c24a1

nixos/intel-sgx: add option for SGX provisioning

ac60e78

nixos/aesmd: add module

0b5c9f8

Co-authored-by: Alex Zero <joseph@marsden.space>

nixosTests.aesmd: init

d6cc0ad

veehaitch force-pushed the sgx-psw branch from 5d37943 to d6cc0ad Compare December 10, 2021 09:20

blitz approved these changes Dec 10, 2021

View reviewed changes

RealityAnomaly approved these changes Dec 13, 2021

View reviewed changes

Mic92 merged commit afa3c99 into NixOS:master Dec 13, 2021

veehaitch mentioned this pull request Jan 10, 2022

sgx-sdk, sgx-psw: improve samples #153237

Merged

13 tasks

rvolosatovs mentioned this pull request Jul 25, 2022

nixos/aesmd: add qcnl config #181347

Open

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sgx-psw: init package and module #148593

sgx-psw: init package and module #148593

veehaitch commented Dec 4, 2021 •

edited

Loading

veehaitch commented Dec 4, 2021

blitz left a comment

RealityAnomaly left a comment

tomberek Dec 6, 2021

veehaitch Dec 6, 2021 •

edited

Loading

tomberek Dec 7, 2021

veehaitch Dec 7, 2021

veehaitch Dec 7, 2021 •

edited

Loading

blitz commented Dec 6, 2021

veehaitch commented Dec 6, 2021 •

edited

Loading

blitz commented Dec 7, 2021

veehaitch commented Dec 8, 2021

blitz commented Dec 8, 2021

veehaitch commented Dec 10, 2021

blitz left a comment

veehaitch commented Dec 11, 2021

Mic92 commented Dec 13, 2021

sgx-psw: init package and module #148593

sgx-psw: init package and module #148593

Conversation

veehaitch commented Dec 4, 2021 • edited Loading

Motivation for this change

Things done

veehaitch commented Dec 4, 2021

blitz left a comment

Choose a reason for hiding this comment

RealityAnomaly left a comment

Choose a reason for hiding this comment

tomberek Dec 6, 2021

Choose a reason for hiding this comment

veehaitch Dec 6, 2021 • edited Loading

Choose a reason for hiding this comment

tomberek Dec 7, 2021

Choose a reason for hiding this comment

veehaitch Dec 7, 2021

Choose a reason for hiding this comment

veehaitch Dec 7, 2021 • edited Loading

Choose a reason for hiding this comment

blitz commented Dec 6, 2021

veehaitch commented Dec 6, 2021 • edited Loading

blitz commented Dec 7, 2021

veehaitch commented Dec 8, 2021

blitz commented Dec 8, 2021

veehaitch commented Dec 10, 2021

blitz left a comment

Choose a reason for hiding this comment

veehaitch commented Dec 11, 2021

Mic92 commented Dec 13, 2021

veehaitch commented Dec 4, 2021 •

edited

Loading

veehaitch Dec 6, 2021 •

edited

Loading

veehaitch Dec 7, 2021 •

edited

Loading

veehaitch commented Dec 6, 2021 •

edited

Loading