Adding setcap-wrapper functionality to Nix #16654

ixmatus · Jul 1, 2016

Motivation for this change

Resolves #6768

Setting capabilities on programs is desirable over setuid because it improves the granularity of security. Nix wasn't able to support setcap-wrappers in the style of setuid-wrappers until recently when the Linux 4.4 kernel was added. Linux 4.3 included a capability named Ambient which enables a wrapper program to propagate its capabilities to an execv'ed program.

This PR adds support for setcap-wrappers (with notable improvements to the wrapper program over setuid-wrapper.c) and adds a conditional task that sets ping and ping6 to be setcap wrapped if the kernel is at-least 4.3, if not then we default to the setuid behavior.

Things done

Tested using sandboxing
(nix.useChroot on NixOS,
or option build-use-chroot in nix.conf
on non-NixOS)
Built on platform(s)
- NixOS
- OS X
- Linux
Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
Tested execution of all binary files (usually in ./result/bin/), NOTE: only tested ping and ping6
Fits CONTRIBUTING.md.

I tested both a sub 4.3 kernel (it defaulted to the setuid'ed ping and ping6) and with a 4.4 kernel (it defaulted to setcap'ed ping and ping6).

@ixmatus, thanks for your PR! By analyzing the annotation information on this pull request, we identified @wkennington, @edolstra and @fpletz to be potential reviewers

mention-bot · Jul 1, 2016

@ixmatus, thanks for your PR! By analyzing the annotation information on this pull request, we identified @wkennington, @edolstra and @fpletz to be potential reviewers

Interesting!

I think there are at least 3 commits that are better squashed (MHO).

bjornfor · Jul 1, 2016

Interesting!

I think there are at least 3 commits that are better squashed (MHO).

bjornfor · Jul 1, 2016

bjornfor reviewed Jul 1, 2016

View changes

bjornfor · Jul 1, 2016

nixos/modules/security/setcap-wrappers.nix

+    pkgs.stdenv.mkDerivation {
+      name         = "setcap-wrapper";
+      unpackPhase  = "true";
+      buildInputs  = [ linuxHeaders_4_4 libcap libcap_ng ];


It doesn't strike me as a particularly good idea to hardcode the linux header version here (linuxHeaders_4_4). Isn't there an attribute for the "current" kernel header package?

Yes, you're right, that should just be linuxHeaders I believe. I had to specify a hard kernel header version because stable doesn't have the 4.4 headers in it (yet) I had to copy & inject the headers. I'll fix this.

Yeah a bunch can be squashed but I didn't do it not knowing how you guys
felt. Also we could squash merge the PR once it's approved (unless the
process is merge, not squash)...

On Fri, Jul 1, 2016 at 2:55 PM, Bjørn Forsman notifications@github.com
wrote:

Interesting!

I think there are at least 3 commits that are better squashed (MHO).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16654 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAB3-otwQGsZgiOe9rXYsa5eB82QbCkpks5qRXDFgaJpZM4JDbKB
.

Parnell Springmeyer
parnell@digitalmentat.com | digitalmentat.com | 0xDCCF89258EAD874A
http://pgp.mit.edu/pks/lookup?op=get&search=0xDCCF89258EAD874A

ixmatus · Jul 1, 2016

Yeah a bunch can be squashed but I didn't do it not knowing how you guys
felt. Also we could squash merge the PR once it's approved (unless the
process is merge, not squash)...

On Fri, Jul 1, 2016 at 2:55 PM, Bjørn Forsman notifications@github.com
wrote:

Interesting!

I think there are at least 3 commits that are better squashed (MHO).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16654 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAB3-otwQGsZgiOe9rXYsa5eB82QbCkpks5qRXDFgaJpZM4JDbKB
.

Parnell Springmeyer
parnell@digitalmentat.com | digitalmentat.com | 0xDCCF89258EAD874A
http://pgp.mit.edu/pks/lookup?op=get&search=0xDCCF89258EAD874A

Awesome! Thanks a lot for your work! Will test later and give some more feedback.

fpletz · Jul 2, 2016

Awesome! Thanks a lot for your work! Will test later and give some more feedback.

fpletz · Jul 2, 2016

fpletz reviewed Jul 2, 2016

View changes

fpletz · Jul 2, 2016

nixos/modules/security/setcap-wrappers.nix

+            owner   = "nobody";
+            group   = "nogroup";
+            setcap  = true;
+            capabilities = "cap_net_raw+ep";


Could we maybe use a nix list here? Is the format to specify the capabilities documented somewhere? May be worth mentioning where to find it in the description.

@fpletz the format is documented in cap_from_text(3)

I'll add some documentation about where one would find the capabilities format cap_from_text and setcap expect, great point.

fpletz · Jul 2, 2016

fpletz reviewed Jul 2, 2016

View changes

fpletz · Jul 2, 2016

nixos/modules/security/setcap-wrappers.nix

+{
+  options = {
+    security.setcapCapabilities = mkOption {
+      type    = types.listOf types.attrs;


To validate the the attribute sets you could to use types.listOf types.submodule here.

This should be merged with the setuid wrapper so that we don't have to maintain two very similar programs, and bloat $PATH with yet another directory. Capabilities could be specified as an attribute in security.setuidOwners (which should probably be renamed).

edolstra · Jul 4, 2016

This should be merged with the setuid wrapper so that we don't have to maintain two very similar programs, and bloat $PATH with yet another directory. Capabilities could be specified as an attribute in security.setuidOwners (which should probably be renamed).

@edolstra that's a great idea. It was separated originally simply because we needed the feature and just wanted to inject it into our modules tree but now that you're saying this I definitely agree it should be unified.

Additionally, we made significant improvements to the C wrapper program that the setuid C wrapper program should benefit from so I think it's very reasonable to merge these two features together.

@edolstra would you like me to begin making those changes in this PR? Or close this one and open a new PR with the two features merged? I could also do them in a separate branch and open a PR to this branch so we can review the changes required to merge to the features, then once approved approve this PR.

ixmatus · Jul 5, 2016

@edolstra that's a great idea. It was separated originally simply because we needed the feature and just wanted to inject it into our modules tree but now that you're saying this I definitely agree it should be unified.

Additionally, we made significant improvements to the C wrapper program that the setuid C wrapper program should benefit from so I think it's very reasonable to merge these two features together.

@edolstra would you like me to begin making those changes in this PR? Or close this one and open a new PR with the two features merged? I could also do them in a separate branch and open a PR to this branch so we can review the changes required to merge to the features, then once approved approve this PR.

Just adding my 👍 for KDE, which could use a setcap wrapper (currently using a setuid wrapper).

ttuegel · Jul 15, 2016

Just adding my 👍 for KDE, which could use a setcap wrapper (currently using a setuid wrapper).

Okay cool. Because I haven't had a response on how maintainers want the PR for merging setuid-wrapper and setcap-wrapper I'm just going to branch off of this branch and submit a PR for THIS branch.

ixmatus · Jul 15, 2016

Okay cool. Because I haven't had a response on how maintainers want the PR for merging setuid-wrapper and setcap-wrapper I'm just going to branch off of this branch and submit a PR for THIS branch.

@ixmatus Both options seem fine to me. Please just go ahead. Thanks a lot for your work. 👍

fpletz · Jul 15, 2016

@ixmatus Both options seem fine to me. Please just go ahead. Thanks a lot for your work. 👍

Just want to put a bump on this and say I look forward to having it in mainline.

grahamc · Jul 21, 2016

Just want to put a bump on this and say I look forward to having it in mainline.

Thanks 😃 I'm almost done with merging the two wrapper functionalities
(what's left is testing, actually) so soon!

On Jul 20, 2016 8:33 PM, "Graham Christensen" notifications@github.com
wrote:

Just want to put a bump on this and say I look forward to having it in
mainline.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16654 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAB3-ljiLeUZ8ox_kqN-V9VRuTRUez-oks5qXsyGgaJpZM4JDbKB
.

ixmatus · Jul 21, 2016

Thanks 😃 I'm almost done with merging the two wrapper functionalities
(what's left is testing, actually) so soon!

On Jul 20, 2016 8:33 PM, "Graham Christensen" notifications@github.com
wrote:

Just want to put a bump on this and say I look forward to having it in
mainline.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16654 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAB3-ljiLeUZ8ox_kqN-V9VRuTRUez-oks5qXsyGgaJpZM4JDbKB
.

Sorry I've been MIA everyone - I have this 90% complete, just haven't found a few hours to work on this amongst all of the travel for work I've been doing. I should be able to finish this week so everyone can review.

ixmatus · Aug 1, 2016

Sorry I've been MIA everyone - I have this 90% complete, just haven't found a few hours to work on this amongst all of the travel for work I've been doing. I should be able to finish this week so everyone can review.

@joachifm how much time do I have before the 16.09 release? We've been swamped here but would love to see this in 16.09 so I can prioritize time for this if it's coming soon.

ixmatus · Aug 25, 2016

@joachifm how much time do I have before the 16.09 release? We've been swamped here but would love to see this in 16.09 so I can prioritize time for this if it's coming soon.

@ixmatus the branch-off is scheduled for next week (I don't recall if branch-off is at last day of the month or 1st of the next one, so either wednesday or thursday). Is there anything we can do to help move things along?

joachifm · Aug 25, 2016

@ixmatus the branch-off is scheduled for next week (I don't recall if branch-off is at last day of the month or 1st of the next one, so either wednesday or thursday). Is there anything we can do to help move things along?

@joachifm okay thanks. I just saw the email message hiding in my folder from a month ago!

Well, as this branch stands, it works. But I'm right in the middle of addressing @edolstra's feedback; I should be able to get to that this weekend. Though I will say that there's some production mileage on our own systems of the current formulation that his PR represents.

As a suggestion, it might be "safer" to merge this one and run with it, then come back in after release and refactor. Though I know that's not preferable because it's easy to forget to do that...

I'm proceeding with addressing his feedback, will try to knock it out over the weekend, and merging the two functionalities but if you feel like the above strategy (merging what is here now and works) is better, just let me know.

ixmatus · Aug 26, 2016

@joachifm okay thanks. I just saw the email message hiding in my folder from a month ago!

Well, as this branch stands, it works. But I'm right in the middle of addressing @edolstra's feedback; I should be able to get to that this weekend. Though I will say that there's some production mileage on our own systems of the current formulation that his PR represents.

As a suggestion, it might be "safer" to merge this one and run with it, then come back in after release and refactor. Though I know that's not preferable because it's easy to forget to do that...

I'm proceeding with addressing his feedback, will try to knock it out over the weekend, and merging the two functionalities but if you feel like the above strategy (merging what is here now and works) is better, just let me know.

@ixmatus so the only thing that's currently missing is unifying setcap and setuid? If so, my preference would be to merge it, but that's not my call to make.

joachifm · Aug 26, 2016

@ixmatus so the only thing that's currently missing is unifying setcap and setuid? If so, my preference would be to merge it, but that's not my call to make.

It's hard to make that trade-off.

I've seen many times such refactorings don't happen in future, because bugs are just fixed and there's never enough initiative to rewrite/merge.

This PR should follow the boyscout rule and refactor setuid-wrapper to introduce new functionality.

domenkozar · Aug 28, 2016

It's hard to make that trade-off.

I've seen many times such refactorings don't happen in future, because bugs are just fixed and there's never enough initiative to rewrite/merge.

This PR should follow the boyscout rule and refactor setuid-wrapper to introduce new functionality.

👍 I'm working on finishing up the merged functionality today.

On Aug 28, 2016 7:00 AM, "Domen Kožar" notifications@github.com wrote:

It's hard to make that tradeoff.

I've seen many times such refactorings don't happen in future, because
bugs are just fixed and there's never enough initiative to rewrite/merge.

This PR should follow the boyscout rule
http://programmer.97things.oreilly.com/wiki/index.php/The_Boy_Scout_Rule
and refactor setuid-wrapper to introduce new functionality.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16654 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAB3-sVa2Lk-FQYqj9nxj4XvkoGVRSmuks5qkXhRgaJpZM4JDbKB
.

ixmatus · Aug 28, 2016

👍 I'm working on finishing up the merged functionality today.

On Aug 28, 2016 7:00 AM, "Domen Kožar" notifications@github.com wrote:

It's hard to make that tradeoff.

I've seen many times such refactorings don't happen in future, because
bugs are just fixed and there's never enough initiative to rewrite/merge.

This PR should follow the boyscout rule
http://programmer.97things.oreilly.com/wiki/index.php/The_Boy_Scout_Rule
and refactor setuid-wrapper to introduce new functionality.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16654 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAB3-sVa2Lk-FQYqj9nxj4XvkoGVRSmuks5qkXhRgaJpZM4JDbKB
.

This looks really great. How did the merging go?

Reviewed 6 of 7 files at r1, 1 of 1 files at r2.
Review status: all files reviewed at latest revision, 3 unresolved discussions.

Comments from Reviewable

grahamc · Aug 29, 2016

This looks really great. How did the merging go?

Reviewed 6 of 7 files at r1, 1 of 1 files at r2.
Review status: all files reviewed at latest revision, 3 unresolved discussions.

Comments from Reviewable

Honestly, very slowly. I also moved to a new permanent residence this
weekend so I'm hoping to tackle it tomorrow night.

On Sun, Aug 28, 2016 at 9:40 PM, Graham Christensen <
notifications@github.com> wrote:

This looks really great. How did the merging go?

Reviewed 6 of 7 files at r1, 1 of 1 files at r2.
Review status: all files reviewed at latest revision, 3 unresolved

discussions.

Comments from Reviewable
https://reviewable.io:443/reviews/nixos/nixpkgs/16654#-:-KQJ9Tt9zoI8Px3GH06L:b-cgdg9z

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16654 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAB3-vygx_AhQAFsbe_VofLNz9_IypI8ks5qkkaTgaJpZM4JDbKB
.

Parnell Springmeyer
parnell@digitalmentat.com | digitalmentat.com | 0xDCCF89258EAD874A
http://pgp.mit.edu/pks/lookup?op=get&search=0xDCCF89258EAD874A

ixmatus · Aug 29, 2016

Honestly, very slowly. I also moved to a new permanent residence this
weekend so I'm hoping to tackle it tomorrow night.

On Sun, Aug 28, 2016 at 9:40 PM, Graham Christensen <
notifications@github.com> wrote:

This looks really great. How did the merging go?

Reviewed 6 of 7 files at r1, 1 of 1 files at r2.
Review status: all files reviewed at latest revision, 3 unresolved

discussions.

Comments from Reviewable
https://reviewable.io:443/reviews/nixos/nixpkgs/16654#-:-KQJ9Tt9zoI8Px3GH06L:b-cgdg9z

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#16654 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAB3-vygx_AhQAFsbe_VofLNz9_IypI8ks5qkkaTgaJpZM4JDbKB
.

Parnell Springmeyer
parnell@digitalmentat.com | digitalmentat.com | 0xDCCF89258EAD874A
http://pgp.mit.edu/pks/lookup?op=get&search=0xDCCF89258EAD874A

I'm being given a day by my employer tomorrow (Wednesday) to knock this out, I'll be working on it then. Sorry for the delay everyone.

ixmatus · Aug 30, 2016

I'm being given a day by my employer tomorrow (Wednesday) to knock this out, I'll be working on it then. Sorry for the delay everyone.

Well, the 1st of this month has come - I made progress but I don't feel confident yet in the merged functionality, I want to refactor the code a bit more and do more testing. I think unless the freeze gets extended, this will have to wait until the next release or make it into a patch release.

ixmatus · Sep 1, 2016

Well, the 1st of this month has come - I made progress but I don't feel confident yet in the merged functionality, I want to refactor the code a bit more and do more testing. I think unless the freeze gets extended, this will have to wait until the next release or make it into a patch release.

I'm continuing to work on it today so hopefully I can get something I feel confident about in today - but not sure if that's cutting it too close for the freeze.

ixmatus · Sep 1, 2016

I'm continuing to work on it today so hopefully I can get something I feel confident about in today - but not sure if that's cutting it too close for the freeze.

If the feature is complete and you're willing to fix any bugs in upcoming month, we can merge tgis today.

domenkozar · Sep 1, 2016

If the feature is complete and you're willing to fix any bugs in upcoming month, we can merge tgis today.

@domenkozar yeah I am willing, I'll push to this branch in a few hours after a bit more testing.

ixmatus · Sep 1, 2016

@domenkozar yeah I am willing, I'll push to this branch in a few hours after a bit more testing.

Okay, after an honest effort today I'm deciding to not merge this for 16.09. The changes are too big and touch too many components for me to feel comfortable with it going out this quickly, I think this will need more testing with the participation of other users and other means.

However, I am going to push what I have so far so everyone can see. There are some areas that feel like it could be cleaner too, so it makes sense to not merge this now.

ixmatus · Sep 2, 2016

Okay, after an honest effort today I'm deciding to not merge this for 16.09. The changes are too big and touch too many components for me to feel comfortable with it going out this quickly, I think this will need more testing with the participation of other users and other means.

However, I am going to push what I have so far so everyone can see. There are some areas that feel like it could be cleaner too, so it makes sense to not merge this now.

I've made a PR to fixup some things: #22806.

One more thing remains (which I hope @ixmatus can help with) is the handling of the 'program' attribute.
The option documentation says there is a 'program' attribute, but the example doesn't show it and the option is already an "attrset" (so where to put that attribute?).

For instance, if I want a simple setuid wrapper for dumpcap, is this correct?

security.wrappers = {
  dumpcap.program = "dumpcap";
};

If so, perhaps we can simply drop the 'program' attribute and use the top level attribute name (relative to the option) itself?

bjornfor · Feb 14, 2017

I've made a PR to fixup some things: #22806.

One more thing remains (which I hope @ixmatus can help with) is the handling of the 'program' attribute.
The option documentation says there is a 'program' attribute, but the example doesn't show it and the option is already an "attrset" (so where to put that attribute?).

For instance, if I want a simple setuid wrapper for dumpcap, is this correct?

security.wrappers = {
  dumpcap.program = "dumpcap";
};

If so, perhaps we can simply drop the 'program' attribute and use the top level attribute name (relative to the option) itself?

@bjornfor I will comment on your PR, do you have this comment and question on there?

ixmatus · Feb 14, 2017

@bjornfor I will comment on your PR, do you have this comment and question on there?

Ah, I guess the 'program' attribute is a left-over from the previous interface where the option took a list. Now that it's an attrset I think it makes more sense to simply remove 'program'. Agree?

bjornfor · Feb 14, 2017

Ah, I guess the 'program' attribute is a left-over from the previous interface where the option took a list. Now that it's an attrset I think it makes more sense to simply remove 'program'. Agree?

[...] do you have this comment and question on there?

Huh? (No I only wrote the question here where you see it. You can answer wherever you like.)

bjornfor · Feb 14, 2017

[...] do you have this comment and question on there?

Huh? (No I only wrote the question here where you see it. You can answer wherever you like.)

@bjornfor I kept it for two reasons:

The legacy API, and
It seems in other attrset interfaces elsewhere (I cannot think of a concrete example at the moment) that there is an attribute that can be given on the set to override the name inferred from the top-level key of the set; so it seemed idiomatic to me

I'm inclined to keep it because it seems like it is being used elsewhere in the codebase, e.g:

# nixpkgs/nixos/modules/misc/locate.nix
    security.wrappers = mkIf isMLocate {
      mlocate = {
        group = "mlocate";
        owner = "root";
        permissions = "u+rx,g+x,o+x";
        setgid = true;
        setuid = false;
        program = "locate";
      };
    };

ixmatus · Feb 14, 2017

@bjornfor I kept it for two reasons:

The legacy API, and
It seems in other attrset interfaces elsewhere (I cannot think of a concrete example at the moment) that there is an attribute that can be given on the set to override the name inferred from the top-level key of the set; so it seemed idiomatic to me

I'm inclined to keep it because it seems like it is being used elsewhere in the codebase, e.g:

# nixpkgs/nixos/modules/misc/locate.nix
    security.wrappers = mkIf isMLocate {
      mlocate = {
        group = "mlocate";
        owner = "root";
        permissions = "u+rx,g+x,o+x";
        setgid = true;
        setuid = false;
        program = "locate";
      };
    };

mrobbetts · Feb 15, 2017

This comment has been minimized.

Show comment

Hide comment

mrobbetts Feb 15, 2017

This change breaks the build for me...

Does this look reasonable?

# nixos-rebuild build
building Nix...
building the system configuration...
these derivations will be built:
  /nix/store/1zgpa5mgk3bj8md2nsrqw76k6dkd9jv9-etc-os-release.drv
  /nix/store/2fm7grvhrjlywnlmzw0r5xni3l28hbfg-manual-olinkdb.drv
  /nix/store/kzcaac2n5jlnp8kh7f2lp7d9lrbc2qz6-nixos-manual.drv
  /nix/store/iyxx13wfwpql6v23n3qxxhqww8g00ljx-nixos-help.drv
  /nix/store/mxby4vlazqj1w0ng85lsb5k2v8raw5g5-nixos-version.drv
  /nix/store/my070hw9dm0am7jk7l5sz9l800jn2hh8-nixos-manpages.drv
  /nix/store/7ig9gw8ndf5d26gnyi6akf1cqwcwia72-system-path.drv
  /nix/store/vysvxw6g48y8rza7pqhpa0934rf0dmkb-dbus-conf.drv
  /nix/store/795hz0z1lya3hdx0jpzay5q9b424zdr7-unit-dbus.service.drv
  /nix/store/9dy8xwkkvj4cvbp787dvx3qs4k0k0j58-user-units.drv
  /nix/store/wipfxnqixk738w7cygaahrybyy3mrdra-unit-polkit.service.drv
  /nix/store/jmsphjl5mwj9k2rjab3iml3vgra5yawq-system-units.drv
  /nix/store/qch7c232cxqyip03ij7sgkydvia6bc59-issue.drv
  /nix/store/y2gnkqndrzrim0apg6kvrqdinvsf1nqg-etc.drv
  /nix/store/rf104ryjdwvrr57paic8hsny4bd1fa7f-nixos-system-braid-17.03.git.69794e3.drv
building path(s) ‘/nix/store/w0hv680sxd33ss63hzlkfncr2kmjlb0h-etc-os-release’
building path(s) ‘/nix/store/nbhsv98k0hlayg9i8pyq3qys625rcdbr-issue’
building path(s) ‘/nix/store/5qy3g06qa0prcdfb8rpwscz3vddp980c-nixos-version’
building path(s) ‘/nix/store/8rmn93jw18ycm2nbgg4c2vvlx0jz6h1q-manual-olinkdb’
Writing /nix/store/8rmn93jw18ycm2nbgg4c2vvlx0jz6h1q-manual-olinkdb/manual.db for book(book-nixos-manual)
./man-pages.xml:6247: element para: Relax-NG validity error : Did not expect element para there
./man-pages.xml:3: element variablelist: Relax-NG validity error : Element refsection has extra content: variablelist
./man-pages.xml:29: element refsection: Relax-NG validity error : Element refentry has extra content: refsection
./man-pages.xml:3: element reference: Relax-NG validity error : Element reference failed to validate content
./man-pages.xml fails to validate
builder for ‘/nix/store/2fm7grvhrjlywnlmzw0r5xni3l28hbfg-manual-olinkdb.drv’ failed with exit code 3
cannot build derivation ‘/nix/store/my070hw9dm0am7jk7l5sz9l800jn2hh8-nixos-manpages.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/kzcaac2n5jlnp8kh7f2lp7d9lrbc2qz6-nixos-manual.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/iyxx13wfwpql6v23n3qxxhqww8g00ljx-nixos-help.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/7ig9gw8ndf5d26gnyi6akf1cqwcwia72-system-path.drv’: 3 dependencies couldn't be built
cannot build derivation ‘/nix/store/vysvxw6g48y8rza7pqhpa0934rf0dmkb-dbus-conf.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/wipfxnqixk738w7cygaahrybyy3mrdra-unit-polkit.service.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/795hz0z1lya3hdx0jpzay5q9b424zdr7-unit-dbus.service.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/jmsphjl5mwj9k2rjab3iml3vgra5yawq-system-units.drv’: 2 dependencies couldn't be built
cannot build derivation ‘/nix/store/9dy8xwkkvj4cvbp787dvx3qs4k0k0j58-user-units.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/y2gnkqndrzrim0apg6kvrqdinvsf1nqg-etc.drv’: 3 dependencies couldn't be built
cannot build derivation ‘/nix/store/rf104ryjdwvrr57paic8hsny4bd1fa7f-nixos-system-braid-17.03.git.69794e3.drv’: 2 dependencies couldn't be built
error: build of ‘/nix/store/rf104ryjdwvrr57paic8hsny4bd1fa7f-nixos-system-braid-17.03.git.69794e3.drv’ failed

This change breaks the build for me...

Does this look reasonable?

# nixos-rebuild build
building Nix...
building the system configuration...
these derivations will be built:
  /nix/store/1zgpa5mgk3bj8md2nsrqw76k6dkd9jv9-etc-os-release.drv
  /nix/store/2fm7grvhrjlywnlmzw0r5xni3l28hbfg-manual-olinkdb.drv
  /nix/store/kzcaac2n5jlnp8kh7f2lp7d9lrbc2qz6-nixos-manual.drv
  /nix/store/iyxx13wfwpql6v23n3qxxhqww8g00ljx-nixos-help.drv
  /nix/store/mxby4vlazqj1w0ng85lsb5k2v8raw5g5-nixos-version.drv
  /nix/store/my070hw9dm0am7jk7l5sz9l800jn2hh8-nixos-manpages.drv
  /nix/store/7ig9gw8ndf5d26gnyi6akf1cqwcwia72-system-path.drv
  /nix/store/vysvxw6g48y8rza7pqhpa0934rf0dmkb-dbus-conf.drv
  /nix/store/795hz0z1lya3hdx0jpzay5q9b424zdr7-unit-dbus.service.drv
  /nix/store/9dy8xwkkvj4cvbp787dvx3qs4k0k0j58-user-units.drv
  /nix/store/wipfxnqixk738w7cygaahrybyy3mrdra-unit-polkit.service.drv
  /nix/store/jmsphjl5mwj9k2rjab3iml3vgra5yawq-system-units.drv
  /nix/store/qch7c232cxqyip03ij7sgkydvia6bc59-issue.drv
  /nix/store/y2gnkqndrzrim0apg6kvrqdinvsf1nqg-etc.drv
  /nix/store/rf104ryjdwvrr57paic8hsny4bd1fa7f-nixos-system-braid-17.03.git.69794e3.drv
building path(s) ‘/nix/store/w0hv680sxd33ss63hzlkfncr2kmjlb0h-etc-os-release’
building path(s) ‘/nix/store/nbhsv98k0hlayg9i8pyq3qys625rcdbr-issue’
building path(s) ‘/nix/store/5qy3g06qa0prcdfb8rpwscz3vddp980c-nixos-version’
building path(s) ‘/nix/store/8rmn93jw18ycm2nbgg4c2vvlx0jz6h1q-manual-olinkdb’
Writing /nix/store/8rmn93jw18ycm2nbgg4c2vvlx0jz6h1q-manual-olinkdb/manual.db for book(book-nixos-manual)
./man-pages.xml:6247: element para: Relax-NG validity error : Did not expect element para there
./man-pages.xml:3: element variablelist: Relax-NG validity error : Element refsection has extra content: variablelist
./man-pages.xml:29: element refsection: Relax-NG validity error : Element refentry has extra content: refsection
./man-pages.xml:3: element reference: Relax-NG validity error : Element reference failed to validate content
./man-pages.xml fails to validate
builder for ‘/nix/store/2fm7grvhrjlywnlmzw0r5xni3l28hbfg-manual-olinkdb.drv’ failed with exit code 3
cannot build derivation ‘/nix/store/my070hw9dm0am7jk7l5sz9l800jn2hh8-nixos-manpages.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/kzcaac2n5jlnp8kh7f2lp7d9lrbc2qz6-nixos-manual.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/iyxx13wfwpql6v23n3qxxhqww8g00ljx-nixos-help.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/7ig9gw8ndf5d26gnyi6akf1cqwcwia72-system-path.drv’: 3 dependencies couldn't be built
cannot build derivation ‘/nix/store/vysvxw6g48y8rza7pqhpa0934rf0dmkb-dbus-conf.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/wipfxnqixk738w7cygaahrybyy3mrdra-unit-polkit.service.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/795hz0z1lya3hdx0jpzay5q9b424zdr7-unit-dbus.service.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/jmsphjl5mwj9k2rjab3iml3vgra5yawq-system-units.drv’: 2 dependencies couldn't be built
cannot build derivation ‘/nix/store/9dy8xwkkvj4cvbp787dvx3qs4k0k0j58-user-units.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/y2gnkqndrzrim0apg6kvrqdinvsf1nqg-etc.drv’: 3 dependencies couldn't be built
cannot build derivation ‘/nix/store/rf104ryjdwvrr57paic8hsny4bd1fa7f-nixos-system-braid-17.03.git.69794e3.drv’: 2 dependencies couldn't be built
error: build of ‘/nix/store/rf104ryjdwvrr57paic8hsny4bd1fa7f-nixos-system-braid-17.03.git.69794e3.drv’ failed

So, I guess I need to put the <para>'s in a <note>?

Okay, I think I've fixed it, putting up a PR.

#22815 fixes this build error.

I kept it for two reasons:

Ok. Two things come to mind:

We can make the 'program' attribute optional then. (Or is it already? Looks like it but the documentation doesn't say...)
Why was security.wrapper made an attrset instead of a list (like the old interface)? In a way I can see a set being more suitable (it automatically prevents duplicates), but the new API requires more typing (on keyboard). If we kept the list type we could use a simpler "mkRenamedOptionModule".

The 2nd bullet point is just a small rant on my part, because I feel the list -> attrset change is unrelated to this PR.

bjornfor · Feb 15, 2017

I kept it for two reasons:

Ok. Two things come to mind:

We can make the 'program' attribute optional then. (Or is it already? Looks like it but the documentation doesn't say...)
Why was security.wrapper made an attrset instead of a list (like the old interface)? In a way I can see a set being more suitable (it automatically prevents duplicates), but the new API requires more typing (on keyboard). If we kept the list type we could use a simpler "mkRenamedOptionModule".

The 2nd bullet point is just a small rant on my part, because I feel the list -> attrset change is unrelated to this PR.

The ecryptfs tests are failing now: https://hydra.nixos.org/build/48745458/nixlog/1/raw

globin · Feb 15, 2017

The ecryptfs tests are failing now: https://hydra.nixos.org/build/48745458/nixlog/1/raw

There is more breakage due to the list -> attrset change. E.g. in services.atd.

bjornfor · Feb 15, 2017

There is more breakage due to the list -> attrset change. E.g. in services.atd.

Fixed some breakage in aaac02f, a45821e.

bjornfor · Feb 15, 2017

Fixed some breakage in aaac02f, a45821e.

Why did ping change permissions?

New:

-r-x--x--x 1 root root       17680 Feb 15 08:36 ping

vs old (on release-16.09):

-r-s--x--x 1 root root       12856 Feb 15 08:36 ping

I guess this reflects a change in the defaults of the wrappers?

bjornfor · Feb 15, 2017

Why did ping change permissions?

New:

-r-x--x--x 1 root root       17680 Feb 15 08:36 ping

vs old (on release-16.09):

-r-s--x--x 1 root root       12856 Feb 15 08:36 ping

I guess this reflects a change in the defaults of the wrappers?

Probably ping is using the raw network capability now instead of setuid before or similar

globin · Feb 15, 2017

Probably ping is using the raw network capability now instead of setuid before or similar

@bjornfor

Yea, what Robin said. Removing the s bit when possible is the main idea behind this branch.

You can set before running ping WRAPPER_DEBUG=1 and it should print some debug info on what caps it sets.

teh · Feb 15, 2017

@bjornfor

Yea, what Robin said. Removing the s bit when possible is the main idea behind this branch.

You can set before running ping WRAPPER_DEBUG=1 and it should print some debug info on what caps it sets.

Thanks for the info.

bjornfor · Feb 15, 2017

Thanks for the info.

@bjornfor thank you for catching those mistakes! Is the test suite on Hydra something that can be run, say, on my own EC2 machine so that I could catch these mistakes next time while developing? Do we have developer documentation outlining how to run such a test suite? It would be nice to catch these things earlier.

The list to attrset improvement was suggested by @edolstra in this PR and I like it better than the list-based approach as it keeps it consistent with the other APIs in NixOS that expose a similar interface.

And yes, the only two programs I updated to use the new setcap feature are ping and ping6 since those two are the canonical example of using Linux capabilities. This PR paved the way and I expect more of the programs currently being setuid wrapped need to be setcap wrapped with the appropriate capabilities on them.

ixmatus · Feb 15, 2017

@bjornfor thank you for catching those mistakes! Is the test suite on Hydra something that can be run, say, on my own EC2 machine so that I could catch these mistakes next time while developing? Do we have developer documentation outlining how to run such a test suite? It would be nice to catch these things earlier.

The list to attrset improvement was suggested by @edolstra in this PR and I like it better than the list-based approach as it keeps it consistent with the other APIs in NixOS that expose a similar interface.

And yes, the only two programs I updated to use the new setcap feature are ping and ping6 since those two are the canonical example of using Linux capabilities. This PR paved the way and I expect more of the programs currently being setuid wrapped need to be setcap wrapped with the appropriate capabilities on them.

Is the test suite on Hydra something that can be run, say, on my own EC2 machine [...]

I know very little about how Hydra works, but I know a bunch of nixos tests exist in the nixos/tests directory. A single test can be run with nix-build nixos/tests/<suite>. Not sure if the test coverage is sufficient to catch the errors I faced. (I was simply doing nixos-rebuild boot on my system, unfortunately.)

bjornfor · Feb 15, 2017

Is the test suite on Hydra something that can be run, say, on my own EC2 machine [...]

I know very little about how Hydra works, but I know a bunch of nixos tests exist in the nixos/tests directory. A single test can be run with nix-build nixos/tests/<suite>. Not sure if the test coverage is sufficient to catch the errors I faced. (I was simply doing nixos-rebuild boot on my system, unfortunately.)

Okay, I'll investigate using the test suite next time, thanks. Hmm I was also running nixos-rebuild boot on my EC2 instance, did you have a different configuration.nix with some things enabled that a stock configuration.nix might not?

ixmatus · Feb 15, 2017

Okay, I'll investigate using the test suite next time, thanks. Hmm I was also running nixos-rebuild boot on my EC2 instance, did you have a different configuration.nix with some things enabled that a stock configuration.nix might not?

@ixmatus: Yes, I have enabled extra services. And because I don't have enabled "all", there may very well be some places in nixos that are still broken due to this change... (Hence I wish this API change would have been handled a bit more carefully...)

bjornfor · Feb 15, 2017

@ixmatus: Yes, I have enabled extra services. And because I don't have enabled "all", there may very well be some places in nixos that are still broken due to this change... (Hence I wish this API change would have been handled a bit more carefully...)

@edolstra, @ixmatus: The documentation for security.wrappers say that 'source' is optional, and if unset the program will be looked up in $PATH. That is not true. If 'source' is not set nixos-rebuild fails...

bjornfor · Feb 15, 2017

@edolstra, @ixmatus: The documentation for security.wrappers say that 'source' is optional, and if unset the program will be looked up in $PATH. That is not true. If 'source' is not set nixos-rebuild fails...

@bjornfor this API change / PR has been around for more than six months and I'm completely engaged in helping to fix things. What do you suggest I should do different now or next time to make this more graceful or careful?

Yes that documentation is stale, I will fix it ASAP. Thanks for pointing it out.

ixmatus · Feb 15, 2017

@bjornfor this API change / PR has been around for more than six months and I'm completely engaged in helping to fix things. What do you suggest I should do different now or next time to make this more graceful or careful?

Yes that documentation is stale, I will fix it ASAP. Thanks for pointing it out.

Already done: ce0a52f

bjornfor · Feb 15, 2017

Already done: ce0a52f

Also, maintainers appeared unphased that travis reports a red X on builds of PRs so I was assuming that the builds are broken somewhere and it is acknowledged but not blocking merges, or builds are happening elsewhere (on hydra?) that validate work to be merged.

I'm very open to helping with project short-comings (to make merges safer) and receiving constructive feedback on how I could have gone about something better or different.

ixmatus · Feb 15, 2017

Also, maintainers appeared unphased that travis reports a red X on builds of PRs so I was assuming that the builds are broken somewhere and it is acknowledged but not blocking merges, or builds are happening elsewhere (on hydra?) that validate work to be merged.

I'm very open to helping with project short-comings (to make merges safer) and receiving constructive feedback on how I could have gone about something better or different.

I will try enabling all services and building on my EC2 instance to see if other places fail; I'm going to ag for the wrapper interface and then enable all the services I find and build to see if there are more failures.

ixmatus · Feb 15, 2017

I will try enabling all services and building on my EC2 instance to see if other places fail; I'm going to ag for the wrapper interface and then enable all the services I find and build to see if there are more failures.

Also, I'm sorry my change is breaking a few things. I will try a wider build and clean up whatever fails, I will take ownership of not having thought to do that entirely for all my dev test builds :)

ixmatus · Feb 15, 2017

Also, I'm sorry my change is breaking a few things. I will try a wider build and clean up whatever fails, I will take ownership of not having thought to do that entirely for all my dev test builds :)

@ixmatus: Unfortunately Hydra doesn't build PRs (yet). So the (real) CI only catches problems after the fact. (Travis CI doesn't check much compared to Hydra.) This is IMHO one of the biggest problems with nixpkgs development. (Along with "who decides what"...) There is no automation to prevent untested features entering master, and people generally don't care enough (IMHO) about stability on master. (And there are many committers -- both good and bad!)

I think NixOS/hydra#95 is very important. And whatever we need to get to a place where Hydra is more or less a "gateway" for code to enter master branch.

Some might argue that problems are caught within "reasonable" time with the current setup. But I think we should strive for better quality on master branch :-)

bjornfor · Feb 15, 2017

@ixmatus: Unfortunately Hydra doesn't build PRs (yet). So the (real) CI only catches problems after the fact. (Travis CI doesn't check much compared to Hydra.) This is IMHO one of the biggest problems with nixpkgs development. (Along with "who decides what"...) There is no automation to prevent untested features entering master, and people generally don't care enough (IMHO) about stability on master. (And there are many committers -- both good and bad!)

I think NixOS/hydra#95 is very important. And whatever we need to get to a place where Hydra is more or less a "gateway" for code to enter master branch.

Some might argue that problems are caught within "reasonable" time with the current setup. But I think we should strive for better quality on master branch :-)

@bjornfor: okay I ran a nixos-rebuild build on my EC2 machine with this configuration.nix:

{
  imports = [ <nixpkgs/nixos/modules/virtualisation/amazon-image.nix> ];
  ec2.hvm = true;

  virtualisation.virtualbox.host.enable = true;
  services.locate.enable = true;
  services.exim.enable = true;
  services.xserver.desktopManager.kde5.enable = true;
  services.xserver.desktopManager.enlightenment.enable = true;
  services.cron.enable = true;
  services.fcron.enable = true;
  services.atd.enable = true;

# trace: warning: You must run gale-install in order to generate a domain key.
#  services.gale.enable = true;
  services.smokeping.enable = true;

# - PAM support is currently not implemented.
#  security.duosec.pam.enable = true;
  security.chromiumSuidSandbox.enable = true;

# A Nix error is happening here unrelated to security wrappers I think
#  security.pam.enableEcryptfs.enable = true;
  security.polkit.enable = true;
  security.pam.usb.enable = true;
  programs.light.enable = true;
  programs.kbdlight.enable = true;
}

My build succeeded with those lines commented out and I left a comment with the failure. I believe gale can be fixed so I can enable it and test that. I'm not positive what the PAM support is currently not implemented assertion message is, I will try digging into that to see what's going on there.

And finally pam.enableEcryptfs.enable = true; generated an error unrelated to the wrapper API change, I'll put a PR up to fix that ASAP.

I think I'm covering the build pretty well here. I will switch my system to this config and check all of the executables being wrapped to make sure it went the way it was supposed to. A couple of modules have some tricky logic for conditionally including certain executables in the wrappers attrset so I'll to spend some time tonight picking that apart and making sure those code paths are exercised.

Also, yes, I think better CI would help NixOS a lot; I have experience working with CI for NixOS systems because we've had to make it as smooth as possible at work where we're using NixOS. I'm more than happy to help with this but not quite sure where to start or if I have permissions to do this kind of work or what is currently being worked on.

ixmatus · Feb 15, 2017

@bjornfor: okay I ran a nixos-rebuild build on my EC2 machine with this configuration.nix:

{
  imports = [ <nixpkgs/nixos/modules/virtualisation/amazon-image.nix> ];
  ec2.hvm = true;

  virtualisation.virtualbox.host.enable = true;
  services.locate.enable = true;
  services.exim.enable = true;
  services.xserver.desktopManager.kde5.enable = true;
  services.xserver.desktopManager.enlightenment.enable = true;
  services.cron.enable = true;
  services.fcron.enable = true;
  services.atd.enable = true;

# trace: warning: You must run gale-install in order to generate a domain key.
#  services.gale.enable = true;
  services.smokeping.enable = true;

# - PAM support is currently not implemented.
#  security.duosec.pam.enable = true;
  security.chromiumSuidSandbox.enable = true;

# A Nix error is happening here unrelated to security wrappers I think
#  security.pam.enableEcryptfs.enable = true;
  security.polkit.enable = true;
  security.pam.usb.enable = true;
  programs.light.enable = true;
  programs.kbdlight.enable = true;
}

My build succeeded with those lines commented out and I left a comment with the failure. I believe gale can be fixed so I can enable it and test that. I'm not positive what the PAM support is currently not implemented assertion message is, I will try digging into that to see what's going on there.

And finally pam.enableEcryptfs.enable = true; generated an error unrelated to the wrapper API change, I'll put a PR up to fix that ASAP.

I think I'm covering the build pretty well here. I will switch my system to this config and check all of the executables being wrapped to make sure it went the way it was supposed to. A couple of modules have some tricky logic for conditionally including certain executables in the wrappers attrset so I'll to spend some time tonight picking that apart and making sure those code paths are exercised.

Also, yes, I think better CI would help NixOS a lot; I have experience working with CI for NixOS systems because we've had to make it as smooth as possible at work where we're using NixOS. I'm more than happy to help with this but not quite sure where to start or if I have permissions to do this kind of work or what is currently being worked on.

Umm, for future merges I would appreciate some kind of commit cleanup. Ideally into a sequence of reasonably small commits that can be individually reviewed and with clear descriptive commit messages.

I happened to think of this while browsing the log since my last pull to keep myself up to date. Was having a difficult time with commits such as

a26a796 Merging against master - updating smokingpig, rebase was going to be messy
ad8fde5 Andddd more derp
ce36b58 Derp
f64b06a Hmmm
fd97408 It's clearly quite late
61fe8de Silly, should just have one activation script
48a0c5a More fixing
21368c4 Hmm, unnecessary
a4f905a Enhhh I think compile time macros are gross
785684f Ahhh, my compile-time macros confused me...of course they did...
1ad5411 Hmm
e8bec4c Implicit declared function...
a20e657 Fixing

rycee · Feb 15, 2017

Umm, for future merges I would appreciate some kind of commit cleanup. Ideally into a sequence of reasonably small commits that can be individually reviewed and with clear descriptive commit messages.

I happened to think of this while browsing the log since my last pull to keep myself up to date. Was having a difficult time with commits such as

a26a796 Merging against master - updating smokingpig, rebase was going to be messy
ad8fde5 Andddd more derp
ce36b58 Derp
f64b06a Hmmm
fd97408 It's clearly quite late
61fe8de Silly, should just have one activation script
48a0c5a More fixing
21368c4 Hmm, unnecessary
a4f905a Enhhh I think compile time macros are gross
785684f Ahhh, my compile-time macros confused me...of course they did...
1ad5411 Hmm
e8bec4c Implicit declared function...
a20e657 Fixing

+1 for flatting commit history next time, this is also hard to back out

teh · Feb 15, 2017

+1 for flatting commit history next time, this is also hard to back out

I'm more than happy to help with this but not quite sure where to start or if I have permissions to do this kind of work or what is currently being worked on.

Good! I don't know where to start either. IMHO, another big pain point of NixOS is exactly the lack of knowing who has power to do what, and who has the final say in matters... and that some core people are overloaded (or so I've heard).

bjornfor · Feb 15, 2017

I'm more than happy to help with this but not quite sure where to start or if I have permissions to do this kind of work or what is currently being worked on.

Good! I don't know where to start either. IMHO, another big pain point of NixOS is exactly the lack of knowing who has power to do what, and who has the final say in matters... and that some core people are overloaded (or so I've heard).

Yeah, do we squash-merge PRs (or is there a community guideline against that?) That would have at least cleaned it up going into master...

That feedback for general PR hygiene is totally fair and I completely understand the reason, it's painful for me when I have to review PRs that aren't cleaned up or broken into smaller pieces.

This PR has seen a lot of churn (the initial PR had a single clean commit that was squashed) and my personal and work life are such that I don't always have the time I want to devote to OSS projects and I was worried I'd lose momentum again if I started juggling multiple PRs and maintaining those PRs against master.

So, I appreciate people's patience with the messy PR and commit history and I'll try to do better next time, free-time and momentum permitting.

ixmatus · Feb 15, 2017

Yeah, do we squash-merge PRs (or is there a community guideline against that?) That would have at least cleaned it up going into master...

That feedback for general PR hygiene is totally fair and I completely understand the reason, it's painful for me when I have to review PRs that aren't cleaned up or broken into smaller pieces.

This PR has seen a lot of churn (the initial PR had a single clean commit that was squashed) and my personal and work life are such that I don't always have the time I want to devote to OSS projects and I was worried I'd lose momentum again if I started juggling multiple PRs and maintaining those PRs against master.

So, I appreciate people's patience with the messy PR and commit history and I'll try to do better next time, free-time and momentum permitting.

(And there are many committers -- both good and bad!)

I take offense with the statement that any of the committers can be classified into either of these categories! (well, except Eelco, obviously, he is good by definition)

I think NixOS/hydra#95 is very important. And whatever we need to get to a place where Hydra is more or less a "gateway" for code to enter master branch.

Hm, what will be the live non-gridlocked branch then? Some updates have large but rarely used reverse dependencies…

7c6f434c · Feb 15, 2017

(And there are many committers -- both good and bad!)

I take offense with the statement that any of the committers can be classified into either of these categories! (well, except Eelco, obviously, he is good by definition)

I think NixOS/hydra#95 is very important. And whatever we need to get to a place where Hydra is more or less a "gateway" for code to enter master branch.

Hm, what will be the live non-gridlocked branch then? Some updates have large but rarely used reverse dependencies…

I take offense with the statement that any of the committers can be
classified into either of these categories! (well, except Eelco,
obviously, he is good by definition)

Oh, that's not what I meant! (Sorry for bad wording.) I mean that many committers to the main repo is both a blessing and a curse. (I think a hierarchical approach, like the topic specific sub-trees of Linux development is a better model. AFAIK, only Linus has commit access to the main repo.)

Hm, what will be the live non-gridlocked branch then? Some updates have
large but rarely used reverse dependencies…

You mean where to push updates that will cause some regressions that negatively affect some, however few, people? Difficult question, but I think we have to start by knowing the effects of a change before it enters master. Then we can decide on whether that change should go in or not.

We already have a definition of a "working" master branch: it's where Hydra builds pass and a new channel is produced. And this "working" master branch still has many packages in a broken state, but those packages are not blockers.

bjornfor · Feb 16, 2017

I take offense with the statement that any of the committers can be
classified into either of these categories! (well, except Eelco,
obviously, he is good by definition)

Oh, that's not what I meant! (Sorry for bad wording.) I mean that many committers to the main repo is both a blessing and a curse. (I think a hierarchical approach, like the topic specific sub-trees of Linux development is a better model. AFAIK, only Linus has commit access to the main repo.)

Hm, what will be the live non-gridlocked branch then? Some updates have
large but rarely used reverse dependencies…

You mean where to push updates that will cause some regressions that negatively affect some, however few, people? Difficult question, but I think we have to start by knowing the effects of a change before it enters master. Then we can decide on whether that change should go in or not.

We already have a definition of a "working" master branch: it's where Hydra builds pass and a new channel is produced. And this "working" master branch still has many packages in a broken state, but those packages are not blockers.

Oh, that's not what I meant! (Sorry for bad wording.) I mean that many committers to the main repo is both a blessing and a curse. (I think a hierarchical approach, like the topic specific sub-trees of Linux development is a better model. AFAIK, only Linus has commit access to the main repo.)

I think hierarchical model is nice, it also needs more contributors than we have now — I have recently checked one of the Linux releases, and the number of people directly sending changes to Linus Torvalds is larger than the number of committers in NixPkgs. The total number of contributors to a single Linux kernel release is higher than the total number of contributors across NixPkgs history. When some area does have multiple people who can be reasonably declared maintainers, I am all for encouraging the maintainers to set the rules. For most packages I maintain I support people committing improvements if they use the package, tested the improvements and do not know any negatives.

You mean where to push updates that will cause some regressions that negatively affect _some_, however few, people?

You mean known bad effects. In that case we usually provide multiple options in Nixpkgs until nobody admits using them. I ask about running Maxima on fresh SBCL without waiting for Hydra to check that ACL2 didn't get broken by the update. Also, how do we update libxml2 without drowning in merge conflicts.

7c6f434c · Feb 16, 2017

Oh, that's not what I meant! (Sorry for bad wording.) I mean that many committers to the main repo is both a blessing and a curse. (I think a hierarchical approach, like the topic specific sub-trees of Linux development is a better model. AFAIK, only Linus has commit access to the main repo.)

I think hierarchical model is nice, it also needs more contributors than we have now — I have recently checked one of the Linux releases, and the number of people directly sending changes to Linus Torvalds is larger than the number of committers in NixPkgs. The total number of contributors to a single Linux kernel release is higher than the total number of contributors across NixPkgs history. When some area does have multiple people who can be reasonably declared maintainers, I am all for encouraging the maintainers to set the rules. For most packages I maintain I support people committing improvements if they use the package, tested the improvements and do not know any negatives.

You mean where to push updates that will cause some regressions that negatively affect _some_, however few, people?

You mean known bad effects. In that case we usually provide multiple options in Nixpkgs until nobody admits using them. I ask about running Maxima on fresh SBCL without waiting for Hydra to check that ACL2 didn't get broken by the update. Also, how do we update libxml2 without drowning in merge conflicts.

fpletz added the 0.kind: enhancement label Jul 2, 2016

joachifm added this to the 16.09 milestone Aug 19, 2016

ixmatus referenced this pull request Feb 15, 2017
Merged
security-wrapper: Wrap <para> tags in a <note> tag #22815

7c6f434c referenced this pull request Feb 16, 2017
Closed
nixos: i3lock/xlockmore can no longer verify my password through PAM #22862

Mic92 referenced this pull request Feb 21, 2017
Open
add config.services.xserver.windowManager.sway #19663
4 of 7 tasks complete

ixmatus referenced this pull request Mar 7, 2018
Merged
nixos: setuid-wrapper: simplify readlink logic #35893

NixOS/nixpkgs

Join GitHub today

Adding setcap-wrapper functionality to Nix #16654

Conversation

ixmatus commented Jul 1, 2016 • edited Edited 1 time ixmatus edited Jul 6, 2016 (most recent)

Motivation for this change

Things done

This comment has been minimized.

mention-bot commented Jul 1, 2016

This comment has been minimized.

bjornfor commented Jul 1, 2016

bjornfor reviewed Jul 1, 2016 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

ixmatus commented Jul 1, 2016

fpletz added the 0.kind: enhancement label Jul 2, 2016

This comment has been minimized.

fpletz commented Jul 2, 2016 • edited Edited 1 time fpletz edited Jul 2, 2016 (most recent)

fpletz reviewed Jul 2, 2016 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

ixmatus Jul 5, 2016 • edited Edited 1 time ixmatus edited Jul 5, 2016 (most recent)

fpletz reviewed Jul 2, 2016 View changes

This comment has been minimized.

This comment has been minimized.

edolstra commented Jul 4, 2016

This comment has been minimized.

ixmatus commented Jul 5, 2016 • edited Edited 1 time ixmatus edited Jul 5, 2016 (most recent)

This comment has been minimized.

ttuegel commented Jul 15, 2016

This comment has been minimized.

ixmatus commented Jul 15, 2016

This comment has been minimized.

fpletz commented Jul 15, 2016

This comment has been minimized.

grahamc commented Jul 21, 2016

This comment has been minimized.

ixmatus commented Jul 21, 2016

This comment has been minimized.

ixmatus commented Aug 1, 2016

joachifm added this to the 16.09 milestone Aug 19, 2016

This comment has been minimized.

ixmatus commented Aug 25, 2016

This comment has been minimized.

joachifm commented Aug 25, 2016

This comment has been minimized.

ixmatus commented Aug 26, 2016

This comment has been minimized.

joachifm commented Aug 26, 2016

This comment has been minimized.

domenkozar commented Aug 28, 2016 • edited Edited 1 time domenkozar edited Aug 28, 2016 (most recent)

This comment has been minimized.

ixmatus commented Aug 28, 2016

This comment has been minimized.

grahamc commented Aug 29, 2016

This comment has been minimized.

This looks really great. How did the merging go?

discussions.

ixmatus commented Aug 29, 2016

This looks really great. How did the merging go?

discussions.

This comment has been minimized.

ixmatus commented Aug 30, 2016

This comment has been minimized.

ixmatus commented Sep 1, 2016 • edited Edited 1 time ixmatus edited Sep 1, 2016 (most recent)

This comment has been minimized.

ixmatus commented Sep 1, 2016

This comment has been minimized.

domenkozar commented Sep 1, 2016

This comment has been minimized.

ixmatus commented Sep 1, 2016

This comment has been minimized.

ixmatus commented Sep 2, 2016

ixmatus added some commits Jun 30, 2016

This comment has been minimized.

bjornfor commented Feb 14, 2017

This comment has been minimized.

ixmatus commented Feb 14, 2017

ixmatus commented Jul 1, 2016 •

edited

Edited 1 time

ixmatus edited Jul 6, 2016 (most recent)

bjornfor reviewed Jul 1, 2016

View changes

fpletz commented Jul 2, 2016 •

edited

Edited 1 time

fpletz edited Jul 2, 2016 (most recent)

fpletz reviewed Jul 2, 2016

View changes

ixmatus Jul 5, 2016 •

edited

Edited 1 time

ixmatus edited Jul 5, 2016 (most recent)

fpletz reviewed Jul 2, 2016

View changes

ixmatus commented Jul 5, 2016 •

edited

Edited 1 time

ixmatus edited Jul 5, 2016 (most recent)

domenkozar commented Aug 28, 2016 •

edited

Edited 1 time

domenkozar edited Aug 28, 2016 (most recent)

ixmatus commented Sep 1, 2016 •

edited

Edited 1 time

ixmatus edited Sep 1, 2016 (most recent)

ixmatus commented Feb 14, 2017 •

edited

Edited 1 time

ixmatus edited Feb 14, 2017 (most recent)

mrobbetts commented on `69794e3` Feb 15, 2017