[WIP] Add (optional) license white/blacklisting #5892

matthiasbeyer · 2015-01-21T20:44:02Z

[WIP]

This adds optional white- or blacklisting for licenses from the users configuration as I proposed on the ML.

Please tell me what you guys think about this.

(I hope the [ci skip] thing worked here... as this is wip)

wmertens · 2015-01-21T21:02:33Z

pkgs/stdenv/generic/default.nix

+        assert (unfreeOrBroken == "Unfree"
+                || unfreeOrBroken == "Broken"
+                || unfreeOrBroken == "AllowedLicense"
+                || unfreeOrBroken == "BlacklistedLicense");


You could use

builtins.elem x xs Return true if a value equal to x occurs in the list xs, and false otherwise.

to make this shorter. Not sure if it is better from a memory or speed standpoint though.

wmertens · 2015-01-21T21:12:51Z

So right now if you want to allow a specific unfree license, you need to set allowUnfree and blacklist all the ones you don't like. Wouldn't it be better to let allowedLicenses be empty by default and work as a whitelist?

matthiasbeyer · 2015-01-21T21:14:18Z

So, I'm done for today. I would like to hear some more feedback on this and whether this is a good idea or not, before putting effort in history-cleanup.

But now I need some sleep.

My idea was not to change the type of the allowUnfree setting at all, to be backwards compatible. Of course, the expected behaviour needs some discussion here! A combination of all three would be nice, where each option as a sane default value, of course.

wmertens · 2015-01-21T21:19:15Z

Behavior I'd expect:

allowUnfree* behavior is unchanged
whitelistedLicenses enables licenses and overrides allowUnfree*, e.g. allow the Oracle one
blacklistedLicense disables licenses even if they're free, e.g. GPLv3

matthiasbeyer · 2015-01-22T07:55:53Z

@wmertens Of course, that's what I'd like too, but how should they play together? If I do not allow unfree licenses but whitelist the Oracle one and then blacklist it again... is it allowed or not?

If I both black- and whitelist GPLv3, is it allowed or not?

bjornfor · 2015-01-22T08:00:10Z

I'm just jumping in here.

"If I both black- and whitelist GPLv3, is it allowed or not?" -> That's a configuration error that can be caught by an assertion (i.e. it's not allowed).

matthiasbeyer · 2015-01-22T08:04:23Z

"If I both black- and whitelist GPLv3, is it allowed or not?" -> That's a configuration error that can be caught by an assertion (i.e. it's not allowed).

Would be a good idea to do it this way, yes.

wmertens · 2015-01-22T11:03:55Z

Indeed, assert that blacklist and whitelist are mutually exclusive.
Blacklist gets precedence, then whitelist, then unfree. IMHO ;-)

On Thu Jan 22 2015 at 9:04:26 AM Matthias Beyer notifications@github.com
wrote:

"If I both black- and whitelist GPLv3, is it allowed or not?" -> That's a
configuration error that can be caught by an assertion (i.e. it's not
allowed).

Would be a good idea to do it this way, yes.

—
Reply to this email directly or view it on GitHub
#5892 (comment).

matthiasbeyer · 2015-01-22T11:27:23Z

I would do it this way:

(Pseudocode ahead!)


if (unfreeAllowed) then
  if (packageBlacklisted? pkg)
    fail
  else install pkg
else # if unfree not allowed
  if (packageWhitelisted? pkg)
    install pkg
  else fail
endif

wmertens · 2015-01-22T11:55:01Z

Not quite - some companies do not want to use e.g. free GPLv3 software so
they should be able to blacklist free licenses.

On Thu Jan 22 2015 at 12:27:26 PM Matthias Beyer notifications@github.com
wrote:

I would do it this way:

(Pseudocode ahead!)

if (unfreeAllowed) then
if (packageBlacklisted? pkg)
fail
else install pkg
else # if unfree not allowed
if (packageWhitelisted? pkg)
install pkg
else fail
endif

—
Reply to this email directly or view it on GitHub
#5892 (comment).

matthiasbeyer · 2015-01-22T12:19:09Z

@wmertens look at my latest patch. It does it this way:

if (hasUnallowedUnfreeLicense pkg) && !(hasWhitelistedLicense pkg) then
  fail
else if hasBlacklistedLicense pkg then
  fail
[...]

I will patch hasUnfreeLicense because I consider the name hasUnallowedUnfreeLicense better right now.

wmertens · 2015-01-22T13:06:08Z

pkgs/stdenv/generic/default.nix

@@ -16,6 +16,12 @@ let

  allowUnfree = config.allowUnfree or false || builtins.getEnv "NIXPKGS_ALLOW_UNFREE" == "1";

+  # Allowed licenses, defaults to no licenses
+  whitelistedLicenses = config.allowedLicenses or [];


I'd also like to see config.allowedLicenses renamed...

wmertens · 2015-01-22T13:14:45Z

I'd expect

    if hasBlacklistedLicense attrs then
      throw...
    else if (! hasWhitelistedLicense attrs) and (hasDeniedUnfreeLicense attrs) then
      throw...
    else

matthiasbeyer · 2015-01-22T13:54:41Z

That's exactly what's implemented right now, isn't it?

The mutual-exclusive check is added as well, of course.

wmertens · 2015-01-22T15:09:51Z

pkgs/stdenv/generic/default.nix

+    if mutualExclusive whitelistedLicenses blacklistedLicenses then
+      throw ''
+          Package blacklist and whitelist are not mutual exclusive.
+      '';


That semicolon is unneeded (and I'm surprised it even evaluates)

I skip the CI.

wmertens · 2015-01-22T18:08:39Z

You don't test locally? Because I think you didn't define mutEx 😁

matthiasbeyer · 2015-01-23T12:26:37Z

I don't test locally because I do not want to test after every change. I want to test once, but do it right, after we agreed on the actual default behaviour.

wmertens · 2015-01-23T12:35:33Z

well the patch looks good to me so start testing 😉

matthiasbeyer · 2015-01-23T13:36:09Z

@wmertens Okay, can you tell me a way how you'd like this to be tested?

wmertens · 2015-01-23T14:17:19Z

@matthiasbeyer we don't have unit tests for configuration afaik so simply clone your tree locally and try to evaluate few configurations with packages that have proper and denied licenses...

To evaluate, export NIX_PATH=nixpkgs=...your-tree/ and run nixos-rebuild dry-run -v. Make sure it uses the correct nixpkgs tree.

matthiasbeyer · 2015-01-23T14:29:07Z

I was able to run nixos-rebuild dry-run -v, but as far as I understand it, this does not actually warn if packages which are already installed have a license which is blacklisted, should it?

Edit: I just blacklisted all GPLv2.* licenses, and it does not warn or fail that the linux kernel is installed... so this works only when installing new packages, I guess. Same applies to the normal allowUnfree configuration, right?

matthiasbeyer · 2015-01-23T14:38:12Z

With GPLv2 blacklisted and a new environment.systemPackages entry for cups, which is GPLv2, the nixos-rebuild dry-run -v actually works, but it should fail, shouldn't it? :-)

wmertens · 2015-01-23T14:41:03Z

Indeed. The -v shows that it is using your copy of the tree? Start
debugging ;-)

On Fri Jan 23 2015 at 3:38:13 PM Matthias Beyer notifications@github.com
wrote:

With GPLv2 blacklisted and a new environment.systemPackages entry for cups,
which is GPLv2, the nixos-rebuild dry-run -v actually works, but it
should fail, shouldn't it? :-)

—
Reply to this email directly or view it on GitHub
#5892 (comment).

matthiasbeyer · 2015-01-23T14:44:47Z

Somehow, setting the environment variable did not work.

Doing

nixos-rebuild dry-run -v -I nixpkgs=~/nixpkgs

Fails because the linux kernel has a blacklisted license... yay!

I will do more debugging now...

wmertens · 2015-01-23T16:26:41Z

I see a spurious ';'. Also, can you squash everything into one commit?

matthiasbeyer · 2015-01-23T16:39:33Z

I guess the work-in-progress label can be removed.

wmertens · 2015-01-23T17:12:18Z

Can you remove the trace calls, that will also fix the travis build failure.

matthiasbeyer · 2015-01-23T17:16:12Z

@wmertens I'd really love to have these kind of notification, as the users can actually see that there are things which can not be filtered.

Why are there packages without a name?

wmertens · 2015-01-23T17:33:28Z

Well, trace is for debugging, it's not generally used in nixpkgs at this point in time. If you want to let people know which ones are allowed through, implement a disallowEmptyLicense flag 😁.

wmertens · 2015-01-23T18:33:32Z

pkgs/stdenv/generic/default.nix

+        assert (builtins.elem unfreeOrBroken [ "Unfree"
+                                               "Broken"
+                                               "AllowedLicense"
+                                               "BlacklistedLicense"


You're not using BlacklistedLicense? Shouldn't AllowedLicense be taken out and instead use BlacklistedLicense?

matthiasbeyer · 2015-01-24T12:59:14Z

Rebased onto latest master as I guess the latest breakage was not caused by my PR...

wmertens · 2015-01-24T13:42:36Z

pkgs/stdenv/generic/default.nix

-      throwEvalHelp "Unfree" "has an unfree license"
+    if !(mutualExclusive whitelistedLicenses blacklistedLicenses) then
+      throw ''
+          Package blacklist and whitelist are not mutual exclusive.


Two more things : could you name the attributes in this error message and can you squash again?

matthiasbeyer · 2015-01-24T14:27:51Z

@wmertens Can you tell me what causes the travis build failure? Is it caused by one of my patches, actually?

wmertens · 2015-01-24T14:57:09Z

No, master is broken because of the ruby or python work. However the
previous failure was die to name missing on stdenv...

On Sat, Jan 24, 2015, 3:27 PM Matthias Beyer notifications@github.com
wrote:

@wmertens https://github.com/wmertens Can you tell me what causes the
travis build failure? Is it caused by one of my patches, actually?

—
Reply to this email directly or view it on GitHub
#5892 (comment).

wmertens · 2015-01-24T15:01:02Z

Alright it looks good to merge now but can you squash it into a single commit?

matthiasbeyer · 2015-01-24T15:06:14Z

I would actually like to keep the "Remove trace calls by outcommenting" commit as seperate commit. If further work needs to be done in this area, one can simply revert this bit and get this going.

If you really want me to squash it I'll do this, though.

wmertens · 2015-01-24T15:56:17Z

Yes squash please, uncommenting the calls is easy enough.

wmertens · 2015-01-24T19:06:17Z

Awesome, thanks! You might want to tell the mailing list about it :-)

Add (optional) license white/blacklisting

edolstra · 2015-01-26T12:17:59Z

pkgs/stdenv/generic/default.nix

    in
-    if !allowUnfree && isUnfree (lib.lists.toList attrs.meta.license or []) && !allowUnfreePredicate attrs then
-      throwEvalHelp "Unfree" "has an unfree license"
+    if !(mutualExclusive whitelistedLicenses blacklistedLicenses) then


This should be lifted out to prevent it from being evaluated on every call to mkDerivation.

Hmmm should have caught that sorry. @matthiasbeyer feel like tackling this or shall I do that?

@wmertens Would be nice if you'd do it, I'm in the exams phase now and don't know how long it would take if I'd do it.

Done in 0feb19b

jgeerds added 2.status: work-in-progress 9.needs: reporter feedback labels Jan 21, 2015

wmertens reviewed Jan 21, 2015
View reviewed changes

wmertens reviewed Jan 22, 2015
View reviewed changes

wolftune mentioned this pull request Jan 22, 2015

Add per-package whitelisting for non-free packages #5899

Closed

wmertens reviewed Jan 22, 2015
View reviewed changes

matthiasbeyer force-pushed the add-license_selection branch 2 times, most recently from a4e955f to 9a45746 Compare January 23, 2015 16:38

jgeerds removed the 2.status: work-in-progress label Jan 23, 2015

wmertens reviewed Jan 23, 2015
View reviewed changes

matthiasbeyer force-pushed the add-license_selection branch from 92cacac to 7eca1f3 Compare January 24, 2015 12:58

wmertens reviewed Jan 24, 2015
View reviewed changes

matthiasbeyer force-pushed the add-license_selection branch from 7eca1f3 to 5245370 Compare January 24, 2015 14:28

Add (optional) license white/blacklisting

aca361f

matthiasbeyer force-pushed the add-license_selection branch from 5245370 to aca361f Compare January 24, 2015 16:29

wmertens added a commit that referenced this pull request Jan 24, 2015

Merge pull request #5892 from matthiasbeyer/add-license_selection

be799bb

Add (optional) license white/blacklisting

wmertens merged commit be799bb into NixOS:master Jan 24, 2015

matthiasbeyer deleted the add-license_selection branch January 24, 2015 19:15

matthiasbeyer mentioned this pull request Jan 25, 2015

Add documentation for nixpkgs.config settings #5953

Closed

edolstra reviewed Jan 26, 2015
View reviewed changes

This was referenced Oct 27, 2023

[Snyk] Security upgrade axios from 0.24.0 to 1.6.0 stnert/nixpkgs#333

Open

[Snyk] Security upgrade axios from 0.24.0 to 1.6.0 stnert/nixpkgs#334

Open

kaocher82 mentioned this pull request Oct 29, 2023

[Snyk] Security upgrade axios from 0.20.0 to 1.6.0 Xtuden-com/nixpkgs#612

Open

[WIP] Add (optional) license white/blacklisting #5892

[WIP] Add (optional) license white/blacklisting #5892

Conversation

matthiasbeyer commented Jan 21, 2015

Choose a reason for hiding this comment

wmertens commented Jan 21, 2015

matthiasbeyer commented Jan 21, 2015

wmertens commented Jan 21, 2015

matthiasbeyer commented Jan 22, 2015

bjornfor commented Jan 22, 2015

matthiasbeyer commented Jan 22, 2015

wmertens commented Jan 22, 2015

matthiasbeyer commented Jan 22, 2015

wmertens commented Jan 22, 2015

matthiasbeyer commented Jan 22, 2015

Choose a reason for hiding this comment

wmertens commented Jan 22, 2015

matthiasbeyer commented Jan 22, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wmertens commented Jan 22, 2015

matthiasbeyer commented Jan 23, 2015

wmertens commented Jan 23, 2015

matthiasbeyer commented Jan 23, 2015

wmertens commented Jan 23, 2015

matthiasbeyer commented Jan 23, 2015

matthiasbeyer commented Jan 23, 2015

wmertens commented Jan 23, 2015

matthiasbeyer commented Jan 23, 2015

wmertens commented Jan 23, 2015

matthiasbeyer commented Jan 23, 2015

wmertens commented Jan 23, 2015

matthiasbeyer commented Jan 23, 2015

wmertens commented Jan 23, 2015

Choose a reason for hiding this comment

matthiasbeyer commented Jan 24, 2015

Choose a reason for hiding this comment

matthiasbeyer commented Jan 24, 2015

wmertens commented Jan 24, 2015

wmertens commented Jan 24, 2015

matthiasbeyer commented Jan 24, 2015

wmertens commented Jan 24, 2015

wmertens commented Jan 24, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment