Pwntools has GPL Dependencies, is not Pure MIT

[zachriggle]

Pwntools' current mixed licensing (specifically including GPL'ed things) restricts its commercial usage.

Specifically, inclusion of any GPL code requires distribution of all code which uses it. A commercial product that wants to include Pwntools would be required to distribute their source for the commercial product.

We should remove all non-MIT-licensed source, and seek out MIT-licensed alternatives or derivatives.

In particular, we need to remove, or find a new source for the following sets of data. I have included a possible alternative for each.

• pwnlib/data/crcsums.txt
  • I have not found a non-GPL source for this data
  • We should find out if we need a license for the constants themselves, or for the software that uses them. In general, the consensus I've found is that headers as statement-of-fact (i.e., constants) are not copyrightable.
    • Alternately, we can ask for a licensing exception from the Reveng developers.
• pwnlib/data/useragents
  • joecampo/random-user-agent is MIT-licensed
  • Some of the sources we use may already be MIT, but some are certainly GPL
• pwnlib/data/includes/generator/linux (i.e. DietLibc headers)
  • musl libc is MIT-licensed
  • Alternately, we should be able to sanitize this once, and stop distributing the headers alongside pwntools. We can migrate the generation / sanitization to a separate project.

[TethysSvensson]

I agree that this is a good change, however I do not have the time to work on it.

[psifertex]

Not a rush, but we'd like to eventually look into adding pwntools to the default Binary Ninja install which might have prompted this investigation. When we get more serious about it (lots of other things we're working on first unfortunately) if no progress has made I'll submit some PRs. Thanks for putting the list together, @zachriggle it's good to see what would need to change and gives me a good idea of the scope of what's left.

Some additional context about whether constants are covered under copyright (https://books.google.com/books?id=89-B1pTiPw8C&pg=PA88&lpg=PA88&dq=are+constants+copyrightable&source=bl&ots=EEuIgPnwL1&sig=nqPAe_GO219C44c-j3WSGY0w46c&hl=en&sa=X&ved=0ahUKEwjK6L2Si8_OAhWFTSYKHQ5oBXkQ6AEITjAH#v=onepage&q=are%20constants%20copyrightable&f=false) I'd definitely say it's easiest to just ask the authors their interpretation as that tends to be the easiest way to be safe.

[zachriggle]

It also looks like we need to figure out the license status of all of our Pip dependencies -- recursively. I'm not sure of a good way to do this.

[zachriggle]

I created a small utility to recursively check licenses: https://github.com/zachriggle/license_check

From the output of the utility below, we should be OK except for ROPgadget. @lieanu did some work to replace it with amoco, but it is also GPL'ed. angr is BSD 3-clause, so that could work better. In the interim, I wonder if @JonathanSalwan would grant us a one-off MIT license.

$ license_check pwntools
       capstone@3.0.4    -- UNKNOWN
           cffi@1.8.2    -- MIT
   cryptography@1.5      -- BSD or Apache License, Version 2.0
         enum34@1.1.6    -- BSD License
           idna@2.1      -- BSD-like
      ipaddress@1.0.17   -- Python Software Foundation License
           Mako@1.0.4    -- MIT
     MarkupSafe@0.23     -- BSD
       paramiko@2.0.2    -- LGPL
         pluggy@0.3.1    -- MIT license
         psutil@4.3.1    -- BSD
       pwntools@3.0.2    -- Mostly MIT, some GPL/BSD, see LICENSE-pwntools.txt
             py@1.4.31   -- MIT license
         pyasn1@0.1.9    -- BSD
      pycparser@2.14     -- BSD
     pyelftools@0.24     -- Public domain
       Pygments@2.1.3    -- BSD License
       pyserial@3.1.1    -- BSD
        PySocks@1.5.7    -- BSD
python-dateutil@2.5.3    -- Simplified BSD
       requests@2.11.1   -- Apache 2.0
      ROPGadget@5.4      -- GLPv2
            six@1.10.0   -- MIT
            tox@2.3.1    -- http://opensource.org/licenses/MIT
     virtualenv@15.0.3   -- MIT

[zachriggle]

@psifertex can you verify my conclusion about the dependencies above?

[zachriggle]

It looks like Angr has an amicable license, but some of its minor dependencies are GPL. I've requested license addendums from @zardus for ana, cooldict, and idalink.

$ license_check angr
            ana@0.2      -- ??? <--- GPL
           angr@5.6.8.22 -- UNKNOWN <-- BSD 3-clause
angr-only-z3-custom@4.4.1.post4 -- MIT License
       archinfo@5.6.8.22 -- UNKNOWN <-- (Angr)
       bintrees@2.0.4    -- MIT License
     cachetools@1.1.6    -- MIT
       capstone@3.0.4    -- UNKNOWN <-- BSD 3-clause
           cffi@1.8.2    -- MIT
        claripy@5.6.8.22 -- UNKNOWN <-- (Angr)
            cle@5.6.8.22 -- UNKNOWN <-- (Angr)
       cooldict@1.2      -- ??? <-- GPL
      decorator@4.0.10   -- new BSD License
       dpkt-fix@1.7      -- UNKNOWN <-- BSD 3-clause 
         future@0.15.2   -- MIT
        futures@3.0.5    -- BSD
        idalink@0.11     -- UNKNOWN <-- GPLv3
    mulpyplexer@0.7      -- ??? <-- No license!
       networkx@1.11     -- BSD
         pefile@2016.3.28 -- UNKNOWN <--- MIT
        plumbum@1.6.2    -- MIT
    progressbar@2.3      -- LICENSE.txt <--- BSD/GPL dual-license
      pycparser@2.14     -- BSD
     pyelftools@0.24     -- Public domain
          pyvex@5.6.8.25 -- UNKNOWN <--- (Angr)
           rpyc@3.3.0    -- MIT
        simuvex@5.6.8.22 -- UNKNOWN <--- (Angr)

[zardus]

I've updated ana, cooldict, idalink, and mulpyplexer to BSD. However, VEX is GPL, and angr relies on it for binary translation. We're currently in the process of making the binary translation backends pluggable (which would allow something else to be used in the place of VEX when necessary), but that is not yet done.

[zachriggle]

Thanks so much, @zardus! Are the VEX dependencies on the language, or on a pre-existing implementation?

[zardus]

PyVEX is a python wrapper around VEX (in fact, we've licensed the C part of it, https://github.com/angr/pyvex/tree/master/pyvex_c, as GPL). The C part of it absolutely depends on a pre-existing implementation. The Python part of pyvex loads the shared object and translates the result to python (the "language", you could say), and the rest of angr purely uses this "language".

An easy "license firewall" would be to make a quick "pyvex server" and have pyvex ship the translated VEX results (Python objects) over rpyc to the process running angr, when licensing is important. It'd slow things down, but might not be too bad with proper caching (although, to be honest, the ROP analysis is going to suffer the worst performance penalty, by far). By my understanding, this should isolate the GPL and non-GPL components properly, and we can probably knock something like that out fairly quickly.

[zachriggle]

Bummer. I expect VEX goes all the way back up to Valgrind, which is not likely to be relicensed ;-)

I don't know how much process separation gets us -- the issue comes with distributing/using, IIRC.

[zardus]

IANAL, but my understanding is that a commercial entity that distributes software containing GPL components only needs to distribute source for the GPL components themselves, with "GPL component" defined as anything that needs to link against GPL code. If the "license firewall" works, then a distributor of angr would only need to provide the source code for VEX and PyVEX on demand.

GPLv2 (which VEX uses), as far as I know, makes no restrictions on usage.

Again, IANAL and I don't even play one on the internet. Just my understanding, which could be very wrong.

[JonathanSalwan]

In the interim, I wonder if @JonathanSalwan would grant us a one-off MIT license.

Is it good for a BSD one?

Edit: JonathanSalwan/ROPgadget@d0165b4

[zachriggle]

Or BSD, whichever.

[zachriggle]

ROPgadget is now licensed under the BSD license: JonathanSalwan/ROPgadget@d0165b4