Wheels licensing

[pombredanne]

The Pypi wheels (in particular the manylinux ones) contain prebuilts libraries for openblas (bsd) and libgfortran.... (gpl with runtime exception?) but no information about their licenses and where to get the corresponding sources. It would be great to have some clarity in this domain.
The same issue exists with numpy numpy/numpy#8689

[rgommers]

Where were you looking for that information or would you expect to find it?

[pombredanne]

I found that in the Linux wheels in the .lib directory. For instance: libgfortran-ed201abd.so.3.0.0 and libopenblasp-r0-39a31c03.2.18.so

I would have expected to have found somewhere in the wheel some documentation about the licenses and versions of these .so and where I could get the sources (in particular for libgfortran). Ideally I guess these source may be could also be released on Pypi, as a side sdist? Just an idea.

Wheels have unfortunately limited ways to add extra data unless this is package data, so that would likely need to be some package data of sorts I guess?

[rgommers]

I would have expected to have found somewhere in the wheel some documentation about the licenses

That is feasible. Inside the license file, adding info about all vendored code and bundled binary libs would be useful.

Ideally I guess these source may be could also be released on Pypi, as a side sdist? Just an idea.

No, that would be completely impractical. We are not going to redistribute source code of other projects. Such an idea would get even more impractical if you go higher up the stack - would an application using numpy and scipy then redistribute numpy/scipy source code?

Wheels have unfortunately limited ways to add extra data unless this is package data, so that would likely need to be some package data of sorts I guess?

Don't think there's a good place in the .dist-info directory. In LICENSE.txt is the way to go I think.

[pombredanne]

@rgommers you wrote

No, that would be completely impractical. We are not going to redistribute source code of other projects. Such an idea would get even more impractical if you go higher up the stack - would an application using numpy and scipy then redistribute numpy/scipy source code?

I agree it may not be super practical but that would likely be needed to be squeaky clean on the compliance side at least for libgfortran.... which requires source code redistribution and is vendored as a binary.

Don't think there's a good place in the .dist-info directory. In LICENSE.txt is the way to go I think.

For licenses, there is one special place in wheels that lands in dist-info (for a single file see https://bitbucket.org/pypa/wheel/issues/138/include-more-than-one-license-file-or ) which is enabled with this setup.cfg snippet:

[metadata]
license_file = LICENSE.txt

[rgommers]

I agree it may not be super practical but that would likely be needed to be squeaky clean on the compliance side at least for libgfortran.... which requires source code redistribution and is vendored as a binary

do you have any examples of projects doing that? a link should suffice I'd think.

[rgommers]

For licenses, there is one special place in wheels that lands in dist-info

Thanks, didn't know about that. Wouldn't it just duplicate what we already have in scipy/LICENSE.txt though? So adding more info to that LICENSE.txt file is still the way to go?

[pombredanne]

@rgommers you wrote:

do you have any examples of projects doing that? a link should suffice I'd think.

I am doing this: For any thrid-party binaries that I vendor in https://github.com/nexB/scancode-toolkit I release the exact corresponding source code and notes here: https://github.com/nexB/scancode-thirdparty-src

[pombredanne]

Thanks, didn't know about that. Wouldn't it just duplicate what we already have in scipy/LICENSE.txt though?

This is weird and less documented feature. Eventually license documentation is a tad messy ;) and I would like to fix it https://github.com/pombredanne/spdx-pypi-pep/

So adding more info to that LICENSE.txt file is still the way to go?

This sure is. That or a separate file for third-parties is commonly done too. I will be glad to help in any way I can there.

[rgommers]

I agree it may not be super practical but that would likely be needed to be squeaky clean on the compliance side at least for libgfortran.... which requires source code redistribution and is vendored as a binary.

As usual there is lots of conflicting information to be found, but https://www.gnu.org/licenses/gcc-exception-3.1.en.html says nothing about redistribution. It's not clear not me that a link to the source code is not enough and actual redistribution is needed. Do you have an authoritative source?

[pombredanne]

This exception is just that: an exception to the GPL. It does not remove the obligations of source code redistribution for libgfortran itself. The exception just weakens the GPL copyleft to something that is actually gentler that an LGPL such the copyleft does not flow to the calling code (here scipy).

[pombredanne]

The authoritative source is the GPL text here. And I guess some would consider me an authority in this domain too.

[njsmith]

@pombredanne: I disagree with your interpretation. The runtime exception states: "You have permission to propagate a work of Target Code formed by combining the Runtime Library with Independent Modules, even if such propagation would otherwise violate the terms of GPLv3". Our wheel is a combination of the Runtime Library (libgfortran) with Independent Modules (scipy), and that sentence pretty clearly states that the GPL text does not apply.

[rgommers]

It does not remove the obligations of source code redistribution for libgfortran itself.

Independent of whether the GPL applies, what is the definition of redistribution:

• a link to the source (which is a one-liner in LICENSE.txt to the version we used)
• a separate repo with source code on GitHub (like in your example)
• a separate sdist on PyPI (which is what you suggested)
• ...?

I always understood it as allowing anyone who asked access to the source code, which means a link is fine.

[njsmith]

@rgommers: you can't really talk about this independent of the GPL, because it's the GPL text that (if it applies) imposes the requirement of "redistributing source", and defines what that means. Specifically it looks like the details are in section 6 of the GPLv3 (that page doesn't seem to let me link to the section directly, unfortunately).

[pombredanne]

@njsmith You need to consider both the exception and the gpl section7: https://www.gnu.org/licenses/gpl-3.0.html#section7

[pombredanne]

@rgommers

I always understood it as allowing anyone who asked access to the source code, which means a link is fine.

It is fine with the GPL 3 as long as this is something you control IMHO. e.g. you could not just link to the GCC official downloads and to comply with obligations easily it should be available at the same time or place that you get the binaries from, e.g. possibly in the description that ends up on Pypi.

There are other ways but they they would be more burdensome IMHO

[njsmith]

@pombredanne: From a quick look at section 7 I don't see how it changes anything. Can you be more specific about how you think it applies?

On further examination, I'm actually more concerned that the definition of "Target Code" might not stretch as far as I assumed in my comment above. The runtime exception certainly allows statically linking against libgfortran and redistributing without worrying about source code, but for dynamic linking and then shipping the .so I'm not as sure. We probably need an Actual Lawyer to help us out here...

[pombredanne]

@njsmith the way I see most everyone interpret this is that redistributing the runtime and your own code under any license does not violate the GPL but the GPL still applies to the runtime itself and its provisions should still be honored. You can take a more relaxed way to read this, but your downstream users may not agree with you, hence why I would recommend to make it easier and cleaner for them by:

1. redistributing the corresponding sources
2. including the license and exceptions and documenting this.

[rgommers]

to comply with obligations easily it should be available at the same time or place that you get the binaries from, e.g. possibly in the description that ends up on Pypi.

Well, the requirement is to comply, not comply "easily". Polluting PyPI with extra files for many projects no one will ever use is not the answer. What may be easier is for example adding it to a multibuild or manylinux repo, which is then effort spent once and will satisty source code availability for many projects.

[njsmith]

@rgommers: if the GPL applies, then it looks like the requirement is that if the binaries are on PyPI, then we have to keep a link on PyPI ("clear directions next to the object code") saying where to download the source, and make sure that that link remains working in perpetuity. Also anyone who wants to redistribute the wheels has to commit to doing the same.

@matthew-brett: do you happen to know how difficult it would be to arrange to statically link libgfortran in manylinux builds? That would avoid this whole mess...

[njsmith]

(actually I said manylinux above, but really it's all platforms...)

[njsmith]

It might make sense to email licensing@gnu.org to ask their opinion too.

[rgommers]

Okay, have looked at this again. Updated the SciPy license file in gh-7844 for all libraries that are bundled in this repo and in source releases.

Adding license info for OpenBLAS/gfortran unconditionally to that license file is incorrect, because source releases don't include OpenBLAS/gfortran. The correct place should be https://github.com/MacPython/scipy-wheels. I plan to add a LICENSE_LINKED_LIBS or similarly named file to that repo, that can then be appended to LICENSE.txt when a wheel is built. This should ensure that each source/binary distribution we have on PyPI and Github has the correct info in it.

then it looks like the requirement is that if the binaries are on PyPI, then we have to keep a link on PyPI ("clear directions next to the object code") saying where to download the source, and make sure that that link remains working in perpetuity. Also anyone who wants to redistribute the wheels has to commit to doing the same.

I looked at this again, and the gfortran build in https://github.com/MacPython/gfortran-install/ came originally from https://gcc.gnu.org/wiki/GFortranBinaries. That is a gnu.org site, and it itself doesn't even have clear directions or direct links to sources corresponding to the binaries. If they themselves even can't get this right, I honestly don't see the point of spending more effort on something that may or may not be necessary legally (still unclear), but is definitely not helping anyone in any practical sense.

The correct link is probably https://gcc.gnu.org/viewcvs/gcc/, that'll live long enough to be "in perpetuity".

Once the wheel license appending has been done, I'll remove the 1.0.0 milestone.

[rgommers]

Here is the PR by @pv that appends the correct license info to the license file for wheels builds: MacPython/scipy-wheels#5

[rgommers]

That PR includes a link, so closes this issue:

Name: GCC runtime library
Files: extra-dll\*.dll
Description: statically linked, in DLL files compiled with gfortran only
Availability: https://gcc.gnu.org/viewcvs/gcc/
License: GPLv3 + runtime exception
  Copyright (C) 2002-2017 Free Software Foundation, Inc.

  ... <license text>

[rgommers]

MacPython/scipy-wheels#5 is merged, so this will be all OK for the 1.0 release. Thanks for the contributions all!