Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
c5d8793
Rework of binary distribution licenses
ivankelly Dec 19, 2017
d410991
Pull deps in tarballs, and exclude from rat
ivankelly Dec 19, 2017
7b57b09
Removed unnecessary licenses.
ivankelly Dec 20, 2017
82373e1
Fix rat
ivankelly Dec 20, 2017
9363a0b
Merge remote-tracking branch 'origin/master' into license-rework
ivankelly Jan 9, 2018
6e2fbae
Update copyright year in NOTICE
ivankelly Jan 9, 2018
2242b70
Add rocksdb
ivankelly Jan 9, 2018
d5ee03a
Explicitly call out the licenses to include in the distro
ivankelly Jan 9, 2018
5c2d9c7
Continue checks if no deps dir in tarball
ivankelly Jan 9, 2018
c579f46
Update server to the new license format
ivankelly Jan 9, 2018
ba9ef12
Documentation for how to deal with dependencies
ivankelly Jan 9, 2018
fb0b2ba
Remove circe from NOTICE (it's not needed)
ivankelly Jan 10, 2018
a1f73ae
Wildcard for netty licenses
ivankelly Jan 10, 2018
227ef93
Add license check to travis
ivankelly Jan 10, 2018
87f9b2e
Add documentation for source dependencies and rules of thumb
ivankelly Jan 10, 2018
dd3fc7f
Fixing some typos, adding some links
ivankelly Jan 10, 2018
d45b770
Moved to community section
ivankelly Jan 10, 2018
869ba78
Travis needs to generate packages for them to be checked
ivankelly Jan 10, 2018
d1aa795
OSX tar doesn't have tar --wildcards
ivankelly Jan 10, 2018
0da3051
Move some unneeded stuff from NOTICE
ivankelly Jan 12, 2018
92dabc2
Remove harmony stuff
ivankelly Jan 16, 2018
e1c8d6e
Add pointer to docs on check failure
ivankelly Jan 16, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,9 @@ before_install:
- if [ "$TRAVIS_OS_NAME" == "linux" ]; then jdk_switcher use "$CUSTOM_JDK"; fi

script:
- travis_retry mvn --batch-mode clean apache-rat:check compile spotbugs:check
- travis_retry mvn --batch-mode clean apache-rat:check compile spotbugs:check package -DskipTests
- if [ "$TRAVIS_OS_NAME" == "linux" ]; then dev/check-binary-license ./bookkeeper-dist/all/target/bookkeeper-all-4.7.0-SNAPSHOT-bin.tar.gz; fi
- if [ "$TRAVIS_OS_NAME" == "linux" ]; then dev/check-binary-license ./bookkeeper-dist/server/target/bookkeeper-server-4.7.0-SNAPSHOT-bin.tar.gz; fi
# Disabled the tests here. Since tests are running much slower on Travis than on Jenkins
# - ./dev/ticktoc.sh "mvn --batch-mode clean package"

Expand Down
7 changes: 1 addition & 6 deletions NOTICE
Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
Apache BookKeeper
Copyright 2011-2017 The Apache Software Foundation
Copyright 2011-2018 The Apache Software Foundation

This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).

This product contains a modified version of 'Circe', a high-performance
hash algorithm framework & library from Trevor Robinson:

* LICENSE: Apache License 2.0
* HOMEPAGE: https://github.com/trevorr/circe
17 changes: 17 additions & 0 deletions bookkeeper-dist/src/assemble/bin-all.xml
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,23 @@
<include>${basedir}/*.txt</include>
</includes>
</fileSet>
<fileSet>
<directory>../src/main/resources/deps</directory>
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should not maintain the 3rdparty license. we should only attach the needed notice and license that is required into NOTICE or LICENSE file. maintaining this directory makes things complicated. we should avoid it.

The check script should parse pom files or the assemble tarballs to see what dependencies are included, and fetch their corresponding notices and verify if the NOTICE file includes all dependencies or not and if their licenses are matched and notices are attached.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've moved all the NOTICE stuff into the NOTICE, so that doesn't link anywhere.

The ASF licensing recommendations (http://www.apache.org/dev/licensing-howto.html#permissive-deps) actually say to bundle the license file, instead of putting directly in the LICENSE file unless the license is very short. I actually prefer it like this.
https://github.com/ivankelly/bookkeeper/blob/license-rework/bookkeeper-dist/src/main/resources/LICENSE-all.bin.txt is easier to read than https://github.com/apache/bookkeeper/blob/master/bookkeeper-dist/src/main/resources/LICENSE-all.bin.txt.

LICENSE-all.bin.txt would get huge if we flattened them all. I'm particularly eager to keep the CDDL out of it, that license is huge.

The check script should parse pom files or the assemble tarballs to see what dependencies are included, and fetch their corresponding notices and

We should check on the final output (i.e. the assemble tarball) as that is what we distribute.

verify if the NOTICE file includes all dependencies or not and if their licenses are matched and notices are attached.

This verification is very hard to do in an automated fashion. How will a machine know that the protobuf license contains stuff that isn't relevant? How will it work out which part of the netty NOTICE needs to be pulled in and which doesn't?

It's not hard for a human, but the human will need guidelines which we should put in the wiki.

I've added a check to the script to check if the bundled license files are linked, and ensure all linked files exist.

<outputDirectory>/deps</outputDirectory>
<includes>
<include>javax.servlet-api-3.1.0/CDDL+GPL-1.1</include>
<include>jsr-305/LICENSE</include>
<include>netty-3.10.1.Final/*</include>
<include>netty-4.1.12.Final/*</include>
<include>paranamer-2.8/LICENSE.txt</include>
<include>protobuf-3.4.0/LICENSE</include>
<include>scala-library-2.11.7/LICENSE.md</include>
<include>scala-parser-combinators_2.11-1.0.4/LICENSE.md</include>
<include>scala-reflect-2.11.8/LICENSE.md</include>
<include>slf4j-1.7.25/LICENSE.txt</include>
</includes>
<fileMode>644</fileMode>
</fileSet>
</fileSets>
<files>
<file>
Expand Down
11 changes: 11 additions & 0 deletions bookkeeper-dist/src/assemble/bin-server.xml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,17 @@
<include>${basedir}/*.txt</include>
</includes>
</fileSet>
<fileSet>
<directory>../src/main/resources/deps</directory>
<outputDirectory>/deps</outputDirectory>
<includes>
<include>javax.servlet-api-3.1.0/CDDL+GPL-1.1</include>
<include>netty-4.1.12.Final/*</include>
<include>protobuf-3.4.0/LICENSE</include>
<include>slf4j-1.7.25/LICENSE.txt</include>
</includes>
<fileMode>644</fileMode>
</fileSet>
</fileSets>
<files>
<file>
Expand Down
729 changes: 260 additions & 469 deletions bookkeeper-dist/src/main/resources/LICENSE-all.bin.txt

Large diffs are not rendered by default.

535 changes: 152 additions & 383 deletions bookkeeper-dist/src/main/resources/LICENSE-server.bin.txt

Large diffs are not rendered by default.

163 changes: 116 additions & 47 deletions bookkeeper-dist/src/main/resources/NOTICE-all.bin.txt
Original file line number Diff line number Diff line change
@@ -1,55 +1,124 @@
Apache BookKeeper
Copyright 2011-2017 The Apache Software Foundation
Copyright 2011-2018 The Apache Software Foundation

Licensed under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).

------------------------------------------------------------------------------------
- lib/io.dropwizard.metrics-metrics-core-3.1.0.jar
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is okay and enough to keep versioning in LICENSE. However I would suggest removing versioning from NOTICE file: 1) versioning is not legally required in NOTICE. 2) NOTICE as short as possible as it will affect downstream projects. this is going to cause NOTICE updates everytime we bumped version.

http://www.apache.org/dev/licensing-howto.html#mod-notice

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the versions are there to facilitate automated checking. if we remove the versions we can't ensure that what is in the notice file exists in the shipped tarball. It will need to be checked manually, so more chance of a -1 on a release candidate.

Also, notices can change between version, so the rationale for having them in the LICENSE holds for the NOTICE also.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know notices can change between version. my point is most of the dependencies are having versioning references in LICENSE, which automated checking will fail the a dependency change if the versioning is not updated. automated checking does its job on license/versioning and the contributors/reviewers should check both license/notice on reviewing it. automated checking on LICENSE is enough for capturing this.

The reason I would suggest removing versions from NOTICE, as the ASF polices suggest and the practices that people have, is to avoid unnecessary changes to NOTICE as possible, keep it as brief as possible, and avoid impacting downstream projects as little as possible.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

People will do the bare minimum to make their builds pass. If there are no versions in the NOTICE, they won't touch the notice, and we'll end up with -1 on release candidates.

This notice should have no effect on downstream projects. Downstream depend on our maven jar, not our binary tarballs, which is covered by the top level NOTICE.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ivankelly fine with me.

- lib/io.dropwizard.metrics-metrics-graphite-3.1.0.jar
- lib/io.dropwizard.metrics-metrics-jvm-3.1.0.jar

Metrics
Copyright 2010-2013 Coda Hale and Yammer, Inc.

This product includes software developed by Coda Hale and Yammer, Inc.

This product includes code derived from the JSR-166 project (ThreadLocalRandom, Striped64,
LongAdder), which was released with the following comments:

Written by Doug Lea with assistance from members of JCP JSR-166
Expert Group and released to the public domain, as explained at
http://creativecommons.org/publicdomain/zero/1.0/
------------------------------------------------------------------------------------
- lib/io.netty-netty-3.10.1.Final.jar

The Netty Project
=================

Please visit the Netty web site for more information:

* http://netty.io/

Copyright 2011 The Netty Project

The Netty Project licenses this file to you under the Apache License,
version 2.0 (the "License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at:

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.

This project includes:
Apache Log4j under The Apache Software License, Version 2.0
Caffeine under The Apache Software License, Version 2.0
Commons CLI under The Apache Software License, Version 2.0
Commons Codec under The Apache Software License, Version 2.0
Commons Collections under The Apache Software License, Version 2.0
Commons Configuration under The Apache Software License, Version 2.0
Commons IO under The Apache Software License, Version 2.0
Commons Lang under The Apache Software License, Version 2.0
Commons Logging under The Apache Software License, Version 2.0
Dropwizard Metrics under The Apache Software License, Version 2.0
Guava under The Apache Software License, Version 2.0
Jackson under The Apache Software License, Version 2.0
Java Native Access under The Apache Software License, Version 2.0
Java Servlet API under Common Development and Distribution License 1.0
Javax.inject under The Apache Software License, Version 2.0
Jetty under The Apache Software License, Version 2.0
JSR305 Annotations for Findbugs under The Apache Software License, Version 2.0
ParaNamer under New BSD License
Prometheus under The Apache Software License, Version 2.0
Protocol Buffer Java API under New BSD License
Scala under New BSD License
SLF4J API Module under MIT License
SLF4J LOG4J-12 Binding under MIT License
The Netty Project under The Apache Software License, Version 2.0
Twitter Finagle under The Apache Software License, Version 2.0
Twitter JSR166e under Creative Commons Zero v1.0 Universal
Twitter LibThrift under The Apache Software License, Version 2.0
Twitter Server under The Apache Software License, Version 2.0
Twitter Scrooge under The Apache Software License, Version 2.0
Twitter Util under The Apache Software License, Version 2.0
Vertx under The Apache Software License, Version 2.0
ZooKeeper under The Apache Software License, Version 2.0

This product contains a modified version of 'Circe', a high-performance
hash algorithm framework & library from Trevor Robinson:

* LICENSE: Apache License 2.0
* HOMEPAGE: https://github.com/trevorr/circe
------------------------------------------------------------------------------------
- lib/io.netty-netty-all-4.1.12.Final.jar

The Netty Project
=================

Please visit the Netty web site for more information:

* http://netty.io/

Copyright 2014 The Netty Project

The Netty Project licenses this file to you under the Apache License,
version 2.0 (the "License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at:

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.

------------------------------------------------------------------------------------
- lib/io.prometheus-simpleclient-0.0.21.jar
- lib/io.prometheus-simpleclient_common-0.0.21.jar
- lib/io.prometheus-simpleclient_hotspot-0.0.21.jar
- lib/io.prometheus-simpleclient_servlet-0.0.21.jar

Prometheus instrumentation library for JVM applications
Copyright 2012-2015 The Prometheus Authors

This product includes software developed at
Boxever Ltd. (http://www.boxever.com/).

This product includes software developed at
SoundCloud Ltd. (http://soundcloud.com/).

This product includes software developed as part of the
Ocelli project by Netflix Inc. (https://github.com/Netflix/ocelli/).
------------------------------------------------------------------------------------
- lib/org.eclipse.jetty-jetty-http-9.4.5.v20170502.jar
- lib/org.eclipse.jetty-jetty-io-9.4.5.v20170502.jar
- lib/org.eclipse.jetty-jetty-security-9.4.5.v20170502.jar
- lib/org.eclipse.jetty-jetty-server-9.4.5.v20170502.jar
- lib/org.eclipse.jetty-jetty-servlet-9.4.5.v20170502.jar
- lib/org.eclipse.jetty-jetty-util-9.4.5.v20170502.jar

==============================================================
Jetty Web Container
Copyright 1995-2017 Mort Bay Consulting Pty Ltd.
==============================================================

The Jetty Web Container is Copyright Mort Bay Consulting Pty Ltd
unless otherwise noted.

Jetty is dual licensed under both

* The Apache 2.0 License
http://www.apache.org/licenses/LICENSE-2.0.html

and

* The Eclipse Public 1.0 License
http://www.eclipse.org/legal/epl-v10.html

Jetty may be distributed under either license.

lib/org.eclipse.jetty-jetty-util-9.4.5.v20170502.jar bundles UnixCrypt

The UnixCrypt.java code implements the one way cryptography used by
Unix systems for simple password protection. Copyright 1996 Aki Yoshida,
modified April 2001 by Iris Van den Broeke, Daniel Deville.
Permission to use, copy, modify and distribute UnixCrypt
for non-commercial or commercial purposes and without fee is
granted provided that the copyright notice appears in all copies.
------------------------------------------------------------------------------------
113 changes: 77 additions & 36 deletions bookkeeper-dist/src/main/resources/NOTICE-server.bin.txt
Original file line number Diff line number Diff line change
@@ -1,43 +1,84 @@
Apache BookKeeper
Copyright 2011-2017 The Apache Software Foundation
Copyright 2011-2018 The Apache Software Foundation

Licensed under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
This product includes software developed at
The Apache Software Foundation (http://www.apache.org/).

------------------------------------------------------------------------------------
- lib/io.netty-netty-all-4.1.12.Final.jar

The Netty Project
=================

Please visit the Netty web site for more information:

* http://netty.io/

Copyright 2014 The Netty Project

The Netty Project licenses this file to you under the Apache License,
version 2.0 (the "License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at:

http://www.apache.org/licenses/LICENSE-2.0
http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.

This project includes:
Apache Log4j under The Apache Software License, Version 2.0
Commons CLI under The Apache Software License, Version 2.0
Commons Codec under The Apache Software License, Version 2.0
Commons Collections under The Apache Software License, Version 2.0
Commons Configuration under The Apache Software License, Version 2.0
Commons IO under The Apache Software License, Version 2.0
Commons Lang under The Apache Software License, Version 2.0
Commons Logging under The Apache Software License, Version 2.0
Guava under The Apache Software License, Version 2.0
Jackson under The Apache Software License, Version 2.0
Java Native Access under The Apache Software License, Version 2.0
Java Servlet API under Common Development and Distribution License 1.0
Jetty under The Apache Software License, Version 2.0
Prometheus under The Apache Software License, Version 2.0
Protocol Buffer Java API under New BSD license
SLF4J API Module under MIT License
SLF4J LOG4J-12 Binding under MIT License
The Netty Project under The Apache Software License, Version 2.0
Vertx under The Apache Software License, Version 2.0
ZooKeeper under Apache License, Version 2.0

This product contains a modified version of 'Circe', a high-performance
hash algorithm framework & library from Trevor Robinson:

* LICENSE: Apache License 2.0
* HOMEPAGE: https://github.com/trevorr/circe
------------------------------------------------------------------------------------
- lib/io.prometheus-simpleclient-0.0.21.jar
- lib/io.prometheus-simpleclient_common-0.0.21.jar
- lib/io.prometheus-simpleclient_hotspot-0.0.21.jar
- lib/io.prometheus-simpleclient_servlet-0.0.21.jar

Prometheus instrumentation library for JVM applications
Copyright 2012-2015 The Prometheus Authors

This product includes software developed at
Boxever Ltd. (http://www.boxever.com/).

This product includes software developed at
SoundCloud Ltd. (http://soundcloud.com/).

This product includes software developed as part of the
Ocelli project by Netflix Inc. (https://github.com/Netflix/ocelli/).
------------------------------------------------------------------------------------
- lib/org.eclipse.jetty-jetty-http-9.4.5.v20170502.jar
- lib/org.eclipse.jetty-jetty-io-9.4.5.v20170502.jar
- lib/org.eclipse.jetty-jetty-security-9.4.5.v20170502.jar
- lib/org.eclipse.jetty-jetty-server-9.4.5.v20170502.jar
- lib/org.eclipse.jetty-jetty-servlet-9.4.5.v20170502.jar
- lib/org.eclipse.jetty-jetty-util-9.4.5.v20170502.jar

==============================================================
Jetty Web Container
Copyright 1995-2017 Mort Bay Consulting Pty Ltd.
==============================================================

The Jetty Web Container is Copyright Mort Bay Consulting Pty Ltd
unless otherwise noted.

Jetty is dual licensed under both

* The Apache 2.0 License
http://www.apache.org/licenses/LICENSE-2.0.html

and

* The Eclipse Public 1.0 License
http://www.eclipse.org/legal/epl-v10.html

Jetty may be distributed under either license.

lib/org.eclipse.jetty-jetty-util-9.4.5.v20170502.jar bundles UnixCrypt

The UnixCrypt.java code implements the one way cryptography used by
Unix systems for simple password protection. Copyright 1996 Aki Yoshida,
modified April 2001 by Iris Van den Broeke, Daniel Deville.
Permission to use, copy, modify and distribute UnixCrypt
for non-commercial or commercial purposes and without fee is
granted provided that the copyright notice appears in all copies.
------------------------------------------------------------------------------------
Loading