HBASE-25966 Fix typo in NOTICE.vm #3349
Conversation
ndimiduk
requested review from
busbey,
saintstack,
apurtell,
Apache9 and
Reidddddd
|
Perhaps one of you can suggest how to verify that the license content is being included in the appropriate places after this change -- what those places are and whether it should be present or absent? |
|
Trying to interpret this output. I think our build is configured such that the only module that bundles bootstrap is hbase-server. However, I think we also bundle it in hbase-thrift. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
I think @busbey is the expert here. |
|
I'm building this branch locally now so I can check through stuff. Thanks for finding and fixing this Nick! |
There was a problem hiding this comment.
Here’s how I’d first check for jar files that include the bootstrap files:
(base) sbusbey@Seans-MacBook-Pro hbase % for artifact in $(find . -name '*.jar' -not -path './hbase-assembly/*'); do for matches in $(jar tf "${artifact}" | grep -i "bootstrap" | grep -E "\\.js$"); do echo "$(basename "${artifact}"):${matches}"; done; done
hbase-thrift-3.0.0-SNAPSHOT-sources.jar:hbase-webapps/static/js/bootstrap.min.js
hbase-thrift-3.0.0-SNAPSHOT.jar:hbase-webapps/static/js/bootstrap.min.js
hbase-server-3.0.0-SNAPSHOT.jar:hbase-webapps/static/js/bootstrap.min.js
hbase-shaded-testing-util-3.0.0-SNAPSHOT.jar:webapps/static/bootstrap-3.3.7/js/bootstrap-editable.min.js
hbase-shaded-testing-util-3.0.0-SNAPSHOT.jar:webapps/static/bootstrap-3.3.7/js/bootstrap.js
hbase-shaded-testing-util-3.0.0-SNAPSHOT.jar:webapps/static/bootstrap-3.3.7/js/bootstrap.min.js
hbase-shaded-testing-util-3.0.0-SNAPSHOT.jar:webapps/static/bootstrap-3.3.7/js/npm.js
hbase-shaded-testing-util-3.0.0-SNAPSHOT.jar:webapps/static/dataTables.bootstrap.js
hbase-shaded-testing-util-3.0.0-SNAPSHOT.jar:hbase-webapps/static/js/bootstrap.min.js
hbase-shaded-mapreduce-3.0.0-SNAPSHOT.jar:hbase-webapps/static/js/bootstrap.min.js
The Bootstrap notice we put in looks like this:
This product includes portions of the Bootstrap project v3.0.0
Licensed under the Apache License v2.0 as a part of the Bootstrap project.
Next I’d check for what jar files include this notice:
(base) sbusbey@Seans-MacBook-Pro hbase % for artifact in $(find . -name '*.jar' -not -path './hbase-assembly/*'); do unzip -c "${artifact}" META-INF/NOTICE | grep --with-filename --label="$(basename "${artifact}")" -i "bootstrap" | head ; done
hbase-thrift-3.0.0-SNAPSHOT-sources.jar:This product includes portions of the Bootstrap project v3.0.0
hbase-thrift-3.0.0-SNAPSHOT-sources.jar:Licensed under the Apache License v2.0 as a part of the Bootstrap project.
hbase-thrift-3.0.0-SNAPSHOT-test-sources.jar:This product includes portions of the Bootstrap project v3.0.0
hbase-thrift-3.0.0-SNAPSHOT-test-sources.jar:Licensed under the Apache License v2.0 as a part of the Bootstrap project.
hbase-thrift-3.0.0-SNAPSHOT.jar:This product includes portions of the Bootstrap project v3.0.0
hbase-thrift-3.0.0-SNAPSHOT.jar:Licensed under the Apache License v2.0 as a part of the Bootstrap project.
hbase-shaded-testing-util-3.0.0-SNAPSHOT.jar: This product includes portions of the Bootstrap project v3.0.0
hbase-shaded-testing-util-3.0.0-SNAPSHOT.jar: Licensed under the Apache License v2.0 as a part of the Bootstrap project.
hbase-shaded-client-3.0.0-SNAPSHOT.jar: This product includes portions of the Bootstrap project v3.0.0
hbase-shaded-client-3.0.0-SNAPSHOT.jar: Licensed under the Apache License v2.0 as a part of the Bootstrap project.
That looks like we’re missing hbase-server and hbase-shaded-mapreduce. Also we have the NOTICE where we should not in the test-sources jar of hbase-thrift and the main jar for hbase-shaded-client.
Next I’d look for which of our assemblies have one or more of the bootstrap files, or the jars mentioned above:
hbase-3.0.0-SNAPSHOT-bin.tar.gz:hbase-3.0.0-SNAPSHOT/hbase-webapps/static/js/bootstrap.min.js
(base) sbusbey@Seans-MacBook-Pro hbase % for assembly in $(find hbase-assembly/target -name '*.tar.gz'); do for matches in $(tar tzf "${assembly}" | grep -E '(hbase-thrift|hbase-server|hbase-shaded-testing-util|hbase-shaded-mapreduce)' | grep -E "\\.jar$"); do echo "$(basename "${assembly}"):${matches}"; done; done;
hbase-3.0.0-SNAPSHOT-client-bin.tar.gz:hbase-3.0.0-SNAPSHOT-client/lib/hbase-server-3.0.0-SNAPSHOT-tests.jar
hbase-3.0.0-SNAPSHOT-client-bin.tar.gz:hbase-3.0.0-SNAPSHOT-client/lib/hbase-server-3.0.0-SNAPSHOT.jar
hbase-3.0.0-SNAPSHOT-client-bin.tar.gz:hbase-3.0.0-SNAPSHOT-client/lib/hbase-thrift-3.0.0-SNAPSHOT.jar
hbase-3.0.0-SNAPSHOT-client-bin.tar.gz:hbase-3.0.0-SNAPSHOT-client/lib/shaded-clients/hbase-shaded-mapreduce-3.0.0-SNAPSHOT.jar
hbase-3.0.0-SNAPSHOT-bin.tar.gz:hbase-3.0.0-SNAPSHOT/lib/hbase-server-3.0.0-SNAPSHOT-tests.jar
hbase-3.0.0-SNAPSHOT-bin.tar.gz:hbase-3.0.0-SNAPSHOT/lib/hbase-server-3.0.0-SNAPSHOT.jar
hbase-3.0.0-SNAPSHOT-bin.tar.gz:hbase-3.0.0-SNAPSHOT/lib/hbase-thrift-3.0.0-SNAPSHOT.jar
hbase-3.0.0-SNAPSHOT-bin.tar.gz:hbase-3.0.0-SNAPSHOT/lib/shaded-clients/hbase-shaded-mapreduce-3.0.0-SNAPSHOT.jar
(base) sbusbey@Seans-MacBook-Pro hbase %
And then which of these include the bootstrap license information:
(base) sbusbey@Seans-MacBook-Pro hbase % for assembly in $(find hbase-assembly -name '*.tar.gz'); do tar --to-stdout -xzf ${assembly} '*/NOTICE.txt' | grep --with-filename --label="$(basename "${assembly}")" -i bootstrap | head
; done
hbase-3.0.0-SNAPSHOT-client-bin.tar.gz: This product includes portions of the Bootstrap project v3.0.0
hbase-3.0.0-SNAPSHOT-client-bin.tar.gz: Licensed under the Apache License v2.0 as a part of the Bootstrap project.
hbase-3.0.0-SNAPSHOT-client-bin.tar.gz: This product includes portions of the Bootstrap project v3.0.0
hbase-3.0.0-SNAPSHOT-client-bin.tar.gz: Licensed under the Apache License v2.0 as a part of the Bootstrap project.
hbase-3.0.0-SNAPSHOT-client-bin.tar.gz: This product includes portions of the Bootstrap project v3.0.0
hbase-3.0.0-SNAPSHOT-client-bin.tar.gz: Licensed under the Apache License v2.0 as a part of the Bootstrap project.
hbase-3.0.0-SNAPSHOT-client-bin.tar.gz: This product includes portions of the Bootstrap project v3.0.0
hbase-3.0.0-SNAPSHOT-client-bin.tar.gz: Licensed under the Apache License v2.0 as a part of the Bootstrap project.
hbase-3.0.0-SNAPSHOT-client-bin.tar.gz: This product includes portions of the Bootstrap project v3.0.0
hbase-3.0.0-SNAPSHOT-client-bin.tar.gz: Licensed under the Apache License v2.0 as a part of the Bootstrap project.
hbase-3.0.0-SNAPSHOT-bin.tar.gz: This product includes portions of the Bootstrap project v3.0.0
hbase-3.0.0-SNAPSHOT-bin.tar.gz: Licensed under the Apache License v2.0 as a part of the Bootstrap project.
hbase-3.0.0-SNAPSHOT-bin.tar.gz: This product includes portions of the Bootstrap project v3.0.0
hbase-3.0.0-SNAPSHOT-bin.tar.gz: Licensed under the Apache License v2.0 as a part of the Bootstrap project.
hbase-3.0.0-SNAPSHOT-bin.tar.gz: This product includes portions of the Bootstrap project v3.0.0
hbase-3.0.0-SNAPSHOT-bin.tar.gz: Licensed under the Apache License v2.0 as a part of the Bootstrap project.
hbase-3.0.0-SNAPSHOT-bin.tar.gz: This product includes portions of the Bootstrap project v3.0.0
hbase-3.0.0-SNAPSHOT-bin.tar.gz: Licensed under the Apache License v2.0 as a part of the Bootstrap project.
hbase-3.0.0-SNAPSHOT-bin.tar.gz: This product includes portions of the Bootstrap project v3.0.0
hbase-3.0.0-SNAPSHOT-bin.tar.gz: Licensed under the Apache License v2.0 as a part of the Bootstrap project.
It looks like both of our assembly need to include the bootstrap information in their NOTICE files and both do. I’d assume the multiple copies is a side effect of how we aggregate module and dependency NOTICE details.
|
I'll dig in later this evening to see what's going on here. for the jars in particular this looks like the opposite of what I thought was going to happen after reading the patch. |
There was a problem hiding this comment.
hbase-thrift jars
Instead of using the hbase-source-bundle as a source for generating the velocity template, this module uses the default apache foundation bundle with an addendum in hbase-thrift/src/main/appended-resources/META-INF/NOTICE . If we check, that file has the bootstrap notice.
We define an override to set a different NOTICE file for the hbase-thrift test jar and javadoc jar. All the others must be reusing this one we set for the main jar. Probably a bug that the test-source jar isn’t using the one defined for the test jar. Not related to things here though.
hbase-shaded-testing-util and hbase-shaded-client
These are both getting their NOTICE contents from aggregating the one from Hadoop. That’s the right thing to do since they both include Hadoop contents. Unfortunately the Hadoop project uses a single NOTICE for all of their jars, so it’s correct for what we include in shaded-testing-util but wrong for shaded-client. Again, not related to the change here.
hbase-server and hbase-shaded-mapreduce
I finally managed to chase down what’s happening. the build for hbase-server never executes our configured license creation because its defined execution doesn’t bind to a goal.
AFAICT this is a bug introduced when HBASE-16335 upgraded our apache parent pom. In the earlier version 12 pom there was a un-named execution that set the process goal for handling the default LICENSE/NOTICE creation for jar files (ref). Our execution definition in hbase-server is set to change the configuration for that run and does so by setting an id of default; due to a maven-ism that id will match the first un-named execution. When we upgraded to version 18 of the parent pom this execution changed to be named process-resource-bundles (ref) and so our execution became silently unmoored.
Once I update the execution to have a goal everything executes as expected. hbase-server gets a LICENSE/NOTICE different from the default, and hbase-shaded-mapreduce gets the correct NOTICE for bootstrap when it aggregates the NOTICE from hbase-server. If I make that fix but do not include your correction here then I get a non-default NOTICE for hbase-server that looks correct. But in that case also get incorrect inclusion of the bootstrap NOTICE details on every module that makes use our custom resource bundle (because of a type mismatch between string properties and velocity booleans I think).
The problems above all sound unrelated to the specific problem you address here. I’m +1 on this landing as-is and then I can make follow-on jiras for the things we want to fix. If you wanted to incorporate fixing hbase-server’s execution for the remote resources plugin as a part of this PR I would also be fine with that.
|
Let's get this in? @ndimiduk |
Yes, I suppose this doesn't make things worse. It sounds like we need further fixes in this area, and some kind of test that the notice files are generated as we intend. |
Signed-off-by: Sean Busbey <busbey@apache.org>
ndimiduk
force-pushed
the
25966-notice.vm
branch
from
21ff244
to
b50538d
Compare
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
No description provided.