x/pkgsite: Override licensecheck logic

[spakin]

What is the URL of the page with the issue?

https://pkg.go.dev/github.com/lanl/clp

What is your user agent?

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/100.0.4896.127 Safari/537.36

Screenshot

What did you do?

Navigated to https://pkg.go.dev/github.com/lanl/clp

What did you expect to see?

Documentation

What did you see instead?

A notice that "Documentation not displayed due to license restrictions."

Commentary

clp is licensed under what's really a BSD-3-Clause license. Unfortunately, my institution dictates specific wording that's just different enough from the stock wording that licensecheck reports too low confidence for pkg.go.dev to show the documentation:

57.8% of text covered by licenses:
BSD-3-Clause at [45:93] IsURL=true
BSD-3-Clause at [1176:2684] IsURL=false

Is there some way a license file can indicate that it really is some particular license, despite what the checker is able to discern? As you can see, I added the BSD-3-Clause URL to clp's LICENSE.md file, but this merely increases licensecheck's confidence rather than forcing it to honor the specified license.

If there's no such feature, let's treat this as a feature request.

(This issue is somewhat related to #49155.)

[jba]

Is there some way a license file can indicate that it really is some particular license, despite what the checker is able to discern?

IANAL, but you can see how that raises issues. "This is a BSD-3 license, even though it clearly states that you cannot redistribute our software." How would a court rule on that? For the same reason, we do not look at SPDX identifiers.

If your wording is not too divergent or idiosyncratic, consider a PR that modifies https://github.com/google/licensecheck/blob/main/licenses/BSD.lre. That way everyone can learn about your variant.

As a fallback, we may be able to add it to https://go.googlesource.com/pkgsite/+/refs/heads/master/internal/licenses/exceptions.

[spakin]

IANAL, but you can see how that raises issues. "This is a BSD-3 license, even though it clearly states that you cannot redistribute our software." How would a court rule on that? For the same reason, we do not look at SPDX identifiers.

Ah, good point.

Let me therefore ask a more technical question: What threshold does pkg.go.dev require for a license to be considered a match? I'll use that to see if a modest change to BSD.ire will accept my institution's variant wording. As you can see from my initial post, I'm currently at 57.8%. If I need to get to 60%, that's one thing; if I need to get to 95%, that's a tougher challenge.

[jba]

Each License Regular Expression (LRE) is binary: it either matches some text, or it doesn't.

75% of the file must be matched by some LRE. (Some license files are concatenations of multiple licenses.)

Your LICENSE.md sort of looks like a BSD-3 license; there's not a lot of extra text. I would expect it to be close to the BSD-3 LRE. I didn't look carefully enough to see where it diverges. If you can point out the differences to me, perhaps we can see how to tweak the LRE together.

[spakin]

Your LICENSE.md sort of looks like a BSD-3 license; there's not a lot of extra text. I would expect it to be close to the BSD-3 LRE. I didn't look carefully enough to see where it diverges. If you can point out the differences to me, perhaps we can see how to tweak the LRE together.

Thanks for your offer to help. A wdiff comparison suggests that the bulk of the difference is an extra paragraph added to the beginning of clp's LICENSE.md:

This software was produced under U.S. Government contract 89233218CNA000001 for Los Alamos National Laboratory (LANL), which is operated by Triad National Security, LLC for the U.S. Department of Energy/National Nuclear Security Administration. All rights in the program are reserved by Triad National Security, LLC, and the U.S. Department of Energy/National Nuclear Security Administration. The Government is granted for itself and others acting on its behalf a nonexclusive, paid-up, irrevocable worldwide license in this material to reproduce, prepare derivative works, distribute copies to the public, perform publicly and display publicly, and to permit others to do so. NEITHER THE GOVERNMENT NOR TRIAD NATIONAL SECURITY, LLC MAKES ANY WARRANTY, EXPRESS OR IMPLIED, OR ASSUMES ANY LIABILITY FOR THE USE OF THIS SOFTWARE. If software is modified to produce derivative works, such modified software should be clearly marked, so as not to confuse it with the version available from LANL.

In particular, the list of conditions (the three clauses that make up BSD-3-Clause) and the final no-warranties/no-liability paragraph are unmodified apart from filling in the copyright-holder placeholders.

If I delete the "bonus" paragraph and the BSD-3-Clause assertion I inserted in my failed attempt to pass the license check, my LICENSE.md passes with flying colors:

100.0% of text covered by licenses:
BSD-3-Clause at [0:1592] IsURL=false

Given that, what do you suggest?

A quick Web search reveals that the line "The Government is granted for itself and others acting on its behalf a nonexclusive, paid-up, irrevocable worldwide license in this material to reproduce, prepare derivative works, distribute copies to the public, perform publicly and display publicly, and to permit others to do so" appears with only minor wording changes in a number of different licenses. Is it possible—and would it help—to tell the matcher that this line is a no-op for comparing licenses? I believe the point it's making is that regardless of the terms laid out in the rest of the license, the U.S. Government has permission to make the material freely available if it so chooses. That shouldn't affect one way or the other pkg.go.dev's ability to reformat the documentation for presentation online.

[jba]

Sorry I haven't replied; I'm attending a multi-day meeting. I hope to revisit this Friday or next Monday.

[spakin]

It looks like I may be allowed by my institution to move the troublesome paragraph from LICENSE.md to README.md. I believe that will solve the problem in the specific case of github.com/lanl/clp.

It would still be helpful to others releasing software paid for in part by the US Government if the license checker knew to ignore text that looks like a US Government rights clause. Does that sound like a practical feature addition?

[jba]

I'm sorry I haven't had the bandwidth to pursue this. Although I agree on the general solution, it would be much better for all at this point if you moved the paragraph.

[spakin]

Done.