How to handle non-stable ClearlyDefined results?

[dhendriks]

We use the Dash license check tool to check whether our dependencies are OK, license wise.

Today I get the following:

12:20:29  Checking for differences between generated and stored dependency lists...
12:20:29  --- DEPENDENCIES.txt	2021-07-02 10:18:13.476439687 +0000
12:20:29  +++ DEPENDENCIES.generated.processed.txt	2021-07-02 10:20:29.746468720 +0000
12:20:29  @@ -3,9 +3,9 @@
12:20:29   maven/mavencentral/org.apache.ant/ant-launcher/1.10.9, Apache-2.0 AND W3C AND LicenseRef-Public-Domain, approved, CQ15560
12:20:29   maven/mavencentral/org.apache.ant/ant/1.10.9, Apache-2.0 AND W3C AND LicenseRef-Public-Domain, approved, CQ15560
12:20:29   maven/mavencentral/org.codehaus.groovy/groovy-ant/3.0.8, Apache-2.0, approved, clearlydefined
12:20:29  -maven/mavencentral/org.codehaus.groovy/groovy-groovydoc/3.0.8, Apache-2.0, approved, clearlydefined
12:20:29  +maven/mavencentral/org.codehaus.groovy/groovy-groovydoc/3.0.8, Apache-2.0, restricted, clearlydefined
12:20:29   maven/mavencentral/org.codehaus.groovy/groovy-json/3.0.8, Apache-2.0, approved, clearlydefined
12:20:29  -maven/mavencentral/org.codehaus.groovy/groovy/3.0.8, Apache-2.0, approved, clearlydefined
12:20:29  +maven/mavencentral/org.codehaus.groovy/groovy/3.0.8, Apache-2.0, restricted, clearlydefined
12:20:29   p2/orbit/p2.eclipse-plugin/com.ibm.icu/67.1.0.v20200706-1749, ICU AND Unicode-TOU AND BSD-3-Clause AND BSD-2-Clause AND LicenseRef-ipadic-license AND LicenseRef-Public-Domain, approved, CQ22320
12:20:29   p2/orbit/p2.eclipse-plugin/com.sun.jna.platform/5.8.0.v20210406-1004, Apache-2.0 OR LGPL-2.1-or-later, approved, CQ23218
12:20:29   p2/orbit/p2.eclipse-plugin/com.sun.jna/5.8.0.v20210503-0343, Apache-2.0 OR LGPL-2.1-or-later, approved, CQ23217
12:20:29  
12:20:29  Checking for restricted dependencies...
12:20:29  maven/mavencentral/org.codehaus.groovy/groovy-groovydoc/3.0.8, Apache-2.0, restricted, clearlydefined
12:20:29  maven/mavencentral/org.codehaus.groovy/groovy/3.0.8, Apache-2.0, restricted, clearlydefined

It seems 2 dependencies that were OK yesterday, are no longer OK today. I assume the ClearlyDefined score has changed or so. How should things like this be handled? Randomly changing outcomes like this lead to significant, where something that was allowed suddenly isn't allowed anymore. It's not very stable this way, unlike CQs, which remain approved once they are approved. Has the Eclipse Foundation thought about this when they introduced the option to use ClearlyDefined data?

[waynebeaton]

Theoretically, a CQ could change state. In part, the purpose of CQs was to track usage so that--in the event that we learn of a mistake or oversight--we can figure out where mitigation will be required. By way of expectation management, it's entirely possible (and desirable) that new information (e.g., a curation) may change whether or not a dependency passes.

But I don't think that this is the actual problem here.

I made a change to the logic that determines whether or not ClearlyDefined data is acceptable. Previously, we accepted an overall score > 60 or a license score > 60. In consideration of issue #67, I changed this to just look at the license score. As I stated in that issue, I'm not at all certain that the score is actually interesting. I need to put more thought into this. I'll see what I can sort out tomorrow afternoon.

In the meantime, I've run the groovy/3.0.8 though the IP Due Diligence process and it will pass now.

It looks like groovydoc is passing now. I'm not sure why it failed six days ago; given the date of the last update of the ClearlyDefined record, it's possible that their harvester came up with a better result in the meantime.

[dhendriks]

It seems the current 'Licensed' score is only 60, for groovydoc. I've created IPLab issue #819 for this.

[waynebeaton]

FWIW, I noticed that the Maven plugin was configured to use a higher threshold for the ClearlyDefined score than the CLI. I've set them to the same value (60). That would explain why groovydoc passed for me, but not for you.

[waynebeaton]

I'm assuming that either we've fixed the problem or you accept my answer. Reopen if I've made a terrible error.