License violations don't include Severity. Also lots of licenses get reported as Unknown.

[imranzunzani]

Describe the bug

When reporting license violations, severity is not reported, both in the PR scan as well as Repo scan. Xray on-demand scans do include severity information for licenses. Also, a lot of licenses are reported as Unknown when using a watch with policy to report Unknown licenses. Please see the screenshots uploaded in 'Current behaviour'.

Current behavior

In PR scan:

In Repo scan:

Reproduction steps

Run Frogbot scans with a watch configured to report license violations along with unknown licenses.

Expected behavior

Severity is reported, so higher ones could be spotted over lower ones. There should not be too many unknown licenses, only legitimate cases should be reported.

JFrog Frogbot version

2.19.10

Package manager info

npm. package.json.

Git provider

GitHub

JFrog Frogbot configuration yaml file

No response

Operating system type and version

Github Actions. Ubuntu 22.04.4.

JFrog Xray version

3.87.9

[attiasas]

Hi @imranzunzani,
Thank you for using Frogbot!

It seems like this issue is related to how you've configured your watch and policy. You can find more information about configuring trigger violations using Xray policy rules in the following link: Xray Policy Rules Documentation

The entries you're seeing are for packages with unknown licenses, which we treat as violations. You can remove these types of violations by editing the rule.

To do so, you can remove the option "Disallow Unknown Licenses", that is shown as enabled in the screenshot below:

Please feel free to reach out if you have any further questions or encounter any issues.

[imranzunzani]

Hi @attiasas ,
Thank you for the response.
We actually want the 'Disallow Unknown License' to report the unknown licenses, but the problem seems to be that it is reporting too many licenses as 'Unknown' which actually have licenses associated. Eg. in the screenshot, you could see hasown which is MIT, but reported as Unknown. You can check this at https://github.com/inspect-js/hasOwn/tree/v2.0.0?tab=readme-ov-file.
Also, could you please check and confirm, if there are plans to include 'Severity' information?

[attiasas]

Hi @imranzunzani,

Regarding the absence of 'Severity' information regarding the license, currently, it's not displayed. However, we have added it to our backlog, and it will be available in future versions after prioritization.

"Unknown" licenses may arise from various situations:

• The component license exists but isn't identified by Xray, meaning it's not in Xray's license database. In such cases, an Admin can add a Custom License via the UI (Administration > Xray > settings > Compliance Licenses). Once added to the Xray DB, if the artifact contains the licenses Xray can detect it.
• The component isn't in our Software catalog (XUC), or it's in our catalog but lacks license info, and the artifact doesn't include any license info in the component. In such cases, the user may assign a license to the component from the Descendants screen.

[imranzunzani]

Hi @attiasas ,
Thank you for adding the 'Severity' info, for the future releases.

Regarding the 'Unknown' licenses, it seems like an anomalous behaviour, as the license info is there in Xray (Artifactory), but it is getting reported as 'Unknown'.
Eg. For the neo-async package, the following information is present:

but that still gets flagged as 'Unknown'.
Could you please check, if this could be happening due to any reason with the logic at Frogbot, or should this be raised to JFrog support, so that it gets looked into by Xray?

[Avi706]

Hi @attiasas,

Thank you for the clarification. Regarding your 'Unknown' license related various situations (point 2), please find these explanations with attached datasets as mentioned below:

2. The component isn't in our Software catalog (XUC), or it's in our catalog but lacks license info, and the artifact doesn't include any license info in the component. In such cases, the user may assign a license to the component from the Descendants screen.

Explanation: As mentioned below "xmlchars 2.2.0" package is present in the artifactory and x-ray is able to identify its license information. This showcase that the package metadata is present within the Jfrog Software catalog (XUC) along with the license info.
On the other hand, frogbot is not able to identify same package's license information.

My assumption is that the frogbot and x-ray are pointing to the same knowledge base. If so, then frogbot should be able to showcase the same result set as its shown in the xray. Let me know if its otherwise.

[orz25]

Hey @Avi706 @imranzunzani
Xray on demand scanning and Frogbot scanning have different scan methodologies.
Xray on-demand scans a binary file while Frogbot scans a source code, the context of these 2 methodologies is different, and therefore the results may vary. In this case, for npm package xmlchars 2.2.0 we do miss license information in our Software catalog (XUC). We will work on adding it.

[imranzunzani]

Hi @orz25,
Thank you for the response. Please do note that there are many packages getting reported as with 'Unknown' licenses, not just xmlchars 2.2.0, so this behaviour would need fixing some logic systemically in XUC, we think.

[orz25]

@imranzunzani We will take a look at this. Thank you for the feedback!

[imranzunzani]

Hi @attiasas and @orz25,
I have submitted #672, which adds the 'Severity' column to the PR's Licenses violations info. Please consider that change, as it is required for us. As we also need that information in the Repository Scan results as well, i.e. in GHAS's SARIFs submission, we would appreciate if you could look into that portion of the functionality. Currently the classification on the list (code-scanning) is getting reported as 'Error' and 'Note', which is not helpful for filtering:

As mentioned before (screenshot), the information is not present in the description either, when clicked on an element in the list.

[attiasas]

Hi @imranzunzani,

Thank you for your submission. I will review it as soon as possible. Additionally, I will address the issue you have raised.

Could you please provide more details about the information you expect for your filtering? What specific information is missing? Are you referring to the severity? Kindly elaborate on your requirements.

Thank you.

[imranzunzani]

Hi @attiasas ,

Thank you for looking into this.

The PR that I have submitted adds the Severity Field for the PR Scans. Hope you would review that soon.

Other than that, there are two requirements:

1. 'Severity' is required for filtering the violations in the list of alerts in 'code scanning' for the Repository Scan. It works for the CVEs, but is not there for the license violations, as the screenshots show.
2. There is the problem of licenses being flagged as 'Unknown', while they are fine in Xray. Could this be related to the untrimmed spaces, as mentioned in this PR: Only trim spaces of array parameter entries #670? Could you please take a look, if that could to be the problem, for the licenses getting flagged as 'Unknown'?

[attiasas]

Hi @imranzunzani,

The pull request you opened has been merged, and the fix will be available in the next release of Frogbot.

About the requirements:

1. Changes to the Scan-Repository command (which generates the SARIF file) should be implemented here: Link to code. We always add the security-severity property equal to 0 when we need to calculate it based on severity.

2. As mentioned by @orz25, the two scans are different. Frogbot scans source-code, while Xray-On-Demand scans binary. When scanning source code, we only send the dependencies ID. If it exists in our database, we return the license information. When scanning binary, we are using a different method. To change this behavior, we need to investigate and prioritize a solution. If you want, you can pass us a list of the dependencies with their license information, and we will check if we can add it to our general catalog database. If not, you can always add it yourself as mentioned above.

[imranzunzani]

Hi @attiasas ,

Thank you for merging the PR.
Regarding the requirements, @Avi706 will take it up from here on those, probably with the ESL.

[eranturgeman]

Hello @imranzunzani and @Avi706
does this issue has been resolved? do you need any further assistance with it?
If not- can I close the ticket?

[Avi706]

Hello @eranturgeman, thanks for reaching out to us. We managed to fix the unknown license issue in x-ray and we are going to test the same in Frogbot soon to check the behavior and update here.

[eranturgeman]

Great @Avi706
Let me know if all worked out and I can resolve this issue :)

[pru-qmir]

Hi @imranzunzani,

The pull request you opened has been merged, and the fix will be available in the next release of Frogbot.

About the requirements:

1. Changes to the Scan-Repository command (which generates the SARIF file) should be implemented here: Link to code. We always add the security-severity property equal to 0 when we need to calculate it based on severity.
2. As mentioned by @orz25, the two scans are different. Frogbot scans source-code, while Xray-On-Demand scans binary. When scanning source code, we only send the dependencies ID. If it exists in our database, we return the license information. When scanning binary, we are using a different method. To change this behavior, we need to investigate and prioritize a solution. If you want, you can pass us a list of the dependencies with their license information, and we will check if we can add it to our general catalog database. If not, you can always add it yourself as mentioned above.

---

Hi @attiasas

Was #2 addressed? We have also observed this issue concerning Frogbot's detection of Licenses linked to some dependencies. Several components are being indicated with an 'Unknown' license in Frogbot Scan. However, upon cross-checking within Artifactory and Xray, we have found that these reported components indeed have an license associated with them.

Eg: For a python fast-api project, several (indirect) dependencies are being flagged with an unknown license

The impacted dependency “Pygments==2.17.2” is listed as Unknown License but in Artifactory it is associated with BSD-2-Clause License

And also, frogbot is unable to detect license for core packages for the project (python, root)

[attiasas]

Hi @pru-qmir,

Regarding point #2, we do not support detecting licenses in first-party source code. We only support license detection when the code is in binary form or for dependencies. If you need this feature, please let us know, and we will add it to our backlog for implementation.

As for the other issues you have raised:

The row related to root is a bug and should not appear. I believe I know the cause and will work on resolving it.
Regarding Pygments 2.17.2, we are currently investigating, is it also the case for typing-extensions \ python?

I will provide updates on the open issues as soon as I can.

[pru-qmir]

Regarding Pygments 2.17.2, we are currently investigating, is it also the case for typing-extensions \ python?

Yes, that's correct for typing-extensions. I am not sure for python, I don't see this binary in Artifactory. Should python not appear similar to root?