From fa878796f5c17e6f3729d80ffdc969077720dd4c Mon Sep 17 00:00:00 2001 From: Bertrand Chevrier Date: Tue, 4 Mar 2025 15:55:50 +0100 Subject: [PATCH 1/2] feat: security jfrog --- docs/hub/_toctree.yml | 2 ++ docs/hub/security-jfrog.md | 25 +++++++++++++++++++++++++ docs/hub/security.md | 1 + 3 files changed, 28 insertions(+) create mode 100644 docs/hub/security-jfrog.md diff --git a/docs/hub/_toctree.yml b/docs/hub/_toctree.yml index 611d832f0..ab5e9b512 100644 --- a/docs/hub/_toctree.yml +++ b/docs/hub/_toctree.yml @@ -390,6 +390,8 @@ title: Secrets Scanning - local: security-protectai title: "Protect AI" + - local: security-jfrog + title: "JFrog" - local: moderation title: Moderation - local: paper-pages diff --git a/docs/hub/security-jfrog.md b/docs/hub/security-jfrog.md new file mode 100644 index 000000000..a48ac92f0 --- /dev/null +++ b/docs/hub/security-jfrog.md @@ -0,0 +1,25 @@ +# Third-party scanner: JFrog + + +[JFrog](https://protectai.com/)'s security scanner detects malicious behavior in machine learning models. + +![JFrog report for the danger.dat file contained in mcpotato/42-eicar-street](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/jfrog-report.png) +*Example of a report for [danger.dat](https://huggingface.co/mcpotato/42-eicar-street/blob/main/danger.dat)* + +We partnered with JFrog to provide scanning in order to make the Hub safer. Model files are scanned by the JFrog scanner and reports you the scanning result. + +
+ + +
+ +Here is an example repository you can check out to see the feature in action: [mcpotato/42-eicar-street](https://huggingface.co/mcpotato/42-eicar-street). + +## Model security refresher + +To share models, we serialize the data structures we use to interact with the models, in order to facilitate storage and transport. Some serialization formats are vulnerable to nasty exploits, such as arbitrary code execution (looking at you pickle), making sharing models potentially dangerous. + +As Hugging Face has become a popular platform for model sharing, we’d like to protect the community from this, hence why we have developed tools like [picklescan](https://github.com/mmaitre314/picklescan) and why we integrate third party scanners. + +Pickle is not the only exploitable format out there, [see for reference](https://github.com/Azure/counterfit/wiki/Abusing-ML-model-file-formats-to-create-malware-on-AI-systems:-A-proof-of-concept) how one can exploit Keras Lambda layers to achieve arbitrary code execution. + diff --git a/docs/hub/security.md b/docs/hub/security.md index a4190a6eb..a84763a69 100644 --- a/docs/hub/security.md +++ b/docs/hub/security.md @@ -21,4 +21,5 @@ For any other security questions, please feel free to send us an email at securi - [Pickle Scanning](./security-pickle) - [Secrets Scanning](./security-secrets) - [Third-party scanner: Protect AI](./security-protectai) +- [Third-party scanner: JFrog](./security-jfrog) - [Resource Groups](./security-resource-groups) From fced21ab3628af600c7cf3d758336815271993cc Mon Sep 17 00:00:00 2001 From: Luc Georges Date: Tue, 4 Mar 2025 17:26:07 +0100 Subject: [PATCH 2/2] feat: add description of JFrog scanner functionality --- docs/hub/security-jfrog.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/hub/security-jfrog.md b/docs/hub/security-jfrog.md index a48ac92f0..49f7d118e 100644 --- a/docs/hub/security-jfrog.md +++ b/docs/hub/security-jfrog.md @@ -1,13 +1,15 @@ # Third-party scanner: JFrog -[JFrog](https://protectai.com/)'s security scanner detects malicious behavior in machine learning models. +[JFrog](https://jfrog.com/)'s security scanner detects malicious behavior in machine learning models. ![JFrog report for the danger.dat file contained in mcpotato/42-eicar-street](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/jfrog-report.png) *Example of a report for [danger.dat](https://huggingface.co/mcpotato/42-eicar-street/blob/main/danger.dat)* We partnered with JFrog to provide scanning in order to make the Hub safer. Model files are scanned by the JFrog scanner and reports you the scanning result. +JFrog's scanner is built with the goal to reduce false positives. Indeed, what we currently observe is that code contained within model weights is not always malicious. When code is detected in a file, JFrog's scanner will parse it and analyze to check for potential malicious usage. +