Photon-OS ClairCore Support

[MVrachev]

Hi, I have already spent a lot of time working on this pr for Clair: https://github.com/quay/clair/pull/886/files.

After contacting @ldelossa I was advised to move the pr here.

Can you please help me figure out how much work will that be?

[ldelossa]

@MVrachev

Can we get some information about Photon OS first.
What package management technology does it use? e.g. rpm, dpkg ?
Are container images publicly available ?
Does VMWare have their own security tracker for photo OS issues?

[MVrachev]

Does VMWare have their own security tracker for photo OS issues?

Yes, we have our own security tracker for Photon OS security vulnerabilities. We are uploading the results in the repository itself as a security advisory https://github.com/vmware/photon/wiki/Security-Advisories and as a JSON file for every single Photon OS release here: https://vmware.bintray.com/photon_cve_metadata/

Are container images publicly available ?
Yes, they are https://hub.docker.com/_/photon?tab=description

I will investigate your first question and I will answer on this issue.

[ldelossa]

@MVrachev
This is a good start. Let me review your security data.

[MVrachev]

What package management technology does it use? e.g. rpm, dpkg ?

Photon is using RPM as package management technology.

[ldelossa]

@MVrachev
great, it's likely you can re-use our RPM package scanner.

I'm looking at your docker files and it appears photon releases have well defined OS release files:

❯ docker run --rm photon:$release "/bin/sh" "-c" "cat /etc/os-release"
Unable to find image 'photon:3.0' locally
3.0: Pulling from library/photon
663010993c44: Pull complete 
Digest: sha256:9785fc642d79e7d2f89ab82c289098b7e40383d86a54f85b8f6475598b627c57
Status: Downloaded newer image for photon:3.0
NAME="VMware Photon OS"
VERSION="3.0"
ID=photon
VERSION_ID=3.0
PRETTY_NAME="VMware Photon OS/Linux"
ANSI_COLOR="1;34"
HOME_URL="https://vmware.github.io/photon/"
BUG_REPORT_URL="https://github.com/vmware/photon/issues"

~/git 6s
❯ release=3.0-20200202                                                

~/git
❯ docker run --rm photon:$release "/bin/sh" "-c" "cat /etc/os-release"
Unable to find image 'photon:3.0-20200202' locally
3.0-20200202: Pulling from library/photon
Digest: sha256:9785fc642d79e7d2f89ab82c289098b7e40383d86a54f85b8f6475598b627c57
Status: Downloaded newer image for photon:3.0-20200202
NAME="VMware Photon OS"
VERSION="3.0"
ID=photon
VERSION_ID=3.0
PRETTY_NAME="VMware Photon OS/Linux"
ANSI_COLOR="1;34"
HOME_URL="https://vmware.github.io/photon/"
BUG_REPORT_URL="https://github.com/vmware/photon/issues"

~/git
❯ release=2.0-20200202                                                

~/git
❯ docker run --rm photon:$release "/bin/sh" "-c" "cat /etc/os-release"
Unable to find image 'photon:2.0-20200202' locally
2.0-20200202: Pulling from library/photon
4658afbc405e: Pull complete 
Digest: sha256:00a0d6bb13600c845f18221eefd1fcfadcbf2f099f1eb597f9bb8cfcb75e4cd9
Status: Downloaded newer image for photon:2.0-20200202
NAME="VMware Photon OS"
VERSION="2.0"
ID=photon
VERSION_ID=2.0
PRETTY_NAME="VMware Photon OS/Linux"
ANSI_COLOR="1;34"
HOME_URL="https://vmware.github.io/photon/"
BUG_REPORT_URL="https://github.com/vmware/photon/issues"

Would you like to take a stab at writing a DistributionScanner for PhotonOS?
You can follow an example here:
https://github.com/quay/claircore/blob/master/debian/distributionscanner.go

The distribution scanner should return a *claircore.Distribution struct representing the discovered PhotoOS release in the given layer. You are free to use any files that may exist in a PhotoOS container layer, but OS-Release looks pretty good for you guys.

We have been hard coding the claircore.Distribution structures for now - like so: https://github.com/quay/claircore/blob/master/debian/releases.go
When we parse your security database we will use those same structs to tag your vulnerabilities with a distribution. So far this is the easiest method for us to have consistent matching.

If you'd like to open a PR for the DistributionScanner first, we can work together on it. Once that's complete we can work on parsing your vulnerability database.

[MVrachev]

Hello again @ldelossa.
Right now I don't have the time to work on this issue but I will try to ensure that someone will work on this issue in near future.

[ldelossa]

@ppadmavilasom @MVrachev
so far so good, indexing is working.

We now need to create a few things to match Vulnerabilities to a Photo-OS IndexReport.
These interfaces must be implemented:

https://github.com/quay/claircore/blob/master/libvuln/driver/matcher.go#L37
https://github.com/quay/claircore/blob/master/libvuln/driver/updater.go#L13

For Updater, you will want to implement the full interface, including embedded interfaces.

A good example is ubuntu:

https://github.com/quay/claircore/blob/master/ubuntu/matcher.go
https://github.com/quay/claircore/blob/master/ubuntu/updater.go

The idea here is to fetch your security database, parse it into claircore.Vulnerability structures, and then implement a Matcher which tells clairore how to match your packages to the vulnerabilities.

This portion can be a little abstract so ping me if you need assistance.

[ppadmavilasom]

@ldelossa appreciate your help with this. will work on the rest.
Do you follow readme local dev and test during development?

[ldelossa]

@ppadmavilasom we try to drive ppl to our mdbook documentation:
https://quay.github.io/claircore/

I am a heavy user of the local development environment. So I will often use
make local-dev-up to get all the containers running, and our cli tool cctool to run a report against our local dev environment.

❯ cctool report -dump docker.io/ubuntu:latest

You are able to pass cctool a repository/namespace:tag and it will pull the layers and create a IndexReport and a Vulnerability report for you. cc @hdonnay if you want to add anything.

Tho you make a good point, we need to get better at testing our IndexReports.

[ldelossa]

@ppadmavilasom currently Clair tests are failining on vmware:

fetch component=pkg/ovalutil/Fetcher.Fetch database=https://packages.vmware.com/photon/security/oval/com.vmware.phsa-photon3.xml

is your sec db down ?

[ppadmavilasom]

@ldelossa sorry about this. there was a url change as there is a new process to publish which changed locations for better consolidation. this change is final and i did a PR here: #155

[ldelossa]

@ppadmavilasom not a problem, thanks for the quick PR!

[hdonnay]

We're currently having problems where the xml document has missing and nonsensical dates in the issued and updated elements, and possibly others.

Empty date strings are now dealt with in our OVAL package, but there's the problem of the nonsensical dates. For example, I don't think PHSA-2020:00033 was actually issued in the early third century.

[smaliakkal]

@hdonnay Thanks for checking and sorry for this.

We used PHSA-2020:00033 as an id in the document not as a date.
similarly dates used like following.

Should these date be in any other format ?

Is there any other's distro's document I can use as a reference ?

[hdonnay]

@smaliakkal Sorry, I was pointing out that PHSA-2020:00033 has a date of 209-09-10, which seems wrong. There may be others, but that's where our importing is falling down first.

[smaliakkal]

@hdonnay Thanks for pointing.

[ldelossa]

@ppadmavilasom any work being done to continue this integration?

[smaliakkal]

@ldelossa we have corrected bogus date and empty dates ?
Could you try now ?

[ldelossa]

@smaliakkal I was referring to the original topic of his thread. currently photon-os updater is working and the distribution scanner is working, but a Matcher must be implemented.

[suezzelur]

@ldelossa we will send a PR in the next couple of weeks

[ashwin-h]

@suezzelur @ppadmavilasom @ldelossa @smaliakkal
With below PR, photon based container images should work with claircore
#185

[ldelossa]

merged! thanks for contributing.

[ashwin-h]

Thanks @ldelossa