Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Feature types #672

Merged
merged 12 commits into from
Feb 20, 2019

Conversation

KeyboardNerd
Copy link
Contributor

@KeyboardNerd KeyboardNerd commented Dec 12, 2018

The PR is to differentiate packages by their types, and vulnerabilities to affect different types of features.

Change set:

  1. Feature Detectors: Extract both binary and source packages from image layers

  2. Vulnerability matching algorithm change: Each vulnerability matches a specific type of package, either binary or source package.

Vulnerability source Matching Package Type
Debian: "https://security-tracker.debian.org/tracker" matching source package name
Ubuntu: "https://git.launchpad.net/ubuntu-cve-tracker" matching source package name
Alpine: "https://github.com/alpinelinux/alpine-secdb" matching binary package name
Oracle: "https://linux.oracle.com/oval/" matching binary package name
RHEL: "https://www.redhat.com/security/data/oval/" matching binary package name

Please correct the above assumption for each vulnerability data source if any error.

  1. API change: Clair returns all found binary and source packages to the client, and let the client to filter.
    For each Feature, we add a field to indicate the type of the feature, currently it's Binary or Source package.

@KeyboardNerd KeyboardNerd changed the title Source -> Binary package: Rename affected feature type to feature type Structured layer feature for supporting source -> binary features Dec 12, 2018
@KeyboardNerd KeyboardNerd changed the title Structured layer feature for supporting source -> binary features WIP: Structured layer feature for supporting source -> binary features Feb 12, 2019
@KeyboardNerd KeyboardNerd changed the title WIP: Structured layer feature for supporting source -> binary features WIP: Source package to binary package mapping Feb 12, 2019
@KeyboardNerd KeyboardNerd changed the title WIP: Source package to binary package mapping WIP: Implement Package types Feb 13, 2019
@KeyboardNerd KeyboardNerd force-pushed the source_package/feature_type branch 2 times, most recently from bd4a4ec to a42632d Compare February 14, 2019 15:53
feature_type is for differentiating the binary packages and source
packages.
@KeyboardNerd KeyboardNerd changed the title WIP: Implement Package types Implement Package types Feb 19, 2019
@KeyboardNerd KeyboardNerd changed the title Implement Package types Implement Feature types Feb 19, 2019
Copy link
Contributor Author

@KeyboardNerd KeyboardNerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comments for PR reviewer

@@ -1,26 +0,0 @@
// Copyright 2018 clair authors
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is renamed to database/feature_type.go


// ErrMissingEntities is an error that occurs when an associated immutable
// entity doesn't exist in the database. This error can indicate a wrong
// implementation or corrupted database.
ErrMissingEntities = errors.New("database: associated immutable entities are missing in the database")
ErrMissingEntities = NewStorageError("associated immutable entities are missing in the database")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one by one we're moving to use special error type for the database.

@@ -155,18 +155,33 @@ type Namespace struct {
VersionFormat string
}

func NewNamespace(name string, versionFormat string) *Namespace {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we're moving to use the proper initializer for the database structs


fmt.Printf("%d features, %d vulnerabilities are generated", len(nsFeatures), len(vulnerabilities))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the concurrent insertion because the vulnerabilities <-> feature relationships are cached after the features are inserted into the database

The featurefmt now extracts both binary packages and source packages
from the package manager infos.
Remove source name/version fields
Add Type field to indicate if it's binary package or source package
postgres 9.4 doesn't support ON CONFLICT, which is required in our
implementation.
API now returns every feature found by the detectors and also indicates
the type of feature.
@@ -179,6 +194,11 @@ type NamespacedFeature struct {
Namespace Namespace
}

func NewNamespacedFeature(namespace *Namespace, feature *Feature) *NamespacedFeature {
// TODO: namespaced feature should use pointer values
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it only desirable to use pointers here to save memory?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's idiomatic to use the pointer value for structs

database/pgsql/complex_test.go Outdated Show resolved Hide resolved
database/pgsql/complex_test.go Show resolved Hide resolved
database/pgsql/complex_test.go Outdated Show resolved Hide resolved
database/pgsql/feature.go Outdated Show resolved Hide resolved
@KeyboardNerd KeyboardNerd merged commit 73bc2bc into quay:master Feb 20, 2019
@KeyboardNerd KeyboardNerd deleted the source_package/feature_type branch February 20, 2019 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants