Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix build of container image for native executable #195

Merged
merged 1 commit into from
Jan 4, 2023

Conversation

nscuro
Copy link
Member

@nscuro nscuro commented Jan 4, 2023

Signed-off-by: nscuro nscuro@protonmail.com

Signed-off-by: nscuro <nscuro@protonmail.com>
@nscuro nscuro added defect Something isn't working domain/vuln-analysis labels Jan 4, 2023
Copy link
Collaborator

@VinodAnandan VinodAnandan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kodiakhq kodiakhq bot merged commit 29b4d31 into main Jan 4, 2023
@kodiakhq kodiakhq bot deleted the fix-native-container-build branch January 4, 2023 09:23
@sonarcloud
Copy link

sonarcloud bot commented Jan 4, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

sahibamittal added a commit that referenced this pull request Jan 4, 2023
commit 3a6b8ef
Author: Niklas <nscuro@protonmail.com>
Date:   Wed Jan 4 10:44:12 2023 +0100

    Use GH Action to build and push container image in one step (#196)

    Signed-off-by: nscuro <nscuro@protonmail.com>

    Signed-off-by: nscuro <nscuro@protonmail.com>

commit 29b4d31
Author: Niklas <nscuro@protonmail.com>
Date:   Wed Jan 4 10:23:26 2023 +0100

    Fix build of container image for native executable (#195)

    Signed-off-by: nscuro <nscuro@protonmail.com>

    Signed-off-by: nscuro <nscuro@protonmail.com>

commit 68de797
Author: Niklas <nscuro@protonmail.com>
Date:   Tue Jan 3 22:30:31 2023 +0100

    Implementation of stateful retries for vulnerability analyzers (#174)

    * Rough implementation of stateful retries for OSS Index

    Implements the OSS Index analyzer as stream processor in order to have access to state stores. Because processors can only forward or drop records, they can't publish records to arbitrary channels, just the one they're connected to in the topology.

    Incoming records are assembled into batches of up to 128. Batches are analyzed when their size exceeds 128, or 5 seconds since the last batch analysis have passed.

    Records that failed with a "retryable reason" are written to a separate state store, with additional retry metadata. The retry store is checked every second for records that are due for retry. Records due for retry are re-added to the batch. The retry mechanism uses exponential backoff with jitter.

    A circuit breaker is added so that retries do not overwhelm the OSS Index API. Per the current default, if 50% out of 100 requests to OSS Index fail, the circuit is opened for 15 seconds.

    The `VulnerabilityResult` class has been extended with a `failureReason` field, that can be used to communicate *if* and *why* the analysis of a given record failed. Just a quick & dirty solution.

    Cache is still missing from this implementation.

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Move retry logic into abstract processor class

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Keep track of entry counts in retry state stores

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Remove unused processors

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Track number of entries in batch state stores for OSS Index

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Add initial test for SnykProcessor

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Completely migrate analyzers to processors

    Also move analyzer configs into centralized classes, add configuration classes for dependencies of processors, add support for multiple Snyk tokens, make usage of batch state store for OSS Index more efficient, configure HTTP client timeouts for both OSS Index and Snyk.

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Make `RetryableRecord` extend `Record`

    Makes treating of retryable records more seamless, and also retains metadata like headers.

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Unify retry config; Re-enable SnykClientTest

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Add OssIndexProcessor tests

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Migrate internal analyzer to processor

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Fix missing `@Transactional` context for internal scanner processor

    Can't use `@Transactional` here as processors are not (and can not be) CDI-managed beans.

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Update processors for recent cache and completion changes

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Disable k8s dev services

    It starts a cluster which we do not use

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Add tests for AuthorizationClientHeaderFactory

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Fix Vulnerability being marked as deprecated

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Report analyzer identity `NONE` for records with unknown identifiers

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Make sonar happy

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Fix test; Reduce retry log level to `debug`

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Port circuit breaker tests

    Also populate circuit breaker configs with default values as per https://resilience4j.readme.io/docs/circuitbreaker#create-and-configure-a-circuitbreaker and remove the, now unused, CircuitBreakerCustom class.

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Update Snyk property names in docker-compose.yml

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Fix "No serializer found for class" exceptions when running as native image

    Classes that are (de-)serialized with Jackson have to be explicitly registered for reflection, see https://quarkus.io/guides/writing-native-applications-tips#registering-for-reflection

    Also fix `ClassNotFoundException: org.apache.commons.logging.impl.LogFactoryImpl`, see quarkusio/quarkus#10128

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Fix broken merge artifacts

    Signed-off-by: nscuro <nscuro@protonmail.com>

    Signed-off-by: nscuro <nscuro@protonmail.com>

commit ce46eb9
Author: mehab <meha.bhargava2@gmail.com>
Date:   Tue Jan 3 20:17:56 2023 +0000

    Feature/trygraal (#194)

    * added changes for graalvm native image

    * resolved merge conflicts

    * resolved merge conflicts

    * separating out build jar and build native

    * separating out build jar and build native

    * renaming file to see build jar work correctly

    * renaming file to see build jar work correctly

    * renaming file to see build jar work correctly

    * changes

    * trying from main branch

    * trying from main branch

    * changed name

    * change name

    * degrading quarkus to 2.14 to see if it works

    * try with 2.15.0

    * reduced to version 2.14.2.final

    * reduced to version 2.14.2.final

    * renaming image

    * removing snapshot in caps

    * removing snapshot in caps

    * another test

    * another test

    * another test

    * another test

    * trying quarkus container build

    * trying quarkus container build

    * trying quarkus container build

    * trying with ls and copy command

    * trying with commented out docker file

    * trying with commented out docker file

    * trying docker commands directly

    * trying docker commands directly

    * trying docker commands directly

    * trying docker commands directly

    * trying docker commands directly

commit aae6b1a
Author: Niklas <nscuro@protonmail.com>
Date:   Tue Jan 3 16:59:37 2023 +0100

    Add script to generate test fixture BOMs for load testing (#189)

    Closes #162

    Signed-off-by: nscuro <nscuro@protonmail.com>

    Signed-off-by: nscuro <nscuro@protonmail.com>

commit 0d2f490
Author: mehab <meha.bhargava2@gmail.com>
Date:   Tue Jan 3 11:47:17 2023 +0000

    Feature/trygraal (#193)

    * added changes for graalvm native image

    * resolved merge conflicts

    * resolved merge conflicts

    * separating out build jar and build native

    * separating out build jar and build native

    * renaming file to see build jar work correctly

    * renaming file to see build jar work correctly

    * renaming file to see build jar work correctly

    * changes

    * trying from main branch

    * trying from main branch

    * changed name

    * change name

commit 543b6f1
Author: Niklas <nscuro@protonmail.com>
Date:   Tue Jan 3 10:59:54 2023 +0100

    Simplify handling of secret key (#188)

    Signed-off-by: nscuro <nscuro@protonmail.com>

    Signed-off-by: nscuro <nscuro@protonmail.com>

commit cdf56f1
Author: Vinod Anandan <vinod@owasp.org>
Date:   Fri Dec 30 20:08:41 2022 +0000

    Downgrade dependency io.pebbletemplates:pebble to v3.1.6 (#187)

    Downgrade dependency io.pebbletemplates:pebble to v3.1.6

commit 7e5e8d6
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Fri Dec 30 19:57:59 2022 +0000

    Update dependency io.pebbletemplates:pebble to v3.2.0 (#113)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit e721dad
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Fri Dec 30 19:57:23 2022 +0000

    Update dependency io.quarkiverse.helm:quarkus-helm to v0.2.1 (#182)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 0f91f10
Author: mehab <meha.bhargava2@gmail.com>
Date:   Fri Dec 30 18:21:13 2022 +0000

    fix version for native binary (#186)

commit 2138b6c
Author: mehab <meha.bhargava2@gmail.com>
Date:   Fri Dec 30 18:18:47 2022 +0000

    graalvm native binary creation (#185)

commit 262c81b
Author: mehab <meha.bhargava2@gmail.com>
Date:   Fri Dec 30 18:14:31 2022 +0000

    graalvm native binary tryout (#184)

commit 06309d5
Author: mehab <meha.bhargava2@gmail.com>
Date:   Fri Dec 30 17:52:28 2022 +0000

    change to graalvm image generation (#183)

commit 41ac711
Merge: 0b2812e cfcedad
Author: japurva1502 <111765683+japurva1502@users.noreply.github.com>
Date:   Fri Dec 30 11:31:10 2022 -0600

    Merge pull request #177 from mehab/feature/helmCharts

    Feature/helm charts

commit cfcedad
Author: mehab <meha.bhargava2@gmail.com>
Date:   Fri Dec 30 17:17:40 2022 +0000

    Update InternalAnalyzer.java

    removing duplicate @transactional annotation

commit 78419a0
Merge: a1cd404 0b2812e
Author: mehab <meha.bhargava2@gmail.com>
Date:   Fri Dec 30 16:57:38 2022 +0000

    Merge branch 'main' into feature/helmCharts

commit 0b2812e
Author: japurva1502 <111765683+japurva1502@users.noreply.github.com>
Date:   Fri Dec 30 10:50:04 2022 -0600

    Feature/analyzer completion (#180)

    * dummy PR

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Internal Analyzer changes:
    Added persistence classes
    Added repository for querying data
    restructure internal analyzer

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Internal Analyzer changes:
    Added persistence classes
    Added repository for querying data
    restructure internal analyzer

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed schema validation for newly added persistent entities

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed schema validation for newly added persistent entities

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed schema and updated TopologyTest to include internal Analyzer

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed few enities Schema
    Added integration test for Internal Analyzer
    Unit Test for ComponentRepository

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed PK violation in test cases

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Test cases for VulnerableSoftwareRepository
    InternalAnalyzer

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Test cases

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Sonar warning

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Internal analyzer cache implementation changes

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * add analysis test for more coverage

    * Squashed commit of the following:

    commit ef6135c
    Author: Sahiba Mittal <sahibamittal98@gmail.com>
    Date:   Thu Dec 15 20:53:23 2022 +0530

        Issue 125 : Caching repo meta analyzer (#161)

        * WIP

        * cache moved to common, added for meta analyzer

        * Update RepositoryMetaAnalyzerTopology.java

        * Update RepositoryMetaAnalyzerTopology.java

        * fix code smells

        * Update MetaAnalyzerCacheKey.java

        * test in progress

        * tests fixed

        * fix recursive bug

        * test getName added

    commit af26631
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Thu Dec 15 15:23:11 2022 +0000

        Update docker.redpanda.com/vectorized/redpanda Docker tag to v22.3.8 (#166)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit 94b7040
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Wed Dec 14 22:22:34 2022 +0000

        Update dependency us.springett:alpine-common to v2.2.0 (#165)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit 88ac8e7
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Wed Dec 14 16:50:22 2022 +0000

        Update docker.redpanda.com/vectorized/redpanda Docker tag to v22.3.6 (#160)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit fbf451e
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Wed Dec 14 16:50:13 2022 +0000

        Update prom/prometheus Docker tag to v2.40.7 (#163)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit 4b60147
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Wed Dec 14 16:49:46 2022 +0000

        Update actions/setup-java action to v3.9.0 (#164)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit bf18e9e
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Fri Dec 9 20:06:10 2022 +0000

        Update resilience4j to v2.0.2 (#158)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit 5d85507
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Fri Dec 9 19:54:00 2022 +0000

        Update returntocorp/semgrep-action digest to 031b0ac (#157)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit 7116bcd
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Fri Dec 9 19:53:33 2022 +0000

        Update prom/prometheus Docker tag to v2.40.6 (#159)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit 3eab7f9
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Thu Dec 8 11:24:14 2022 +0000

        Update actions/setup-java action to v3.8.0 (#154)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit a67ff17
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Thu Dec 8 11:23:58 2022 +0000

        Update returntocorp/semgrep-action digest to 027b0fd (#155)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    * Create SeverityConverterTest.java

    * Test coverage

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Refactored repo classes

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * add component version test

    * PR review fixes

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Internal Analyzer changes for CPE and Purl

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Returning VulnrabilityResult with null vulnerability when no vulns are found

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Wip : Analyzer completion

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed errors in ser/deser and JPA exception because of missing @transactional annotation

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed broken test cases

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed Typo

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed internalAnalyzer property

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Test coverage

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Test coverage

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Setting Vuln description as blank

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed broken test case

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed code smell

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed code smell

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * corrected internal cache name

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * removed unwanted files

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Converted model class to record

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    Signed-off-by: Apurva <apurva.jain@gatech.edu>
    Co-authored-by: Sahiba Mittal <sahibamittal98@gmail.com>

commit a1cd404
Author: mehab <meha.bhargava2@gmail.com>
Date:   Fri Dec 30 16:48:23 2022 +0000

    PR review

commit 57763bf
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 29 16:10:34 2022 +0000

    @transactional added for internal analyzer

commit 84d8688
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 29 15:32:31 2022 +0000

    pr review addressed

commit 80d727e
Merge: be2d62e 60fbe5c
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 29 15:20:57 2022 +0000

    Merge branch 'main' of https://github.com/mehab/DTKafkaPOC into feature/helmCharts

commit be2d62e
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 29 12:57:26 2022 +0000

    intermediate commit

commit e7f4eb4
Merge: ecb0b11 eef206d
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 29 12:21:45 2022 +0000

    merged main branch

commit ecb0b11
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 28 15:49:41 2022 +0000

    solution with 2 docker files

commit 73f517e
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 28 14:23:49 2022 +0000

    intermediate changes

commit e7a682a
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 28 12:24:51 2022 +0000

    PR review changes

commit f34900b
Author: mehab <meha.bhargava2@gmail.com>
Date:   Fri Dec 23 15:57:27 2022 +0000

    updated readme

commit 5d9f959
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 17:41:53 2022 +0000

    removed unused dependency

commit 3ca3378
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 17:30:56 2022 +0000

    final commit. Readme updated

commit 1008f1b
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 13:00:53 2022 +0000

    getting a new name for image

commit f9c4821
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 12:52:58 2022 +0000

    trying changes

commit 7450afb
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 12:24:26 2022 +0000

    getting a new name for image

commit 061e234
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 12:07:35 2022 +0000

    getting a new name for image

commit d4d9d87
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 10:52:12 2022 +0000

    getting a new name for image

commit b6d91dc
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 10:47:29 2022 +0000

    getting a new name for image

commit 2a00e74
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 10:45:40 2022 +0000

    getting a new name for image

commit 60af856
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 10:27:51 2022 +0000

    final test of image

commit 1473749
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 10:10:07 2022 +0000

    trying changes

commit e192b92
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 21 22:04:47 2022 +0000

    using kubernetes secrets

commit c61b6ce
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 21 20:59:39 2022 +0000

    testing manual environment variables

commit fdd4fad
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 21 20:55:58 2022 +0000

    testing manual environment variables

commit be8228d
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 21 20:48:57 2022 +0000

    testing manual environment variables

commit b035b6f
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 21 13:46:42 2022 +0000

    changes to check if kafka connection can also work from outside kafka

commit 9e5a47d
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 21 13:08:37 2022 +0000

    changes to check if kafka connection can also work from outside kafka

commit bed1378
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 21 12:11:31 2022 +0000

    changes to check if kafka connection can also work from outside kafka

commit 7a46a56
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 21 11:48:37 2022 +0000

    changes to check if kafka connection can also work from outside kafka

commit 8a9c7eb
Author: mehab <meha.bhargava2@gmail.com>
Date:   Tue Dec 20 22:09:18 2022 +0000

    trying workflow

commit 35ab9a9
Author: mehab <meha.bhargava2@gmail.com>
Date:   Tue Dec 20 21:58:42 2022 +0000

    trying workflow

commit 266ebb1
Author: mehab <meha.bhargava2@gmail.com>
Date:   Tue Dec 20 21:25:10 2022 +0000

    added changes for helm charts
kodiakhq bot pushed a commit that referenced this pull request Jan 6, 2023
* OSV mirror setup WIP

* add new kafka topic

* Update create-topics.sh

* OSV list of ecosystem to be consumed from event

* Test in progress

* Squashed commit of the following:

commit 3a6b8ef
Author: Niklas <nscuro@protonmail.com>
Date:   Wed Jan 4 10:44:12 2023 +0100

    Use GH Action to build and push container image in one step (#196)

    Signed-off-by: nscuro <nscuro@protonmail.com>

    Signed-off-by: nscuro <nscuro@protonmail.com>

commit 29b4d31
Author: Niklas <nscuro@protonmail.com>
Date:   Wed Jan 4 10:23:26 2023 +0100

    Fix build of container image for native executable (#195)

    Signed-off-by: nscuro <nscuro@protonmail.com>

    Signed-off-by: nscuro <nscuro@protonmail.com>

commit 68de797
Author: Niklas <nscuro@protonmail.com>
Date:   Tue Jan 3 22:30:31 2023 +0100

    Implementation of stateful retries for vulnerability analyzers (#174)

    * Rough implementation of stateful retries for OSS Index

    Implements the OSS Index analyzer as stream processor in order to have access to state stores. Because processors can only forward or drop records, they can't publish records to arbitrary channels, just the one they're connected to in the topology.

    Incoming records are assembled into batches of up to 128. Batches are analyzed when their size exceeds 128, or 5 seconds since the last batch analysis have passed.

    Records that failed with a "retryable reason" are written to a separate state store, with additional retry metadata. The retry store is checked every second for records that are due for retry. Records due for retry are re-added to the batch. The retry mechanism uses exponential backoff with jitter.

    A circuit breaker is added so that retries do not overwhelm the OSS Index API. Per the current default, if 50% out of 100 requests to OSS Index fail, the circuit is opened for 15 seconds.

    The `VulnerabilityResult` class has been extended with a `failureReason` field, that can be used to communicate *if* and *why* the analysis of a given record failed. Just a quick & dirty solution.

    Cache is still missing from this implementation.

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Move retry logic into abstract processor class

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Keep track of entry counts in retry state stores

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Remove unused processors

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Track number of entries in batch state stores for OSS Index

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Add initial test for SnykProcessor

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Completely migrate analyzers to processors

    Also move analyzer configs into centralized classes, add configuration classes for dependencies of processors, add support for multiple Snyk tokens, make usage of batch state store for OSS Index more efficient, configure HTTP client timeouts for both OSS Index and Snyk.

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Make `RetryableRecord` extend `Record`

    Makes treating of retryable records more seamless, and also retains metadata like headers.

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Unify retry config; Re-enable SnykClientTest

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Add OssIndexProcessor tests

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Migrate internal analyzer to processor

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Fix missing `@Transactional` context for internal scanner processor

    Can't use `@Transactional` here as processors are not (and can not be) CDI-managed beans.

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Update processors for recent cache and completion changes

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Disable k8s dev services

    It starts a cluster which we do not use

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Add tests for AuthorizationClientHeaderFactory

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Fix Vulnerability being marked as deprecated

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Report analyzer identity `NONE` for records with unknown identifiers

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Make sonar happy

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Fix test; Reduce retry log level to `debug`

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Port circuit breaker tests

    Also populate circuit breaker configs with default values as per https://resilience4j.readme.io/docs/circuitbreaker#create-and-configure-a-circuitbreaker and remove the, now unused, CircuitBreakerCustom class.

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Update Snyk property names in docker-compose.yml

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Fix "No serializer found for class" exceptions when running as native image

    Classes that are (de-)serialized with Jackson have to be explicitly registered for reflection, see https://quarkus.io/guides/writing-native-applications-tips#registering-for-reflection

    Also fix `ClassNotFoundException: org.apache.commons.logging.impl.LogFactoryImpl`, see quarkusio/quarkus#10128

    Signed-off-by: nscuro <nscuro@protonmail.com>

    * Fix broken merge artifacts

    Signed-off-by: nscuro <nscuro@protonmail.com>

    Signed-off-by: nscuro <nscuro@protonmail.com>

commit ce46eb9
Author: mehab <meha.bhargava2@gmail.com>
Date:   Tue Jan 3 20:17:56 2023 +0000

    Feature/trygraal (#194)

    * added changes for graalvm native image

    * resolved merge conflicts

    * resolved merge conflicts

    * separating out build jar and build native

    * separating out build jar and build native

    * renaming file to see build jar work correctly

    * renaming file to see build jar work correctly

    * renaming file to see build jar work correctly

    * changes

    * trying from main branch

    * trying from main branch

    * changed name

    * change name

    * degrading quarkus to 2.14 to see if it works

    * try with 2.15.0

    * reduced to version 2.14.2.final

    * reduced to version 2.14.2.final

    * renaming image

    * removing snapshot in caps

    * removing snapshot in caps

    * another test

    * another test

    * another test

    * another test

    * trying quarkus container build

    * trying quarkus container build

    * trying quarkus container build

    * trying with ls and copy command

    * trying with commented out docker file

    * trying with commented out docker file

    * trying docker commands directly

    * trying docker commands directly

    * trying docker commands directly

    * trying docker commands directly

    * trying docker commands directly

commit aae6b1a
Author: Niklas <nscuro@protonmail.com>
Date:   Tue Jan 3 16:59:37 2023 +0100

    Add script to generate test fixture BOMs for load testing (#189)

    Closes #162

    Signed-off-by: nscuro <nscuro@protonmail.com>

    Signed-off-by: nscuro <nscuro@protonmail.com>

commit 0d2f490
Author: mehab <meha.bhargava2@gmail.com>
Date:   Tue Jan 3 11:47:17 2023 +0000

    Feature/trygraal (#193)

    * added changes for graalvm native image

    * resolved merge conflicts

    * resolved merge conflicts

    * separating out build jar and build native

    * separating out build jar and build native

    * renaming file to see build jar work correctly

    * renaming file to see build jar work correctly

    * renaming file to see build jar work correctly

    * changes

    * trying from main branch

    * trying from main branch

    * changed name

    * change name

commit 543b6f1
Author: Niklas <nscuro@protonmail.com>
Date:   Tue Jan 3 10:59:54 2023 +0100

    Simplify handling of secret key (#188)

    Signed-off-by: nscuro <nscuro@protonmail.com>

    Signed-off-by: nscuro <nscuro@protonmail.com>

commit cdf56f1
Author: Vinod Anandan <vinod@owasp.org>
Date:   Fri Dec 30 20:08:41 2022 +0000

    Downgrade dependency io.pebbletemplates:pebble to v3.1.6 (#187)

    Downgrade dependency io.pebbletemplates:pebble to v3.1.6

commit 7e5e8d6
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Fri Dec 30 19:57:59 2022 +0000

    Update dependency io.pebbletemplates:pebble to v3.2.0 (#113)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit e721dad
Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Date:   Fri Dec 30 19:57:23 2022 +0000

    Update dependency io.quarkiverse.helm:quarkus-helm to v0.2.1 (#182)

    Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

commit 0f91f10
Author: mehab <meha.bhargava2@gmail.com>
Date:   Fri Dec 30 18:21:13 2022 +0000

    fix version for native binary (#186)

commit 2138b6c
Author: mehab <meha.bhargava2@gmail.com>
Date:   Fri Dec 30 18:18:47 2022 +0000

    graalvm native binary creation (#185)

commit 262c81b
Author: mehab <meha.bhargava2@gmail.com>
Date:   Fri Dec 30 18:14:31 2022 +0000

    graalvm native binary tryout (#184)

commit 06309d5
Author: mehab <meha.bhargava2@gmail.com>
Date:   Fri Dec 30 17:52:28 2022 +0000

    change to graalvm image generation (#183)

commit 41ac711
Merge: 0b2812e cfcedad
Author: japurva1502 <111765683+japurva1502@users.noreply.github.com>
Date:   Fri Dec 30 11:31:10 2022 -0600

    Merge pull request #177 from mehab/feature/helmCharts

    Feature/helm charts

commit cfcedad
Author: mehab <meha.bhargava2@gmail.com>
Date:   Fri Dec 30 17:17:40 2022 +0000

    Update InternalAnalyzer.java

    removing duplicate @transactional annotation

commit 78419a0
Merge: a1cd404 0b2812e
Author: mehab <meha.bhargava2@gmail.com>
Date:   Fri Dec 30 16:57:38 2022 +0000

    Merge branch 'main' into feature/helmCharts

commit 0b2812e
Author: japurva1502 <111765683+japurva1502@users.noreply.github.com>
Date:   Fri Dec 30 10:50:04 2022 -0600

    Feature/analyzer completion (#180)

    * dummy PR

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Internal Analyzer changes:
    Added persistence classes
    Added repository for querying data
    restructure internal analyzer

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Internal Analyzer changes:
    Added persistence classes
    Added repository for querying data
    restructure internal analyzer

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed schema validation for newly added persistent entities

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed schema validation for newly added persistent entities

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed schema and updated TopologyTest to include internal Analyzer

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed few enities Schema
    Added integration test for Internal Analyzer
    Unit Test for ComponentRepository

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed PK violation in test cases

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Test cases for VulnerableSoftwareRepository
    InternalAnalyzer

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Test cases

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Sonar warning

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Internal analyzer cache implementation changes

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * add analysis test for more coverage

    * Squashed commit of the following:

    commit ef6135c
    Author: Sahiba Mittal <sahibamittal98@gmail.com>
    Date:   Thu Dec 15 20:53:23 2022 +0530

        Issue 125 : Caching repo meta analyzer (#161)

        * WIP

        * cache moved to common, added for meta analyzer

        * Update RepositoryMetaAnalyzerTopology.java

        * Update RepositoryMetaAnalyzerTopology.java

        * fix code smells

        * Update MetaAnalyzerCacheKey.java

        * test in progress

        * tests fixed

        * fix recursive bug

        * test getName added

    commit af26631
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Thu Dec 15 15:23:11 2022 +0000

        Update docker.redpanda.com/vectorized/redpanda Docker tag to v22.3.8 (#166)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit 94b7040
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Wed Dec 14 22:22:34 2022 +0000

        Update dependency us.springett:alpine-common to v2.2.0 (#165)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit 88ac8e7
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Wed Dec 14 16:50:22 2022 +0000

        Update docker.redpanda.com/vectorized/redpanda Docker tag to v22.3.6 (#160)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit fbf451e
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Wed Dec 14 16:50:13 2022 +0000

        Update prom/prometheus Docker tag to v2.40.7 (#163)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit 4b60147
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Wed Dec 14 16:49:46 2022 +0000

        Update actions/setup-java action to v3.9.0 (#164)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit bf18e9e
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Fri Dec 9 20:06:10 2022 +0000

        Update resilience4j to v2.0.2 (#158)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit 5d85507
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Fri Dec 9 19:54:00 2022 +0000

        Update returntocorp/semgrep-action digest to 031b0ac (#157)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit 7116bcd
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Fri Dec 9 19:53:33 2022 +0000

        Update prom/prometheus Docker tag to v2.40.6 (#159)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit 3eab7f9
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Thu Dec 8 11:24:14 2022 +0000

        Update actions/setup-java action to v3.8.0 (#154)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    commit a67ff17
    Author: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
    Date:   Thu Dec 8 11:23:58 2022 +0000

        Update returntocorp/semgrep-action digest to 027b0fd (#155)

        Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>

    * Create SeverityConverterTest.java

    * Test coverage

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Refactored repo classes

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * add component version test

    * PR review fixes

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Internal Analyzer changes for CPE and Purl

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Returning VulnrabilityResult with null vulnerability when no vulns are found

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Wip : Analyzer completion

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed errors in ser/deser and JPA exception because of missing @transactional annotation

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed broken test cases

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed Typo

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed internalAnalyzer property

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Test coverage

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Test coverage

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Setting Vuln description as blank

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed broken test case

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed code smell

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Fixed code smell

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * corrected internal cache name

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * removed unwanted files

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    * Converted model class to record

    Signed-off-by: Apurva <apurva.jain@gatech.edu>

    Signed-off-by: Apurva <apurva.jain@gatech.edu>
    Co-authored-by: Sahiba Mittal <sahibamittal98@gmail.com>

commit a1cd404
Author: mehab <meha.bhargava2@gmail.com>
Date:   Fri Dec 30 16:48:23 2022 +0000

    PR review

commit 57763bf
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 29 16:10:34 2022 +0000

    @transactional added for internal analyzer

commit 84d8688
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 29 15:32:31 2022 +0000

    pr review addressed

commit 80d727e
Merge: be2d62e 60fbe5c
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 29 15:20:57 2022 +0000

    Merge branch 'main' of https://github.com/mehab/DTKafkaPOC into feature/helmCharts

commit be2d62e
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 29 12:57:26 2022 +0000

    intermediate commit

commit e7f4eb4
Merge: ecb0b11 eef206d
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 29 12:21:45 2022 +0000

    merged main branch

commit ecb0b11
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 28 15:49:41 2022 +0000

    solution with 2 docker files

commit 73f517e
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 28 14:23:49 2022 +0000

    intermediate changes

commit e7a682a
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 28 12:24:51 2022 +0000

    PR review changes

commit f34900b
Author: mehab <meha.bhargava2@gmail.com>
Date:   Fri Dec 23 15:57:27 2022 +0000

    updated readme

commit 5d9f959
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 17:41:53 2022 +0000

    removed unused dependency

commit 3ca3378
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 17:30:56 2022 +0000

    final commit. Readme updated

commit 1008f1b
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 13:00:53 2022 +0000

    getting a new name for image

commit f9c4821
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 12:52:58 2022 +0000

    trying changes

commit 7450afb
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 12:24:26 2022 +0000

    getting a new name for image

commit 061e234
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 12:07:35 2022 +0000

    getting a new name for image

commit d4d9d87
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 10:52:12 2022 +0000

    getting a new name for image

commit b6d91dc
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 10:47:29 2022 +0000

    getting a new name for image

commit 2a00e74
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 10:45:40 2022 +0000

    getting a new name for image

commit 60af856
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 10:27:51 2022 +0000

    final test of image

commit 1473749
Author: mehab <meha.bhargava2@gmail.com>
Date:   Thu Dec 22 10:10:07 2022 +0000

    trying changes

commit e192b92
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 21 22:04:47 2022 +0000

    using kubernetes secrets

commit c61b6ce
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 21 20:59:39 2022 +0000

    testing manual environment variables

commit fdd4fad
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 21 20:55:58 2022 +0000

    testing manual environment variables

commit be8228d
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 21 20:48:57 2022 +0000

    testing manual environment variables

commit b035b6f
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 21 13:46:42 2022 +0000

    changes to check if kafka connection can also work from outside kafka

commit 9e5a47d
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 21 13:08:37 2022 +0000

    changes to check if kafka connection can also work from outside kafka

commit bed1378
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 21 12:11:31 2022 +0000

    changes to check if kafka connection can also work from outside kafka

commit 7a46a56
Author: mehab <meha.bhargava2@gmail.com>
Date:   Wed Dec 21 11:48:37 2022 +0000

    changes to check if kafka connection can also work from outside kafka

commit 8a9c7eb
Author: mehab <meha.bhargava2@gmail.com>
Date:   Tue Dec 20 22:09:18 2022 +0000

    trying workflow

commit 35ab9a9
Author: mehab <meha.bhargava2@gmail.com>
Date:   Tue Dec 20 21:58:42 2022 +0000

    trying workflow

commit 266ebb1
Author: mehab <meha.bhargava2@gmail.com>
Date:   Tue Dec 20 21:25:10 2022 +0000

    added changes for helm charts

* test added for topology

* ecosystem as key

* fix file stream methods

* osv client test

* osv parser tests

* Update pom.xml

* osv analyzer test

* server port fix

* Create FileUtilTest.java

* Update OsvAnalyzerTest.java

* Update OsvAnalyzerTest.java

* fix sonar bug

* review comments addressed

* Update OsvClientTest.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
defect Something isn't working domain/vuln-analysis
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants