fix(resource-service): fix nats subscription and added retry logic #7215

warber · 2022-03-21T10:08:14Z

closes #6817

added the retry logic from the POC
deleted nats related code in package nats which was copied from shippyard controller
added new package pkg/nats/subscriber that contains logic to subscribe to nats events

Integration tests

https://github.com/keptn/keptn/actions/runs/2032483837
https://github.com/keptn/keptn/actions/runs/2032553974

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

…ling Signed-off-by: warber <bernd.warmuth@dynatrace.com>

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

…urce service Signed-off-by: warber <bernd.warmuth@dynatrace.com>

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

resource-service/pkg/nats/subscriber/sub.go

github-actions · 2022-03-21T10:14:11Z

resource-service/pkg/nats/subscriber/sub.go

+	"errors"
+	"fmt"
+	"github.com/keptn/keptn/resource-service/models"
+	"github.com/nats-io/nats.go"


[golangci-lint] _{reported by reviewdog 🐶}
"github.com/nats-io/nats.go" imported but not used (typecheck)

codecov · 2022-03-21T10:22:33Z

Codecov Report

Merging #7215 (020f0f7) into master (a6c0877) will decrease coverage by 0.02%.
The diff coverage is 73.38%.

@@            Coverage Diff             @@
##           master    #7215      +/-   ##
==========================================
- Coverage   58.61%   58.58%   -0.03%     
==========================================
  Files         530      528       -2     
  Lines       30856    30784      -72     
  Branches     1508     1508              
==========================================
- Hits        18086    18035      -51     
+ Misses      11498    11485      -13     
+ Partials     1272     1264       -8

Impacted Files	Coverage Δ
resource-service/pkg/nats/subscriber/sub.go	`65.11% <65.11%> (ø)`
resource-service/handler/service_manager.go	`87.03% <75.86%> (-12.97%)`	⬇️
resource-service/handler/resource_manager.go	`94.24% <78.94%> (-1.55%)`	⬇️
resource-service/common/git.go	`55.42% <100.00%> (ø)`
resource-service/handler/nats/eventhandler.go	`100.00% <100.00%> (ø)`
shipyard-controller/handler/sequencedispatcher.go	`71.54% <0.00%> (-1.63%)`	⬇️

Flag	Coverage Δ
api	`71.61% <ø> (ø)`
approval-service	`71.92% <ø> (ø)`
bridge-server	`66.66% <ø> (ø)`
bridge2	`73.48% <ø> (ø)`
cli	`43.83% <ø> (ø)`
configuration-service	`8.38% <ø> (ø)`
distributor	`70.80% <ø> (ø)`
helm-service	`49.23% <ø> (ø)`
jmeter-service	`13.88% <ø> (ø)`
lighthouse-service	`73.35% <ø> (ø)`
mongodb-datastore	`59.86% <ø> (ø)`
remediation-service	`79.77% <ø> (ø)`
resource-service	`80.70% <73.38%> (+0.37%)`	⬆️
secret-service	`81.23% <ø> (ø)`
shipyard-controller	`58.03% <ø> (-0.04%)`	⬇️
statistics-service	`59.96% <ø> (ø)`
webhook-service	`80.41% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

…ling Signed-off-by: warber <bernd.warmuth@dynatrace.com>

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

…urce service Signed-off-by: warber <bernd.warmuth@dynatrace.com>

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

… into feat/6817/retry-delete-resource

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>

sonarcloud · 2022-03-24T07:09:06Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

…7215) * introduces retry logic from POC Signed-off-by: warber <bernd.warmuth@dynatrace.com> * created test Signed-off-by: warber <bernd.warmuth@dynatrace.com> * added entry to int test matrix, implemented correct nats message handling Signed-off-by: warber <bernd.warmuth@dynatrace.com> * fixed compile error and disabled other integration tests for now Signed-off-by: warber <bernd.warmuth@dynatrace.com> * removed comments in integration_tests.yml Signed-off-by: warber <bernd.warmuth@dynatrace.com> * introduce KEPTN_NAMESPACE_SUFFIX Signed-off-by: warber <bernd.warmuth@dynatrace.com> * trigger build Signed-off-by: warber <bernd.warmuth@dynatrace.com> * changed cloud provider to just GKE Signed-off-by: warber <bernd.warmuth@dynatrace.com> * revert to master integration_tests.yml with multiple replicas of resource service Signed-off-by: warber <bernd.warmuth@dynatrace.com> * cleanup Signed-off-by: warber <bernd.warmuth@dynatrace.com> * enable DIRECTORY_STAGE_STRUCTURE in integration tests Signed-off-by: warber <bernd.warmuth@dynatrace.com> * trigger build Signed-off-by: warber <bernd.warmuth@dynatrace.com> * added retry logic in service manager Signed-off-by: warber <bernd.warmuth@dynatrace.com> * try to fix retry logic Signed-off-by: warber <bernd.warmuth@dynatrace.com> * added hard reset before each pull Signed-off-by: warber <bernd.warmuth@dynatrace.com> * moved git reset out of retry Signed-off-by: warber <bernd.warmuth@dynatrace.com> * introduces retry logic from POC Signed-off-by: warber <bernd.warmuth@dynatrace.com> * created test Signed-off-by: warber <bernd.warmuth@dynatrace.com> * added entry to int test matrix, implemented correct nats message handling Signed-off-by: warber <bernd.warmuth@dynatrace.com> * fixed compile error and disabled other integration tests for now Signed-off-by: warber <bernd.warmuth@dynatrace.com> * removed comments in integration_tests.yml Signed-off-by: warber <bernd.warmuth@dynatrace.com> * introduce KEPTN_NAMESPACE_SUFFIX Signed-off-by: warber <bernd.warmuth@dynatrace.com> * changed cloud provider to just GKE Signed-off-by: warber <bernd.warmuth@dynatrace.com> * revert to master integration_tests.yml with multiple replicas of resource service Signed-off-by: warber <bernd.warmuth@dynatrace.com> * cleanup Signed-off-by: warber <bernd.warmuth@dynatrace.com> * enable DIRECTORY_STAGE_STRUCTURE in integration tests Signed-off-by: warber <bernd.warmuth@dynatrace.com> * added retry logic in service manager Signed-off-by: warber <bernd.warmuth@dynatrace.com> * try to fix retry logic Signed-off-by: warber <bernd.warmuth@dynatrace.com> * added hard reset before each pull Signed-off-by: warber <bernd.warmuth@dynatrace.com> * moved git reset out of retry Signed-off-by: warber <bernd.warmuth@dynatrace.com> * reverted moving git reset out of retry Signed-off-by: warber <bernd.warmuth@dynatrace.com> * reverted unintentional commit in distributor module Signed-off-by: warber <bernd.warmuth@dynatrace.com> * magic Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com> * set number of replicas back to 1 Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com> * turn off directory structure Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com> * removed git hard resets Signed-off-by: warber <bernd.warmuth@dynatrace.com> * fix minor issues in integration tests Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com> * fixed backup restore test Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com> Co-authored-by: odubajDT <ondrej.dubaj@dynatrace.com> Signed-off-by: Florian Bacher <florian.bacher@dynatrace.com>

warber added 15 commits March 7, 2022 15:54

introduces retry logic from POC

0e23d34

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

created test

0593026

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

merge master

ab0d528

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

added entry to int test matrix, implemented correct nats message hand…

ec3ae91

…ling Signed-off-by: warber <bernd.warmuth@dynatrace.com>

fixed compile error and disabled other integration tests for now

437a226

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

merge master

8494aed

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

removed comments in integration_tests.yml

a2f8a2a

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

introduce KEPTN_NAMESPACE_SUFFIX

eb24e74

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

trigger build

ad39ce4

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

changed cloud provider to just GKE

5eddb32

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

revert to master integration_tests.yml with multiple replicas of reso…

9e8a290

…urce service Signed-off-by: warber <bernd.warmuth@dynatrace.com>

cleanup

56520ba

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

enable DIRECTORY_STAGE_STRUCTURE in integration tests

4143228

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

trigger build

827ada5

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

added retry logic in service manager

6b01fb3

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

warber added the CI:trigger-build-everything Trigger CI Build: Set BUILD_EVERYTHING=TRUE label Mar 21, 2022

warber requested review from mowies, christian-kreuzberger-dtx and a team as code owners March 21, 2022 10:08

github-actions bot reviewed Mar 21, 2022

View reviewed changes

warber added 9 commits March 21, 2022 11:33

merge master

d7ba7cb

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

try to fix retry logic

cce36fe

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

added hard reset before each pull

8857bef

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

merge master

8adadbc

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

moved git reset out of retry

7e1fb04

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

merge master

b85a85a

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

introduces retry logic from POC

cea4d03

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

created test

603ced8

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

added entry to int test matrix, implemented correct nats message hand…

e10d2e3

…ling Signed-off-by: warber <bernd.warmuth@dynatrace.com>

warber added 10 commits March 23, 2022 07:22

removed comments in integration_tests.yml

e854db6

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

introduce KEPTN_NAMESPACE_SUFFIX

f4c85ff

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

changed cloud provider to just GKE

0feb8f6

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

revert to master integration_tests.yml with multiple replicas of reso…

f59ef75

…urce service Signed-off-by: warber <bernd.warmuth@dynatrace.com>

cleanup

2d8565f

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

enable DIRECTORY_STAGE_STRUCTURE in integration tests

64459df

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

added retry logic in service manager

449decd

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

try to fix retry logic

399b00c

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

added hard reset before each pull

c2f5fd7

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

moved git reset out of retry

dcc28a6

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

odubajDT force-pushed the feat/6817/retry-delete-resource branch from b85a85a to dcc28a6 Compare March 23, 2022 07:49

warber and others added 4 commits March 23, 2022 09:35

Merge remote-tracking branch 'origin/feat/6817/retry-delete-resource'…

75fb431

… into feat/6817/retry-delete-resource

reverted moving git reset out of retry

5618ae3

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

reverted unintentional commit in distributor module

34aa783

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

magic

1b99bfc

Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>

RealAnna mentioned this pull request Mar 23, 2022

test: fixing integration tests #7207

Merged

odubajDT and others added 4 commits March 23, 2022 14:47

set number of replicas back to 1

0a995f2

Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>

Merge branch 'master' into feat/6817/retry-delete-resource

d8e3af8

turn off directory structure

d289018

Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>

removed git hard resets

01e6420

Signed-off-by: warber <bernd.warmuth@dynatrace.com>

odubajDT force-pushed the feat/6817/retry-delete-resource branch from 8d5fd3f to 01e6420 Compare March 23, 2022 18:07

odubajDT added 2 commits March 23, 2022 20:44

fix minor issues in integration tests

05a2f78

Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>

fixed backup restore test

020f0f7

Signed-off-by: odubajDT <ondrej.dubaj@dynatrace.com>

odubajDT approved these changes Mar 24, 2022

View reviewed changes

RealAnna changed the title ~~feat: run resource service with multiple replicas~~ feat: run resource service in both modes and fix nats Mar 24, 2022

warber changed the title ~~feat: run resource service in both modes and fix nats~~ fix(resource-service): fix nats subscription and added retry logic Mar 24, 2022

odubajDT merged commit 180d833 into master Mar 24, 2022

mowies deleted the feat/6817/retry-delete-resource branch May 4, 2022 06:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(resource-service): fix nats subscription and added retry logic #7215

fix(resource-service): fix nats subscription and added retry logic #7215

warber commented Mar 21, 2022 •

edited

github-actions bot Mar 21, 2022

codecov bot commented Mar 21, 2022 •

edited

sonarcloud bot commented Mar 24, 2022

fix(resource-service): fix nats subscription and added retry logic #7215

fix(resource-service): fix nats subscription and added retry logic #7215

Conversation

warber commented Mar 21, 2022 • edited

Integration tests

github-actions bot Mar 21, 2022

Choose a reason for hiding this comment

codecov bot commented Mar 21, 2022 • edited

Codecov Report

sonarcloud bot commented Mar 24, 2022

warber commented Mar 21, 2022 •

edited

codecov bot commented Mar 21, 2022 •

edited