Fix scan issues discovered during e2e testing #57
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ghost
force-pushed
the
fix_private_network
branch
from
997a66d
to
6e7e47a
Compare
FrimIdan
reviewed
Feb 2, 2023
runtime_scan/pkg/orchestrator/configwatcher/scanconfig_watcher.go
Outdated
Show resolved
Hide resolved
runtime_scan/pkg/orchestrator/configwatcher/scanconfig_watcher.go
Outdated
Show resolved
Hide resolved
runtime_scan/pkg/orchestrator/configwatcher/scanconfig_watcher.go
Outdated
Show resolved
Hide resolved
fishkerez
reviewed
Feb 2, 2023
runtime_scan/pkg/orchestrator/configwatcher/scanconfig_watcher.go
Outdated
Show resolved
Hide resolved
ghost
force-pushed
the
fix_private_network
branch
2 times, most recently
from
b115d3f
to
6c6511f
Compare
FrimIdan
previously approved these changes
Feb 5, 2023
Adds a configuration option LOCAL_DB_PATH for the VMClarity backend to specify the path to store the sqlite DB. It then updates the AWS install configuration so that the DB gets persisted in a mounted volume so that the DB isn't lost between reboots of the VMClarity service.
Public subnets require an elastic IP address to talk to the internet even if we don't require inbound connections. This commit adds a private subnet for the Scanner VMs with a NAT gateway to provide internet connectivity and a Scanner specific SecurityGroup. The SecurityGroup allows us to explictly allow only API communication to the VMClarity backend as well as SSH access from the VMClarity backend to the scanner for debugging.
As we added the name tag to the scanner instances we had to allow the Name tag key in the policy. For Scanner debug we must also allow instances to be created with a key-pair from our account. We also need to allow CopySnapshot actions in the policy for snapshots in other regions that must be moved to the VMClarity region.
This ensures that when we include the rendered scanner configuration yaml in the cloud-init template, it is indented correctly so that it doesn't break the yaml formatting of the rest of the template. This commit adds the sprig library to enhance the functions available to the golang templating library, then uses the indent function from that to fix the indentation of the yaml.
This commit adds some info and debug level logs into the Orchestrator so that it is easier to see when its starts to evaluate the ScanConfigs and the ScanConfigs that it decides need to start scanning.
A snapshot that is copied between regions takes longer to reach ready state than the snapshots which are created in the same region. This commit moves the timeout control for this actions to the job manager instead of the provider client so that we can have different timeouts for different behaviours.
ghost
force-pushed
the
fix_private_network
branch
from
a453501
to
0bbc9f9
Compare
akpsgit
reviewed
Feb 7, 2023
| Type: "AWS::EC2::RouteTable" | ||
| Properties: | ||
| VpcId: !Ref VmClarity | ||
| # Create route rule the pushes all non-local traffic to the NAT gateway for |
akpsgit
approved these changes
Feb 7, 2023
This pull request was closed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes several issues discovered during the E2E testing, see the individual commit messages for more details. With these changes building and installing the latest code, scanning works E2E after creating a single shot scan config.