GCP Scanner Development and Testing

Introduction

Below we describe how GCP Scanner implements functional and unit testing using GCP infrastructure.

We rely on Github Actions to run unit and functional testing and use a self-hosting to actually test GCP Scanner. The workflow implementation can be found here. We deploy GCP Scanner on a self-hosted GCP VM and launch pylint with pylintrc to check that submitted code follow Google Python Code Style Guideline and pytest to actually execute all tests.

There are two Python modules from where tests are launched: test_acceptance.py and test_unit.py.

Acceptance Testing

The self-hosted GCP VM is executed within a test GCP project with various GCP APIs and resources available for querying. We launch GCP Scanner with -m option which is used to query GCP VM credentials from instance metadata. Then, the scanner uses obtained credentials to actually scan all resources available within GCP project. Once the scanning is complete, we compare obtained and expected results by counting number of fetched resources for each individual entry.

Unit Testing

In contrast with acceptance testing, we actually compare scan data returned by GCP Scanner with what we expect. We use test_unit.py to verify scan results and additionally test credentials fetching functionality. For crawler testing, we prepared a set of expected results that are used by the test_unit.py to validate GCP Scanner functionality.

However, this approach has its own flaws due to GCP scan results volatility. The GCP is constantly evolving and some results might change. Additionally, some fields are volatile by design and should be ignored when we compare results. In order to address this problem, we have special token VOLATILE that tells test_unit.py to ignore certain lines during comparison.