Skip to content

Commit

Permalink
[e2e] Add support for platform=none
Browse files Browse the repository at this point in the history
- Add platform=none as new cloud provider in WMCO e2e test suite
- Add a check in testWindowsNodeCreation test to avoid the creation of
  Windows machines via machineSet, and just wait for the Windows instance(s)
  to become Windows node(s).
- Introduce a parametrized entry point in Makefile to invoke the e2e-test
  suite from CI, just for the BYOH use case.
- Add a new step in run-ci-e2e-test.sh script to create the windows-instances
  configMap once the operator deployment process is complete.
- Add section in HACKING.md to describe the process of running the e2e tests
  on a platform-agnostic infrastructure.
  • Loading branch information
jrvaldes committed Jan 7, 2022
1 parent 4379b3b commit 29a59ce
Show file tree
Hide file tree
Showing 6 changed files with 105 additions and 0 deletions.
6 changes: 6 additions & 0 deletions Makefile
Expand Up @@ -183,6 +183,12 @@ run-ci-e2e-test:
run-ci-e2e-upgrade-test:
hack/run-ci-e2e-test.sh -t upgrade

# TODO: Consolidate run-ci-e2e-* targets into a single target with arguments.
# For example: run-ci-e2e-test $(ARGS). Requires updates in release steps.
.PHONY: run-ci-e2e-byoh-test $(ARGS)
run-ci-e2e-byoh-test:
hack/run-ci-e2e-test.sh ${ARGS}

.PHONY: clean
clean:
rm -rf ${OUTPUT_DIR}
Expand Down
34 changes: 34 additions & 0 deletions docs/HACKING.md
Expand Up @@ -117,6 +117,40 @@ hack/run-ci-e2e-test.sh -s -m 2 -c 1
```
Please note that you do not need to run `hack/olm.sh run` before `hack/run-ci-e2e-test.sh`.

### Running e2e tests on platform-agnostic infrastructure

To run the WMCO' e2e tests on a bare metal or other platform-agnostic infrastructure (platform=none), where there
is no cloud provider specification, the desired number of Windows instances must be available beforehand, and you
need provide the instance(s) information to the e2e test suite using an environment variable.

To deploy machines in any of the [infrastructure providers supported by WMCO](wmco-prerequisites.md#supported-cloud-providers-based-on-okdocp-version-and-wmco-version),
refer to the specific provider documentation. See Windows instance [pre-requisites](https://github.com/openshift/windows-machine-config-operator/blob/master/docs/byoh-instance-pre-requisites.md).

The information you need to collect from each Windows instance is:
- the internal IP address, and
- the Windows administrator username

Export `WINDOWS_INSTANCES_DATA` as an environment variable with the corresponding [windows-instances configMap](https://github.com/openshift/windows-machine-config-operator#adding-instances)
data section, and a new `windows-instances` configMap will be created during the execution of the e2e test suite.

For example:
```shell
export WINDOWS_INSTANCES_DATA='
data:
10.1.42.1: |-
username=Administrator
'
```
where `10.1.42.1` is the IP address and `Administrator` is the Windows' administrator username of the Windows instance.

Once the `WINDOWS_INSTANCES_DATA` environment variable is declared and exported you can run:
```shell
hack/run-ci-e2e-test.sh -m 0 -c 1
```
where, `-m 0` indicates no Windows instance will be configured by the Windows Machine controller and `-c 1`
accounts for the number of instances that will be configured by the ConfigMap controller, and must match the
number of entries in the `WINDOWS_INSTANCES_DATA` environment variable, in this example only one (1).

## Bundling the Windows Machine Config Operator
This directory contains resources related to installing the WMCO onto a cluster using OLM.

Expand Down
19 changes: 19 additions & 0 deletions hack/run-ci-e2e-test.sh
Expand Up @@ -11,6 +11,9 @@ BYOH_NODE_COUNT_OPTION=""
SKIP_NODE_DELETION=""
WMCO_PATH_OPTION=""

# WINDOWS_INSTANCES_DATA holds the windows-instances configMap data section
declare -r WINDOWS_INSTANCES_DATA=${WINDOWS_INSTANCES_DATA:-}

export CGO_ENABLED=0

get_WMCO_logs() {
Expand Down Expand Up @@ -101,6 +104,22 @@ if ! [[ "$OPENSHIFT_CI" = "true" && "$TEST" = "upgrade" ]]; then
done
fi

# Check windows-instances data and create the windows-instances configMap, if
# any. The configMap creation must takes place after the operator's deployment
# process is complete, because the namespace is removed as part of the cleanup
# function (cleanup_WMCO) while retrying the deployment.
if [[ -n "$WINDOWS_INSTANCES_DATA" ]]; then
# create configMap
cat <<EOF | oc apply -f -
kind: ConfigMap
apiVersion: v1
metadata:
name: windows-instances
namespace: ${WMCO_DEPLOY_NAMESPACE}
${WINDOWS_INSTANCES_DATA}
EOF
fi

# The bool flags in golang does not respect key value pattern. They follow -flag=x pattern.
# -flag x is allowed for non-boolean flags only(https://golang.org/pkg/flag/)

Expand Down
12 changes: 12 additions & 0 deletions test/e2e/create_test.go
Expand Up @@ -58,6 +58,18 @@ func testWindowsNodeCreation(t *testing.T) {
// Create a private key secret with the known private key.
require.NoError(t, testCtx.createPrivateKeySecret(true), "could not create known private key secret")

// in case platform=none is detected just wait for Windows worker node to become available
if testCtx.CloudProvider.GetType() == config.NonePlatformType {
// TODO: setPowerShellDefaultShell for given Windows instance(s)
// TODO: validate windows-instances configMap
t.Run("[platform=none] Windows instance configured by ConfigMap controller", func(t *testing.T) {
err = testCtx.waitForWindowsNodes(gc.numberOfBYOHNodes, false, false, true)
assert.NoError(t, err, "Windows node creation failed")
})
// Expected number of Windows instances configured as worker nodes, return
return
}

t.Run("Machine controller", testCtx.testMachineConfiguration)
// BYOH creation must occur after the Machine creation, as the MachineConfiguration tests change the private key
// multiple times, and BYOH doesnt have the functionality of rotating keys on the VMs. This would result in BYOH
Expand Down
3 changes: 3 additions & 0 deletions test/e2e/providers/cloudprovider.go
Expand Up @@ -10,6 +10,7 @@ import (
oc "github.com/openshift/windows-machine-config-operator/test/e2e/clusterinfo"
awsProvider "github.com/openshift/windows-machine-config-operator/test/e2e/providers/aws"
azureProvider "github.com/openshift/windows-machine-config-operator/test/e2e/providers/azure"
noneProvider "github.com/openshift/windows-machine-config-operator/test/e2e/providers/none"
vSphereProvider "github.com/openshift/windows-machine-config-operator/test/e2e/providers/vsphere"
)

Expand Down Expand Up @@ -37,6 +38,8 @@ func NewCloudProvider(hasCustomVXLANPort bool) (CloudProvider, error) {
return azureProvider.New(openshift, hasCustomVXLANPort)
case config.VSpherePlatformType:
return vSphereProvider.New(openshift)
case config.NonePlatformType:
return noneProvider.New(openshift)
default:
return nil, fmt.Errorf("the '%v' cloud provider is not supported", provider)
}
Expand Down
31 changes: 31 additions & 0 deletions test/e2e/providers/none/none.go
@@ -0,0 +1,31 @@
package none

import (
"github.com/pkg/errors"

config "github.com/openshift/api/config/v1"
mapi "github.com/openshift/api/machine/v1beta1"
"github.com/openshift/windows-machine-config-operator/test/e2e/clusterinfo"
)

// Provider is a provider struct for testing platform=none
type Provider struct {
oc *clusterinfo.OpenShift
}

// New returns a Provider implementation for platform=none
func New(clientset *clusterinfo.OpenShift) (*Provider, error) {
return &Provider{
oc: clientset,
}, nil
}

// GenerateMachineSet is not supported for platform=none and throws an exception
func (p *Provider) GenerateMachineSet(withWindowsLabel bool, replicas int32) (*mapi.MachineSet, error) {
return nil, errors.New("MachineSet generation not supported for platform=none")
}

// GetType returns the platform type for platform=none
func (p *Provider) GetType() config.PlatformType {
return config.NonePlatformType
}

0 comments on commit 29a59ce

Please sign in to comment.