Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Integration Test Pipeline #120

Merged
merged 4 commits into from
Jan 11, 2024
Merged

ci: Integration Test Pipeline #120

merged 4 commits into from
Jan 11, 2024

Conversation

c-ryan-k
Copy link
Member

@c-ryan-k c-ryan-k commented Dec 28, 2023

Adds pipelines for cluster integration tests and resource cleanup.
Due to the lengthy cleanup and deployment process, the cleanup pipeline is:

  • Optionally called at the end of int_test.yml (with a boolean input)
  • Able to be triggered by manual dispatch
  • Scheduled to run nightly

Pipeline stages:

  • int_test.yml (can be called by external workflows)

    • Install python and checkout extension source
    • Create k3s cluster on agent
    • Build, install extension
    • ARC connect cluster
    • AIO Deployment
    • Run 'az iot ops check' and write output to step summary
    • Run smoke test commands and upload any support bundles that are created
  • cluster_cleanup.yml

    • Targets a resource group (and optional cluster/keyvault prefix)
    • Deletes the following resources based on the provided prefixes (or all if no prefix is provided):
      • Connected Clusters
      • Key Vaults
      • MQ instance and child resources
      • Custom locations and deployed AIO hidden resources

Note - also removes helm install / update commands from k3d.sh


This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Thank you for contributing to Azure IoT Operations tooling!

This checklist is used to make sure that common guidelines for a pull request are followed.

General Guidelines

Intent for Production

  • It is expected that pull requests made to default or core branches such as dev or main are of production grade. Corollary to this, any merged contributions to these branches may be deployed in a public release at any given time. By checking this box, you agree and commit to the expected production quality of code.

Basic expectations

  • If introducing new functionality or modified behavior, are they backed by unit and/or integration tests?
  • In the same context as above are command names and their parameter definitions accurate? Do help docs have sufficient content?
  • Have all the relevant unit and integration tests pass? i.e. pytest <project root> -vv. Please provide evidence in the form of a screenshot showing a succesful run of tests locally OR a link to a test pipeline that has been run against the change-set.
  • Have linter checks passed using the .pylintrc and .flake8 rules? Look at the CI scripts for example usage.
  • Have extraneous print or debug statements, commented out code-blocks or code-statements (if any) been removed from the surface area of changes?
  • Have you made an entry in HISTORY.rst which concisely explains your user-facing feature or change?

Azure IoT Operations CLI maintainers reserve the right to enforce any of the outlined expectations.

A PR is considered ready for review when all basic expectations have been met (or do not apply).

Adds pipelines for cluster integration tests and resource cleanup

Removes helm install / update commands from k3d.sh
@c-ryan-k c-ryan-k changed the title test: Integration Test Pipeline ci[test]: Integration Test Pipeline Dec 28, 2023
@c-ryan-k c-ryan-k changed the title ci[test]: Integration Test Pipeline ci: Integration Test Pipeline Dec 28, 2023
K3D -> K3S

Ubuntu-latest -> Ubuntu-22.04

Added smoke tests and support bundle artifacts
@c-ryan-k c-ryan-k marked this pull request as ready for review January 10, 2024 22:34
# install e4k, e4i, opcua
helm install e4k oci://e4kpreview.azurecr.io/helm/az-e4k --version 0.6.0 --set global.quickstart=true --namespace azure-iot-operations
helm upgrade -i e4i oci://e4ipreview.azurecr.io/helm/az-e4i --version 0.5.1 --namespace e4i-runtime --create-namespace --set mqttBroker.authenticationMethod="serviceAccountToken" --set mqttBroker.name="azedge-dmqtt-frontend" --set mqttBroker.namespace="default" --set opcPlcSimulation.deploy=true --wait
helm upgrade -i opcua oci://e4ipreview.azurecr.io/helm/az-e4i-opcua-connector --version 0.5.1 --namespace opcua --create-namespace --set payloadCompression="none" --set opcUaConnector.settings.discoveryUrl="opc.tcp://opcplc.e4i-runtime:50000" --set opcUaConnector.settings.autoAcceptUntrustedCertificates=true --set mqttBroker.name="azedge-dmqtt-frontend" --set mqttBroker.namespace="default" --set mqttBroker.authenticationMethod="serviceAccountToken" --wait
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need this script since we are using k3s?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, both the devcontainer post_create_commands and now the testing pipelines use separate methods to setup the clusters, so we probably don't.

Removed k3d.sh

Removed smoke_test pipeline and references

Add continue-on-error param to tox dispatch workflow

Update repository workflow documentation
@c-ryan-k c-ryan-k merged commit f338a88 into Azure:dev Jan 11, 2024
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants