Skip to content

Conversation

@jstudler
Copy link
Collaborator

Goal: Make the e2e itempotent so we don't have to manually clean up NetBox to make the tests work.

The idea is to patch .spec.preserveInNetbox of the claim, then assert that .spec.preserveInNetbox of the non-Claim is not true before exiting the test (and cleaning up the k8s resources). This is currently not possible for unknown reasons. Observations/Theories:

  • Observation: The tests fail non-deterministically and in NetBox there are leftovers.
  • Observation: The number of failed tests and the number of leftovers in NetBox does not correlate. e.g. it's possible all tests are successful but there are leftovers.
  • There is something weird with booleans, json serialisation and validation in Golang/CRDs: required boolean field will be invalid if the value was false. go-playground/validator#319
  • Theory: The cached client of chainsaw and NetBox Operator are out of sync, chainsaw observes the .spec.preserveInNetbox=false and cleans up the resources while NetBox Operator still has the old state with .spec.preserveInNetbox=true and thus does not clean up things in NetBox.
  • Theory: There is an issue in the Claim Controller updating the non-Claim spec in time

Another approach to solve this would be to run a script in the cleanup block of the tests that cleans up resources directly in NetBox e.g. everything that contains the "e2e/" prefix in the description. The advantage of this would be that we don't misuse the chainsaw test itself for cleanup.

@jstudler
Copy link
Collaborator Author

jstudler commented Jun 19, 2025

Implemented in #321

@jstudler jstudler closed this Jun 19, 2025
@jstudler jstudler deleted the feature/make-e2e-tests-idempotent branch July 15, 2025 06:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants