Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce integration test for restart #718

Merged
merged 15 commits into from
Oct 9, 2019
Merged

Conversation

alenkacz
Copy link
Contributor

@alenkacz alenkacz commented Aug 14, 2019

This introduces and integration test that:

  • creates instance
  • waits for deploy on that instance is finished
  • stops controller
  • updates instance CRD in the meantime
  • starts controller
  • verifies update plan was run

This test was failing prior to the instance controller refactoring (the old controller logic was handling this wrong)

@kudo-ci
Copy link

kudo-ci commented Aug 14, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alenkacz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kudo-ci
Copy link

kudo-ci commented Aug 15, 2019

@alenkacz: The following test failed, say /retest to rerun them all:

Test name Commit Details Rerun command
test 407e311 link /test test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

alenkacz added a commit that referenced this pull request Aug 28, 2019
…rs (#722)

Based on the test I wrote in #718 it became apparent where lies the race that is one of the causes of multiple deploy plan executions.

- instance is created, create event triggers reconciliation
- planexecution is created
- OV is created and triggers reconciliation
- another planexecution is created because ActivePlan is still empty
- PlanExecutionController processes added planexecutions and create link between one of them and instance leaving two deploy planexecutions in progress, only one linked to instance, the other just hanging around forever

This PR proposes to move adding the ActivePlan relation to InstanceController. To make that possible I needed to add some safeguards to the PlanExecutionController as well.
@kudo-ci
Copy link

kudo-ci commented Sep 4, 2019

@alenkacz: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@gerred
Copy link
Member

gerred commented Oct 8, 2019

looks reasonable, what do we need to do to land this?

@alenkacz alenkacz requested a review from zen-dog as a code owner October 8, 2019 18:10
@alenkacz alenkacz changed the title WIP: Introduce integration test for restart Introduce integration test for restart Oct 8, 2019
@alenkacz
Copy link
Contributor Author

alenkacz commented Oct 8, 2019

@gerred now you can :)

Copy link
Contributor

@zen-dog zen-dog left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice test! 🚢

mgrStopped.Wait()

log.Print("And update the instance parameter value")
err := c.Get(context.TODO(), instanceKey, in)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this paranoid programming or do we really expect the instance to disappear? 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haha this is actually necessary because the instance changed in the meantime (controller updated some fields, so we need to pull it again). I'll add comment :)

@alenkacz alenkacz merged commit 2c15fe3 into master Oct 9, 2019
@alenkacz alenkacz deleted the av/test-update-after-restart branch October 9, 2019 18:56
jbarrick-mesosphere pushed a commit that referenced this pull request Oct 17, 2019
* Introduce integration test for restart (#718)

This introduces and integration test that:
- creates instance
- waits for deploy on that instance is finished
- stops controller
- updates instance CRD in the meantime
- starts controller
- verifies update plan was run

This test was failing prior to the instance controller refactoring (the old controller logic was handling this wrong)

* Fix namespace handling for plan command (#925)

* Include CRDs in the -o yaml --dry-run (#923)

* go mod changes makes repo dirty.  this resolves that (#926)

* Fix deep copy and don't modify original state (#919)

* Fixes the release workflow for Docker images

* Attempt to fix 'TestCheckResourceIntegration' flaky test (#914)

* Do not use strategicmergepatch for CRDs (#936)

* Plan status should show last executed plan, not just active plan (#931)

* More consistent logging for install (#933)

* fixing repo merge by name bug (#938)

* Use built in isExists (#942)

* Make it possible to opt out of junit (#950)

* Maintainers is array, not object (#930)

* Handle already exists in flaky test (#941)

* Cleanup lastexecutedplan method to remove the flake (#949)

* Fix docs for instance name in install (#943)

* add namespace flag for kudo init (#903)

* Fix 'run_tests' script

* Use a shell script for E2E test commands

The script is similar to the one used for integration tests. This also
allows to not use JUnit output.

* Copy E2E test script into test container

* Fix permissions of E2E test script

* Distinguish between local and operator E2E tests

Tests in the 'test/e2e' folder are run as part of the E2E tests in addition
to the operator tests.
jbarrick-mesosphere added a commit that referenced this pull request Oct 17, 2019
fix make

docker privs

add go-junit-report

add docker CLI to test dockerfile

fix docker cli install

fix docker run command

fix docker

fix networking

remove CMD from dockerfile

E2E test additions (#958)

* Introduce integration test for restart (#718)

This introduces and integration test that:
- creates instance
- waits for deploy on that instance is finished
- stops controller
- updates instance CRD in the meantime
- starts controller
- verifies update plan was run

This test was failing prior to the instance controller refactoring (the old controller logic was handling this wrong)

* Fix namespace handling for plan command (#925)

* Include CRDs in the -o yaml --dry-run (#923)

* go mod changes makes repo dirty.  this resolves that (#926)

* Fix deep copy and don't modify original state (#919)

* Fixes the release workflow for Docker images

* Attempt to fix 'TestCheckResourceIntegration' flaky test (#914)

* Do not use strategicmergepatch for CRDs (#936)

* Plan status should show last executed plan, not just active plan (#931)

* More consistent logging for install (#933)

* fixing repo merge by name bug (#938)

* Use built in isExists (#942)

* Make it possible to opt out of junit (#950)

* Maintainers is array, not object (#930)

* Handle already exists in flaky test (#941)

* Cleanup lastexecutedplan method to remove the flake (#949)

* Fix docs for instance name in install (#943)

* add namespace flag for kudo init (#903)

* Fix 'run_tests' script

* Use a shell script for E2E test commands

The script is similar to the one used for integration tests. This also
allows to not use JUnit output.

* Copy E2E test script into test container

* Fix permissions of E2E test script

* Distinguish between local and operator E2E tests

Tests in the 'test/e2e' folder are run as part of the E2E tests in addition
to the operator tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants