Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PoC] Adding contract testing #1438

Closed
wants to merge 9 commits into from

Conversation

josvazg
Copy link
Collaborator

@josvazg josvazg commented Mar 12, 2024


% export AKO_CONTRACT_TEST=1; go clean -testcache && go test -v ./test/contract/...
?   	github.com/mongodb/mongodb-atlas-kubernetes/v2/test/contract	[no test files]
2024/03/19 17:30:25 Wipe resources set to false
2024/03/19 17:30:25 Reusing existing resources
=== RUN   TestCreateSearchIndex
--- PASS: TestCreateSearchIndex (0.60s)
PASS
2024/03/19 17:30:26 Trying to stored search test resource references for reuse...
2024/03/19 17:30:26 Stored search test resource references for reuse
ok  	github.com/mongodb/mongodb-atlas-kubernetes/v2/test/contract/search	1.509s
2024/03/19 17:30:25 Wipe resources set to false
2024/03/19 17:30:26 Reusing existing resources
=== RUN   TestUser
2024/03/19 17:30:26 Creating user user...
2024/03/19 17:30:26 User user already created, updating password
--- PASS: TestUser (11.30s)
PASS
2024/03/19 17:30:37 Trying to stored user test resource references for reuse...
2024/03/19 17:30:37 Stored user test resource references for reuse
ok  	github.com/mongodb/mongodb-atlas-kubernetes/v2/test/contract/user	12.395s

All Submissions:

  • Have you signed our CLA?

@josvazg josvazg marked this pull request as draft March 12, 2024 18:20
@josvazg josvazg force-pushed the CLOUDP-221979/search-idx-contract-tests branch 4 times, most recently from 662a08a to b5d9d41 Compare March 13, 2024 12:13
test/contract/db.go Outdated Show resolved Hide resolved
@josvazg josvazg force-pushed the CLOUDP-221979/search-idx-contract-tests branch from 8b04f84 to 99a7b82 Compare March 19, 2024 19:14
@josvazg
Copy link
Collaborator Author

josvazg commented Mar 21, 2024

Feedback after discussing with @helderjs

  1. Not only we should NOT use the API calls directly on the tests, which is pending to be fixed in current PR but also...
  2. Try to reuse the production functions instead of direct calls ALSO for setting up the test infra.

@josvazg josvazg changed the title Adding contract testing [PoC] Adding contract testing Mar 22, 2024
@josvazg josvazg force-pushed the CLOUDP-221979/search-idx-contract-tests branch 2 times, most recently from 5163f08 to decf362 Compare April 1, 2024 14:24
Signed-off-by: jose.vazquez <jose.vazquez@mongodb.com>
@josvazg josvazg force-pushed the CLOUDP-221979/search-idx-contract-tests branch from decf362 to c9456e8 Compare April 1, 2024 14:38
@josvazg
Copy link
Collaborator Author

josvazg commented Apr 2, 2024

Lessons around contract testing:

  • For us it requires quite some infrastructure automation to make sure the initial state is as expected. Usually a project and deployment need to be in place for most tests.
  • For local dev it is useful to keep the infrastructure around to re-test faster. For CI it needs to be cleaned up always at the end.
  • Ad-hoc infra automation is a waste, we probably want to use the Operator itself for this.
  • Calling the API directly is pointless: we must test the production code that calls the API instead. This needs our production code to separate such code and probably include full translation of types as well in such layer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants