Framework for adding compatibility tests #6129
Conversation
This PR includes a basic framework for us to build compatibility tests across releases The actual tests and a lot of code is yet to be written. The framekwork is set up so that we can run operations during a cluster upgrade, and detect incompatibility across releases. The value of the tests will be in the operations we choose to test during upgrades Configuration is done via YAML, and some sample yaml files and skinny classes have been provided.
Codecov Report
@@ Coverage Diff @@
## master #6129 +/- ##
==========================================
+ Coverage 66.44% 72.86% +6.41%
==========================================
Files 1075 1223 +148
Lines 54773 57744 +2971
Branches 8168 8522 +354
==========================================
+ Hits 36396 42075 +5679
+ Misses 15700 12912 -2788
- Partials 2677 2757 +80
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
compatibility-verifier/compCheck.sh
Outdated
| sleep 20 | ||
| #$COMPAT_TESTER pre-controller-upgrade.yaml |
There was a problem hiding this comment.
I'm not sure why 20 seconds sleep was chosen in the first place, but does replacing it with running compat_tester mean the tests are going to take 20 seconds?
Also maybe it reads better if $COMPAT_TESTER is replaced with a function call, something like runCompatiblityTests, which is more consistent with startService, startService, ... here.
There was a problem hiding this comment.
Removed the sleep. It should not be needed.
Replacing with a function call may be fine, but we still have to check the return from that function. I have added the status check now. Let it be like this for now, until we add more meat to it
compatibility-verifier/compCheck.sh
Outdated
| #$COMPAT_TESTER post-controller-rollback.yaml | ||
| sleep 20 |
There was a problem hiding this comment.
If you decide to have(or remove) previous sleeps, please do the same for this one as well?
| boolean runOp() { | ||
| switch(_op) { | ||
| case UPLOAD: | ||
| System.out.println("Generating segment " + _segmentName + " from " + _inputDataFileName + " and uploading to " + |
There was a problem hiding this comment.
We may not need to print it out in the console?
There was a problem hiding this comment.
We will not need it. I put it there so we can see what is being run. Once we add more logic here, we can remove these things
|
we should remove all sleeps. They were added by the earlier committer. I will do that. |
mcvsubbu
changed the title
4 participants
This PR includes a basic framework for us to build compatibility tests across releases
The actual tests and a lot of code is yet to be written.
The framework is set up so that we can run operations during a cluster upgrade, and
detect incompatibility across releases. The value of the tests will be in the operations
we choose to test during upgrades
Configuration is done via YAML, and some sample yaml files and skinny classes have been
provided.
Issue 4854
Description
Add a description of your PR here.
A good description should include pointers to an issue or design document, etc.
Upgrade Notes
Does this PR prevent a zero down-time upgrade? (Assume upgrade order: Controller, Broker, Server, Minion)
backward-incompat, and complete the section below on Release Notes)Does this PR fix a zero-downtime upgrade introduced earlier?
backward-incompat, and complete the section below on Release Notes)Does this PR otherwise need attention when creating release notes? Things to consider:
release-notesand complete the section on Release Notes)Release Notes
If you have tagged this as either backward-incompat or release-notes,
you MUST add text here that you would like to see appear in release notes of the
next release.
If you have a series of commits adding or enabling a feature, then
add this section only in final commit that marks the feature completed.
Refer to earlier release notes to see examples of text
Documentation
If you have introduced a new feature or configuration, please add it to the documentation as well.
See https://docs.pinot.apache.org/developers/developers-and-contributors/update-document