Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add schema migration init container [K8SSAND-625] #57

Conversation

adejanovski
Copy link
Contributor

This PR requires Reaper PR 1096 to be merged and released first.

Fixes #56

This change adds an init container that will start Reaper in schema migration only mode, allowing for it to pass without probes that could trigger a restart mid-migration.

@@ -490,6 +493,13 @@ func newDeployment(reaper *api.Reaper, cassDcService string) *appsv1.Deployment
}
}

initContainerEnvVar := make([]corev1.EnvVar, len(envVars))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be len(envVars) + 1 ? Instead of letting append later on to grow the size again.

@adejanovski adejanovski force-pushed the add-schema-migration-init-container branch 3 times, most recently from ca43162 to 5c02e60 Compare June 29, 2021 13:19
@jdonenine jdonenine changed the title Add schema migration init container Add schema migration init container [K8SSAND-625] Jun 29, 2021
@adejanovski adejanovski force-pushed the add-schema-migration-init-container branch 4 times, most recently from 429a30e to 0b229bb Compare July 2, 2021 15:43
@adejanovski adejanovski requested a review from burmanm July 2, 2021 15:43
@adejanovski
Copy link
Contributor Author

e2e tests are fixed and things work as expected.
We'll have to wait until a new Docker tag is released for Reaper before merging the PR, but it would be good to get an approval ASAP so that we can get this into 1.3.

@@ -3,7 +3,7 @@ kind: Reaper
metadata:
name: reaper-test
spec:
image: docker.io/thelastpickle/cassandra-reaper:2.2.5
image: docker.io/adejanovski/cassandra-reaper:413af30
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably be in k8ssandra repo.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because of Reaper and Medusa being used outside of k8ssandra I believe the decision was made to keep them
in the TLP org. On that basis I don't see a need to publish the image under k8ssandra.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But that's not in the TLP org either.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't mean to commit the PR like this, as said in my comment :)

We'll have to wait until a new Docker tag is released for Reaper before merging the PR, but it would be good to get an approval ASAP so that we can get this into 1.3.

I have mixed feelings about merging the PR on a tag that's not an actual release, but we'll probably have to wait a little bit more until we get a new release of Reaper. I may just create a tag with a name that's easy enough to spot in the tlp org and update this PR so it can be merged.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've moved to a tag on the tlp repo now. I'll try to remember to upgrade this when Reaper 2.3.0 gets released.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming Reaper 2.3.0 will be released before k8ssandra 1.3.0 then we can open a follow up PR to bump the default version. I am in favor of merging and updating in k8ssandra to give the changes some soak time before releasing 1.3.0.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The trick here is that I don't think we're going to get reaper-2.3.0 out before k8ssandra-1.3.0. So if we want to include this in k8ssandra-1.3.0 we'll need to use some patch version of reaper that we could get out sooner.

@adejanovski what do you think about releasing a v2.2.6 of Reaper that would pickup your work in thelastpickle/cassandra-reaper#1096 and use that?

We could hopefully then get reaper-2.3.0 out post k8ssandra-1.3.0 and use that in k8ssandra-1.4.0 to pickup the incremental repair support?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adejanovski what do you think about releasing a v2.2.6 of Reaper that would pickup your work in thelastpickle/cassandra-reaper#1096 and use that?

It's doable but slightly breaks the contract that patch releases are for fixes only.

Since we have no solid ETA for incremental repair improvements, I guess we could release 2.3.0 right now and delay other planned features to 2.4.0.
wdyt @jdonenine ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That works for me @adejanovski

I'd vote we go ahead and release 2.3.0 and we'll come back with 2.4.0 for incremental repair when it's ready.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the record, so you don't think I've gone crazy, I was thinking a patch release because what we're generally trying to fix is a bug: #56

But you're totally right that what we added to reaper itself, to address the bug over here in K8ssandra, isn't a bug fix. So I think you've got the right suggestion with bump to 2.4.0.

@adejanovski adejanovski force-pushed the add-schema-migration-init-container branch from 0b229bb to 8338822 Compare July 5, 2021 13:01
@adejanovski adejanovski requested a review from burmanm July 5, 2021 13:17
@adejanovski adejanovski force-pushed the add-schema-migration-init-container branch from 8338822 to fda2d7a Compare July 9, 2021 08:15
@sonarcloud
Copy link

sonarcloud bot commented Jul 9, 2021

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@adejanovski adejanovski merged commit f404733 into k8ssandra:master Jul 9, 2021
@adejanovski adejanovski deleted the add-schema-migration-init-container branch July 9, 2021 08:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Schema should be fully created prior to starting the Reaper pod
4 participants