Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NTHv2 core functionality #612

Merged
merged 13 commits into from
Apr 14, 2022
Merged

NTHv2 core functionality #612

merged 13 commits into from
Apr 14, 2022

Conversation

cjerad
Copy link
Contributor

@cjerad cjerad commented Apr 4, 2022

Issue #, if available:

#556 Initial NTH v2 design proposal

Description of changes:

  • Add SQS parameters to Terminator spec
  • Add drain parameters to Terminator spec
  • Fetch messages from SQS queue
  • Cordon and drain nodes targeted in SQS messages
  • Add test suite

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

* Add SQS parameters to Terminator spec
* Add drain parameters to Terminator spec
* Fetch messages from SQS queue
* Cordon and drain nodes targeted in SQS messages
@cjerad cjerad requested a review from a team as a code owner April 4, 2022 16:29
src/api/v1alpha1/terminator_logging.go Outdated Show resolved Hide resolved
src/cmd/controller/main.go Outdated Show resolved Hide resolved
src/cmd/controller/main.go Show resolved Hide resolved
src/cmd/controller/main.go Outdated Show resolved Hide resolved
src/pkg/event/asgterminate/v1/awsevent.go Outdated Show resolved Hide resolved
src/pkg/event/scheduledchange/v1/awsevent.go Outdated Show resolved Hide resolved
src/test/reconciliation_test.go Show resolved Hide resolved
Copy link
Contributor

@brycahta brycahta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like the preemptive versioning in the event pkg (v0, v1, v2) is unnecessary and makes the project structure more confusing. My understanding is if we didn't include these versions now, then when/IF we ever release a v2 then we would need to spend additional time to update module versions (potentially logic). Assuming this was the intent, then I don't think the cost is worth the potential savings.

I think it'd be clearer to remove versions altogether. Ex: pkg/event/rebalancerecommendation/event.go

Lmk if this assumption is wrong or if I'm missing context

kubectlcordondrainer.DefaultCordoner,
kubectlcordondrainer.DefaultDrainer,
eventParser := event.NewAggregatedParser(
asgterminateeventv1.Parser{ASGLifecycleActionCompleter: asgClient},
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do the versions (v0, v1, v2) correspond to? Is it necessary to have both asgterminateeventv1 and asgterminateeventv2?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It corresponds to the value of the "version" field from the EventBridge message (stored in AWSMetadata.Version). Each event type has a versioned schema defined in EventBridge, e.g. here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh gotcha, I feel like I'm getting lost in the versions here: CRD/API version, app version, module/pkg version, and now EventBridge schema version.

With that said, I'm still leaning towards removing the version (v0, v1) from the project directories/file paths. I don't think the schema version needs to be exposed in the project, but instead handled within an event's Parse logic. For example with asgterminate event, Parse would try to unmarshal into v1 struct first.. if that fails try v2.

  • Another idea (related to my other comment) is a "super" struct AsgTerminate that is a superset of all fields, unmarshal into that, and then check field(s) for version.

Last concern with breaking out by schema version is if v3 is released tomorrow with very minimal changes, then we need to create a new directory and set of files that would be mostly copy/paste which I don't think is necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if v3 is released tomorrow with very minimal changes [emphasis mine]

It is also possible that new versions would include new and important data fields, or require new API calls.

I think the question is: what should NTH do if it encounters a newer version number of a known event type?

  1. Ignore it
    • This is the currently implemented behavior; an error is logged
  2. Handle it the same as the previous version
    • Should probably log a warning about the unknown version in case there is a failure later

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the question is: what should NTH do if it encounters a newer version number of a known event type?

  1. Ignore it
    This is the currently implemented behavior; an error is logged
  2. Handle it the same as the previous version
    Should probably log a warning about the unknown version in case there is a failure later

I'd go with number 2 and make it a best effort to handle it (or maybe configurable?). However, I think regardless of how we implement unknown version handling we don't need to expose the version at the package level.

src/pkg/terminator/adapter/sqsclient.go Outdated Show resolved Hide resolved
src/pkg/event/asgterminate/v2/awsevent.go Outdated Show resolved Hide resolved
src/pkg/event/rebalancerecommendation/v0/awsevent.go Outdated Show resolved Hide resolved
@cjerad cjerad requested review from snay2 and bwagner5 April 8, 2022 12:27
Copy link
Contributor

@bwagner5 bwagner5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
V2
Awaiting triage
Development

Successfully merging this pull request may close these issues.

None yet

4 participants