Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🥳 node-termination-handler v1.18.0 Automated Release! 🥑 #850

Merged
merged 1 commit into from
Nov 22, 2022

Conversation

ec2-bot
Copy link
Contributor

@ec2-bot ec2-bot commented Nov 22, 2022

node-termination-handler v1.18.0 Automated Chart Sync! 🤖🤖

Release Notes 📝:

Improved logging in Queue Processor mode

v1.18.0 introduces the logFormatVersion Helm chart option, to allow you to opt-in to more detailed logs.

The default value is 1, which keeps logging the same way it did in prior releases (<= v1.17.3).

Setting the value to 2 will give you more detail about which AWS event triggered the cordon/drain. Previously, all these events were bucketed under SQS_TERMINATE and it was difficult to tell what was happening.

This option is also available as a command line flag, --log-format-version

What does the new logging look like?

logFormatVersion=2 modifies several Debug, Info, and Warn logs, as well as Kubernetes events emitted by NTH. These changes improve your observability about what NTH is doing when responding to events via SQS. If your monitoring system is configured to look for any of the specific strings in the tables below, you may need to modify your configuration to use the updated strings if you use the new log format version.

Changes to logs when starting up

  1. Remove event_type field from the Info log when starting a monitor; replace with monitor_type field, with new values. See Table 1.
  2. Remove event_type field from the Warn log when a monitor fails to start; replace with monitor_type field, with new values. See Table 1.

Changes to logs when processing an event

  1. New monitor field in the Info log. See Table 1.
  2. Potentially change value of kind field in the Info log, if running Queue Processor mode. See Table 2.
  3. Potentially change the "reason" field in the k8s event if running Queue Processor mode. See Table 3.

Changes to logs when receiving an SQS message

  1. Include the specific event type instead of SQS_TERMINATE in the Debug log if running Queue Processor mode. See Table 2.

Tables of changed values

Table 1: Monitor types
Previous New
REBALANCE_RECOMMENDATION REBALANCE_RECOMMENDATION_MONITOR
SCHEDULED_EVENT SCHEDULED_EVENT_MONITOR
SPOT_ITN SPOT_ITN_MONITOR
SQS_TERMINATE SQS_MONITOR
Table 2: Event types
Previous New
REBALANCE_RECOMMENDATION REBALANCE_RECOMMENDATION
SCHEDULED_EVENT SCHEDULED_EVENT
SPOT_ITN SPOT_ITN
SQS_TERMINATE REBALANCE_RECOMMENDATION SCHEDULED_EVENT SPOT_ITN STATE_CHANGE ASG_LIFECYCLE
Table 3: Event reasons
Previous reason New reason
RebalanceRecommendation RebalanceRecommendation
ScheduledEvent ScheduledEvent
SpotInterruption SpotInterruption
SQSTermination RebalanceRecommendation ScheduledEvent SpotInterruption StateChange ASGLifecycle

Commits with these changes

Other changes

Full Changelog: aws/aws-node-termination-handler@v1.17.3...v1.18.0

@snay2 snay2 merged commit 91b1607 into aws:master Nov 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants