Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

523 bug fix - Sending notification under more error conditions #667

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Angie-Zhang
Copy link
Contributor

@Angie-Zhang Angie-Zhang commented Jan 18, 2023

Signed-off-by: Angie Zhang langelzh@amazon.com

Issue #, if available:

#523

Description of changes:

  • Add methods to send notifications when managedIndexMetaData is not available.
  • Update runManagedIndexConfig logic.

CheckList:

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Angie Zhang <langelzh@amazon.com>
@codecov-commenter
Copy link

Codecov Report

Merging #667 (af2f5de) into main (76d530f) will decrease coverage by 0.04%.
The diff coverage is 42.16%.

@@             Coverage Diff              @@
##               main     #667      +/-   ##
============================================
- Coverage     75.56%   75.53%   -0.04%     
- Complexity     2657     2663       +6     
============================================
  Files           333      333              
  Lines         15395    15453      +58     
  Branches       2384     2395      +11     
============================================
+ Hits          11633    11672      +39     
- Misses         2410     2414       +4     
- Partials       1352     1367      +15     
Impacted Files Coverage Δ
...ent/indexstatemanagement/util/NotificationUtils.kt 0.00% <0.00%> (ø)
...agement/indexstatemanagement/ManagedIndexRunner.kt 46.95% <43.75%> (-0.19%) ⬇️
...nt/indexstatemanagement/model/destination/Chime.kt 55.55% <0.00%> (-22.23%) ⬇️
...anagement/indexstatemanagement/model/Transition.kt 61.64% <0.00%> (-4.11%) ⬇️
...arch/indexmanagement/rollup/RollupSearchService.kt 61.11% <0.00%> (-3.71%) ⬇️
...gement/indexstatemanagement/action/ShrinkAction.kt 69.11% <0.00%> (-2.95%) ⬇️
...management/indexstatemanagement/MetadataService.kt 62.68% <0.00%> (-2.24%) ⬇️
...nt/indexstatemanagement/ManagedIndexCoordinator.kt 69.67% <0.00%> (-0.59%) ⬇️
...earch/indexmanagement/transform/model/Transform.kt 85.59% <0.00%> (-0.43%) ⬇️
... and 5 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@@ -527,6 +560,7 @@ object ManagedIndexRunner :

@Suppress("TooGenericExceptionCaught")
private suspend fun disableManagedIndexConfig(managedIndexConfig: ManagedIndexConfig) {
logger.info("Start to disable ManagedIndexConfig")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably don't need this.

Comment on lines +879 to +890
private fun compileTemplate(template: Script, managedIndexConfig: ManagedIndexConfig): String {
return try {
scriptService.compile(template, TemplateScript.CONTEXT)
.newInstance(template.params + mapOf("ctx" to managedIndexConfig.convertToMap()))
.execute()
} catch (e: Exception) {
val message = "There was an error compiling mustache template"
logger.error(message, e)
e.message ?: message
}
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactor this. Just one compileTemplate here is enough, the only difference is "ctx", let's explicitly choose fields to put into context

Comment on lines +864 to +877
private suspend fun publishErrorNotification(policy: Policy?, managedIndexConfig: ManagedIndexConfig, additionalMessage: String) {
if (policy == null) {
logger.error("Policy for index ${managedIndexConfig.index} is not available to obtain. Notifications for this policy is aborted.")
return
}
policy.errorNotification?.run {
errorNotificationRetryPolicy.retry(logger) {
val compiledMessage = compileTemplate(messageTemplate, managedIndexConfig)
val finalMessage = "$additionalMessage $compiledMessage"
destination?.buildLegacyBaseMessage(null, finalMessage)?.publishLegacyNotification(client)
channel?.sendNotification(client, ErrorNotification.CHANNEL_TITLE, managedIndexConfig, compiledMessage, policy.user)
}
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we combine this with above same name method? additionalMessage method can be nullable

For managedIndexMetaData, managedIndexConfig, we can just pick out index uuid, index name and input here. You can check if any other fields are useful to compose the notification message

Comment on lines +273 to +277
// doing a check of local cluster health as we do not want to overload cluster manager node with potentially a lot of calls
if (clusterIsRed()) {
val message = "Skipping current execution of ${managedIndexConfig.index} because of red cluster health."
logger.debug(message)
publishErrorNotification(policy, managedIndexConfig, message)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Red cluster check has to be at top, because it may affect all the following step: read metadata, write... Any reason to move it here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants