[watcher] Multiple NullPointerException in watchers #40595
Labels
bug
Fixes for quality problems that affect the customer experience
Feature:Watcher
Team:Kibana Management
Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more
Comments
|
It's not Here's the complete execution context: |
Hermholtz
changed the title
kindsun
added
bug
|
I am having the same issue on cloud elasticsearch v6.7.1 Got two advanced alerts with almost identical jsons - one is fine, another one is having these NPE's. JSON: |
|
Don't worry, they've assigned "BUG" label 5 months ago, it's already a known bug. |
cjcenizal
added
the
Team:Kibana Management
|
Pinging @elastic/es-ui (Team:Elasticsearch UI) |
|
it would have been nice to resolve soon. |
|
Yeah, a year has passed already. |
|
Hey, everyone, thank you for raising (and re-raising!) this issue after all this time has elapsed. This looks like an issue in the underlying Watcher or Painless implementation in Elasticsearch. ES engineers are investigating it now. Thanks again for your patience and persistence. |
|
@Hermholtz apologies for the delay in investigation. @stu-elastic and I were trying to repro this issue locally, but some more information from your end would be helpful if you still have the same setup. Would you please run |
|
Sorry I don't work on it anymore, don't even have this installed... @skmizuho can you do that? |
|
here we go Expand to see Debug.explain(params) "exception": {
"type": "script_exception",
"reason": "runtime error",
"to_string": "{threshold=75}",
"java_class": "java.util.Collections$UnmodifiableMap",
"script_stack": [
"Debug.explain(params); ",
" ^---- HERE"
],
"script": "Debug.explain(params); return false;",
"lang": "painless",
"position": {
"offset": 14,
"start": 0,
"end": 23
},
"caused_by": {
"type": "painless_explain_error",
"reason": null,
"stack_trace": "org.elasticsearch.painless.PainlessExplainError\n\tat org.elasticsearch.painless.api.Debug.explain(Debug.java:23)\n\tat org.elasticsearch.painless.PainlessScript$Script.execute(Debug.explain(params); return false;:15)\n\tat org.elasticsearch.xpack.watcher.condition.ScriptCondition.doExecute(ScriptCondition.java:61)\n\tat org.elasticsearch.xpack.watcher.condition.ScriptCondition.execute(ScriptCondition.java:56)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:513)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:320)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.lambda$executeAsync$5(ExecutionService.java:421)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:627)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:673)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:832)\n"
},
"stack_trace": "ScriptException[runtime error]; nested: PainlessExplainError;\n\tat org.elasticsearch.painless.PainlessScript.convertToScriptException(PainlessScript.java:85)\n\tat org.elasticsearch.painless.PainlessScript$Script.execute(Debug.explain(params); return false;:1)\n\tat org.elasticsearch.xpack.watcher.condition.ScriptCondition.doExecute(ScriptCondition.java:61)\n\tat org.elasticsearch.xpack.watcher.condition.ScriptCondition.execute(ScriptCondition.java:56)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:513)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:320)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.lambda$executeAsync$5(ExecutionService.java:421)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:627)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:673)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:832)\nCaused by: org.elasticsearch.painless.PainlessExplainError\n\tat org.elasticsearch.painless.api.Debug.explain(Debug.java:23)\n\tat org.elasticsearch.painless.PainlessScript$Script.execute(Debug.explain(params); return false;:15)\n\t... 10 more\n"
}and Expand to see Debug.explain(ctx.payload.aggregations.metricAgg) "exception": {
"type": "script_exception",
"reason": "runtime error",
"painless_class": "java.util.HashMap",
"to_string": "{value=null}",
"java_class": "java.util.HashMap",
"script_stack": [
"Debug.explain(ctx.payload.aggregations.metricAgg); ",
" ^---- HERE"
],
"script": "Debug.explain(ctx.payload.aggregations.metricAgg); return false;",
"lang": "painless",
"position": {
"offset": 38,
"start": 0,
"end": 51
},
"caused_by": {
"type": "painless_explain_error",
"reason": null,
"stack_trace": "org.elasticsearch.painless.PainlessExplainError\n\tat org.elasticsearch.painless.api.Debug.explain(Debug.java:23)\n\tat org.elasticsearch.painless.PainlessScript$Script.execute(Debug.explain(ctx.payload.aggregations.metricAgg); return false;:39)\n\tat org.elasticsearch.xpack.watcher.condition.ScriptCondition.doExecute(ScriptCondition.java:61)\n\tat org.elasticsearch.xpack.watcher.condition.ScriptCondition.execute(ScriptCondition.java:56)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:513)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:320)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.lambda$executeAsync$5(ExecutionService.java:421)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:627)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:673)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:832)\n"
},
"stack_trace": "ScriptException[runtime error]; nested: PainlessExplainError;\n\tat org.elasticsearch.painless.PainlessScript.convertToScriptException(PainlessScript.java:85)\n\tat org.elasticsearch.painless.PainlessScript$Script.execute(Debug.explain(ctx.payload.aggregations.metricAgg); return false;:1)\n\tat org.elasticsearch.xpack.watcher.condition.ScriptCondition.doExecute(ScriptCondition.java:61)\n\tat org.elasticsearch.xpack.watcher.condition.ScriptCondition.execute(ScriptCondition.java:56)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.executeInner(ExecutionService.java:513)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.execute(ExecutionService.java:320)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService.lambda$executeAsync$5(ExecutionService.java:421)\n\tat org.elasticsearch.xpack.watcher.execution.ExecutionService$WatchExecutionTask.run(ExecutionService.java:627)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:673)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)\n\tat java.base/java.lang.Thread.run(Thread.java:832)\nCaused by: org.elasticsearch.painless.PainlessExplainError\n\tat org.elasticsearch.painless.api.Debug.explain(Debug.java:23)\n\tat org.elasticsearch.painless.PainlessScript$Script.execute(Debug.explain(ctx.payload.aggregations.metricAgg); return false;:39)\n\t... 10 more\n"
}and watcher configuration is Expand to see watcher configurationPUT _watcher/watch/9c94d33b-4d5e-42cf-9426-355998520387
{
"trigger": {
"schedule": {
"interval": "1m"
}
},
"input": {
"search": {
"request": {
"body": {
"size": 0,
"query": {
"bool": {
"filter": {
"range": {
"timestamp": {
"gte": "{{ctx.trigger.scheduled_time}}||-65d",
"lte": "{{ctx.trigger.scheduled_time}}",
"format": "strict_date_optional_time||epoch_millis"
}
}
}
}
},
"aggs": {
"metricAgg": {
"max": {
"field": "anomaly_score"
}
}
}
},
"indices": [
".ml-anomalies-shared"
]
}
}
},
"condition": {
"script": {
"source": "if (ctx.payload.aggregations.metricAgg.value > params.threshold) { return true; } return false;",
"params": {
"threshold": 75
}
}
},
"transform": {
"script": {
"source": "HashMap result = new HashMap(); result.result = ctx.payload.aggregations.metricAgg.value; return result;",
"params": {
"threshold": 75
}
}
},
"actions": {
"email_1": {
"email": {
"profile": "standard",
"to": [
"aja@aja.ja"
],
"subject": "Watch [{{ctx.metadata.name}}] has exceeded the threshold",
"body": {
"text": "A critical anomaly with score {{ctx.anomaly_score}} usually indicates high response times.\n"
}
}
}
}
} |
|
this issue has appeared during the elastic o11y workshop btw :D cc @arthurgimpel and @petericebear |
|
Any update on this? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Kibana version: 7.2.0
Elasticsearch version: 7.2.0
Server OS version: CentOS 7.5
Browser version: any
Browser OS version: any
Original install method (e.g. download page, yum, from source, etc.): yum
Describe the bug:
Simple watcher yields NullPointerException errors instead of succeeding. Excerpt from the log:
Steps to reproduce:
Expected behavior:
Message logged
Screenshots (if relevant):
Execution output:
Provide logs and/or server output (if relevant):
Any additional context:
I'm going to have a presentation on Friday for an important client about alerting and ML, and just few days before it this disappointing surprise happens...
The text was updated successfully, but these errors were encountered: