The container hawkular-metrics is crashing frequently #331
Comments
|
Could you please provide the logs for the |
|
Please find log details: |
|
Hi Juraci,
Pasted log on the page "
#331"
Thanks,
Rahul
…On Thu, Apr 13, 2017 at 2:36 AM, Juraci Paixão Kröhling < ***@***.***> wrote:
Could you please provide the logs for the hawkular-metrics-* pods?
Something like oc logs hawkular-metrics-yf6me should already be helpful.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#331 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AS2u-Ae60XCCaroIPsfXFtgrC-X8Wgktks5rvcJ5gaJpZM4M73_W>
.
|
|
It seems the reason Hawkular is failing to start is this: This file originally comes from the secret Could you please confirm that the entry If you prefer, join the IRC channel |
|
The problem is most likely that they are trying to deploy metrics using tagged with one version and using ansible meant for a different version. What version of ansible are you using? And what is the tagged metric images you are using? |
|
Ansible: Tagged metrics image was Thanks, |
|
Sorry, I think I have made a few assumptions here.
You cannot just replace the image versions and expect things to continue to work. This is always going to cause issues. Its not just about the image itself. The pod definition's commands may be different between versions and the secrets or configmaps may have also changed. |
|
Hi jpkrohling, I am unable to find your id at IRC #openshift channel. Thanks, |
|
Hi Jurasi,
Please let me know once you are online.
I ran `oc get rc hawkular-metrics -o json -n openshift-infra ` command and
it has 3 commands:
/opt/hawkular/scripts/hawkular-metrics-wrapper.sh
/opt/hawkular/scripts/hawkular-metrics-liveness.py
/opt/hawkular/scripts/hawkular-metrics-readiness.py
Thanks,
Rahul Agarwal
…On Thu, Apr 13, 2017 at 8:32 AM, Juraci Paixão Kröhling < ***@***.***> wrote:
It seems the reason Hawkular is failing to start is this:
Caused by: java.io.FileNotFoundException: hawkular-jgroups.keystore (No such file or directory)
This file originally comes from the secret hawkular-metrics-secrets:
https://github.com/openshift/origin-metrics/blob/v1.4.1/
deployer/scripts/hawkular.sh#L94
Could you please confirm that the entry hawkular-metrics.jgroups.keystore
exists on the secret hawkular-metrics-secrets? Please, *DON'T* paste the
contents of the secret here :)
If you prefer, join the IRC channel #hawkular and I can try to help you
there.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#331 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AS2u-MkITIokuBrU8f7K1o6aH-r_YggLks5rvhXsgaJpZM4M73_W>
.
|
|
I'm closing this issue, as it's it appears to me that this is not a bug in the code. Please, re-read the comments on this issue, but basically, you'd need to bring your services in sync with the image: compare the service definitions that you have in your environment with the definitions on this git repository (hint: https://github.com/openshift/origin-metrics/tree/master/deployer/templates), and either use new service definitions with new images, or use older images with the older service definitions. It's hard to say exactly what's going on and we would benefit from having better debugging info from your side, but based on information you provided, it looks like you have a newer service definition with an older image: the container image requires If you don't know how to do that, I suggest you tear down your environment and start from scratch. Make sure to not use Should you require further assistance, please join the IRC channel As a final "hint", compare the Newer service definitions have this under the If you are using Fedora 25 (or a recent enough version of |
|
@rahul334481 , please retry after increasing memory of the node which is running the metrics pod. I encountered similar problem earlier and increasing memory solved that. How much memory your node has? |
|
Hi Vikas,
Thanks for the suggestion. Node had 24GB memory and I upgraded to 32GB but
no success.
Thanks,
Rahul Agarwal
…On Fri, Apr 21, 2017 at 10:45 PM, Vikas Choudhary ***@***.***> wrote:
@rahul334481 <https://github.com/rahul334481> , please retry after
increasing memory of the node which is running the metrics pod. I
encountered similar problem earlier and increasing memory solved that. How
much memory your node has?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#331 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AS2u-NNiIlX2mftS4mIv83K6ZEn6Hv5Tks5ryWnggaJpZM4M73_W>
.
|
|
|
Please, use the mailing list or IRC for support. GitHub issues are for issues around Origin Metrics code. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi Team,
Hawkular containers are failing whereas it was all good till yesterday. Is there something updated or needs attention to fix this?
Issue:
Current Version:
I am using latest version and have edited all 3 rc/hawkular and heapster to replace
:latestto:v1.4.1. That's not fixing the issue.Please review.
Thanks in Advance!
The text was updated successfully, but these errors were encountered: