New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

amplify-agent is not running after system reboot #62

Open
gpluess opened this Issue Jul 18, 2017 · 5 comments

Comments

Projects
None yet
5 participants
@gpluess
Copy link

gpluess commented Jul 18, 2017

I'm using version 0.44-2~xenial.

To find out, what was going on, I did the following: systemctl status amplify-agent tells me this:

* amplify-agent.service - LSB: Stop/start nginx-amplify-agent
   Loaded: loaded (/etc/init.d/amplify-agent; bad; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2017-07-18 21:39:24 UTC; 7min ago
     Docs: man:systemd-sysv-generator(8)
  Process: 1147 ExecStart=/etc/init.d/amplify-agent start (code=exited, status=1/FAILURE)

I then looked at the /var/log/amplify-agent/agent.log log file and discovered this:

2017-07-18 21:39:24,227 [1300] MainThread Hostname: localhost is local
2017-07-18 21:39:24,270 [1300] MainThread configtest check
2017-07-18 21:39:24,444 [1300] MainThread failed POST "https://receiver.amplify.nginx.com:443/1.3/[my-api-key]/agent/", exception: "("bad handshake: Error([('', 'osrandom_rand_bytes', 'getrandom() initialization failed.')],)",)"
2017-07-18 21:39:24,444 [1300] MainThread [None] post https://receiver.amplify.nginx.com:443/1.3/[my-api-key]/agent/ 500 10 0 0.173

However, when I start the agent manually with systemctl start amplify-agent everything works fine.

@gpluess gpluess changed the title aplify-agent is not running after system reboot amplify-agent is not running after system reboot Jul 19, 2017

@seanmavley

This comment has been minimized.

Copy link

seanmavley commented Nov 3, 2017

Using systemctl doesn't work for me. Having same issue after restart.

Here's my error:

Nov 03 08:26:28 user amplify-agent[14545]:   File "/usr/lib/python2.7/dist-packages/amplify/agent/common/util/http.py", line 131, in post
Nov 03 08:26:28 user amplify-agent[14545]:     return self.make_request(url, 'post', data=data, timeout=timeout, json=json)
Nov 03 08:26:28 user amplify-agent[14545]:   File "/usr/lib/python2.7/dist-packages/amplify/agent/common/util/http.py", line 113, in make_request
Nov 03 08:26:28 user amplify-agent[14545]:     raise e
Nov 03 08:26:28 user amplify-agent[14545]: ConnectionError: HTTPSConnectionPool(host='receiver.amplify.nginx.com', port=443): Max retries exceeded w
Nov 03 08:26:28 user systemd[1]: amplify-agent.service: Control process exited, code=exited status=1

@dedm dedm self-assigned this Nov 5, 2017

@dedm

This comment has been minimized.

Copy link
Contributor

dedm commented Nov 5, 2017

Thanks @seanmavley, it's a known issue, we need to start agent after the system has access to network. We will address this issue soon, please bear with us.

@dvershinin

This comment has been minimized.

Copy link

dvershinin commented Dec 21, 2018

Same issue with CentOS 7. Why this hasn't been addressed in a year?!

After reboot:

● amplify-agent.service - LSB: start and stop nginx-amplify-agent daemon
   Loaded: loaded (/etc/rc.d/init.d/amplify-agent; bad; vendor preset: disabled)
   Active: failed (Result: resources) since Thu 2018-12-20 22:07:18 UTC; 10h ago
     Docs: man:systemd-sysv-generator(8)
  Process: 2497 ExecStart=/etc/rc.d/init.d/amplify-agent start (code=exited, status=0/SUCCESS)

Dec 20 22:07:18 ps.ethicalpets.co.uk amplify-agent[2497]: Could not connect to API via url https://receiver.amplify.nginx.com:443/1.4
Dec 20 22:07:18 ps.ethicalpets.co.uk amplify-agent[2497]: We will try to establish a connection once again in a minute.
Dec 20 22:07:18 ps.ethicalpets.co.uk amplify-agent[2497]: Could not connect to API via url https://receiver.amplify.nginx.com:443/1.4
Dec 20 22:07:18 ps.ethicalpets.co.uk amplify-agent[2497]: Giving up after three attempts...
Dec 20 22:07:18 ps.ethicalpets.co.uk runuser[2665]: pam_unix(runuser:session): session closed for user nginx
Dec 20 22:07:18 ps.ethicalpets.co.uk amplify-agent[2497]: [FAILED]
Dec 20 22:07:18 ps.ethicalpets.co.uk systemd[1]: PID file /var/run/amplify-agent/amplify-agent.pid not readable (yet?) after start.
Dec 20 22:07:18 ps.ethicalpets.co.uk systemd[1]: Failed to start LSB: start and stop nginx-amplify-agent daemon.
Dec 20 22:07:18 ps.ethicalpets.co.uk systemd[1]: Unit amplify-agent.service entered failed state.
Dec 20 22:07:18 ps.ethicalpets.co.uk systemd[1]: amplify-agent.service failed.

Always have to restart it manually after each reboot or get spammed with lots of emails about inactive agent.

@IWriteThings

This comment has been minimized.

Copy link

IWriteThings commented Feb 5, 2019

This issue is still plaguing me in 2019. Any fix for this yet?

@dvershinin

This comment has been minimized.

Copy link

dvershinin commented Feb 10, 2019

I've discovered that if the agent fails to start with similiar agent.log messages as original poster's, that means not enough entropy for TLS connection at boot time.

To fix you may want to use rng-tools as outlined in this post of mine. In CentOS 7 it basically goes down to:

yum -y install rng-tools
systemctl enable rngd

And viola, agent works after each reboot :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment