Skip to content
Permalink
Branch: master
Find file Copy path
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
67 lines (52 sloc) 2.95 KB
owner_slack title parent layout section last_reviewed_on review_in
#govuk-2ndline
Signon app healthcheck not ok
/manual.html
manual_layout
Icinga alerts
2019-09-27
6 months

Signon has a standard /healthcheck endpoint, which is queried periodically by Icinga and is comprised of several checks. In addition to the standard database checks, there are also custom checks for this application.

  • api_tokens: looks at tokens for API Users to see if any are about to expire. You should rotate expiring tokens to ensure the associated application keeps working.

Rotating API Tokens

As a working example, let's say we have an alert like Content Publisher token for Publishing API expires in X days. In this example, the API User is Content Publisher and the application is Publishing API.

  • First login to Signon, go to API Users and click on the API User.

Check the Last synced at time to see if the API User is still using the application. If you are confident the token is unused, the you can just click the Revoke button to remove it and there's no need to continue.

  • Click Add application token, select the application and click Create access token.
  • Copy the new token and prepare to replace it in govuk-secrets.

How to do the last step depends on the application, but it should be something like rake eyaml:edit[integration,apps], depending on the environment you're working on.

  • Find a line like govuk::apps::content_publisher::publishing_api_bearer_token...
  • Replace the long string within GPG[xxxxxx] with the new token.
  • Make a PR with your change and once it is merged deploy the change with Puppet.

Changes to govuk-secrets do not automatically trigger a Puppet deploy. One way to work around this is to rebuild the last release. You then need to wait for Puppet to run on each of the affected machines.

  • Check the token is there with govuk_setenv content-publisher env | grep -i token
  • Check the app can still access the remote application APIs with the new token.
  • Once you're happy the new token works, you can Revoke the old one in Signon.

How to check the new token works depends on the application. One way to check the token works is to manually open a console for the application and call one of the remote APIs using gds-api-adapters.

Finally, most applications should automatically restart when Puppet updates the token on each machine, but you may need to do this manually so that it picks up the new token from the environment.

You can’t perform that action at this time.