Skip to content
This repository has been archived by the owner on Jun 1, 2023. It is now read-only.

Accept 503 responses from PE /status APIs #204

Merged
merged 1 commit into from
Nov 8, 2021

Conversation

Sharpie
Copy link
Member

@Sharpie Sharpie commented Nov 6, 2021

The puppetlabs/trapperkeeper-status library that provides the
/status/v1 API of various PE services will respond with a
HTTP 503 status code if any sub-service is not in a healthy state.
By default, Telegraf HTTP inputs will only accept 200 as a
successful response code. This state of affairs leads to situations
like an outage in pe-orchestration-services resulting in Telegraf
rejecting perfectly good JRuby metrics from pe-puppetserver on
compilers because the pcp-broker sub-service is unhealthy.

This commit updates the Telegraf configuration to accept HTTP 503
as a successful return code in addition to 200 so that an attempt
is made to process and store metrics data during outages of external
services.

The puppetlabs/trapperkeeper-status library that provides the
`/status/v1` API of various PE services will respond with a
HTTP 503 status code if any sub-service is not in a healthy state.
By default, Telegraf HTTP inputs will only accept 200 as a
successful response code. This state of affairs leads to situations
like an outage in `pe-orchestration-services` resulting in Telegraf
rejecting perfectly good JRuby metrics from `pe-puppetserver` on
compilers because the `pcp-broker` sub-service is unhealthy.

This commit updates the Telegraf configuration to accept HTTP 503
as a successful return code in addition to 200 so that an attempt
is made to process and store metrics data during outages of external
services.
@Sharpie Sharpie requested a review from a team as a code owner November 6, 2021 00:12
@puppet-community-rangefinder
Copy link

puppet_metrics_dashboard::profile::compiler is a type

that may have no external impact to Forge modules.

puppet_metrics_dashboard::profile::puppetdb is a type

that may have no external impact to Forge modules.

This module is declared in 0 of 578 indexed public Puppetfiles.


These results were generated with Rangefinder, a tool that helps predict the downstream impact of breaking changes to elements used in Puppet modules. You can run this on the command line to get a full report.

Exact matches are those that we can positively identify via namespace and the declaring modules' metadata. Non-namespaced items, such as Puppet 3.x functions, will always be reported as near matches only.

Copy link
Contributor

@jarretlavallee jarretlavallee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call.

@jarretlavallee jarretlavallee merged commit 45839d2 into puppetlabs:main Nov 8, 2021
@MartyEwings MartyEwings added enhancement New feature or request bug Something isn't working and removed enhancement New feature or request labels Feb 24, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants