Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include custom check definition attributes in the check request payload #1360

Closed
calebhailey opened this issue Jul 7, 2016 · 5 comments

Comments

Projects
None yet
3 participants
@calebhailey
Copy link
Member

commented Jul 7, 2016

The Sensu check request payload has historically only contained the minimum number of check definition attributes needed to execute the request on a sensu-client: the command, source, and extension attributes.

Example:

{
  "name": "example_check",
  "issued": 1234567890,
  "command": "check-http.rb -u https://sensuapp.org",
  "source": "sensuapp.org"
}

NOTE: the other half of this legacy check request "shaping" behavior happens during check result processing. The Sensu server merges check result data w/ the original check definition when processing check results, thus restoring any custom check definition attributes to the event context and making them available for handlers, and in the Sensu API (i.e. for dashboards like Uchiwa to display).

With the introduction of "extended check token substitution" in Sensu Core 0.24 (i.e. allowing for token substitution on any check definition attribute – not just the command field), this check request payload "shaping" behavior is resulting in some confusion for Sensu users who are appropriately expecting to be able to perform token substitution on custom check definition attribute fields.

For example, if a check definition contains a custom attribute called graph that has a client :::name::: token in it, this graph attribute is not being included in the check request, and is thus not available for the sensu-client to perform token substitution on. The exception to this rule – further adding to the confusion – is that the sensu-client process will merge check requests with local check definitions of the same name prior to executing the request, potentially restoring custom check definition attributes to the check request payload and making them available for the sensu-client to perform token substitution on – so a sensu-client process running on the Sensu server will always have access to the check definition, thus applying the token substitution to all the fields in the check (and seemingly exhibiting a different behavior than a sensu-client running on a remote system).

PR #1359 will change this behavior to make the new extended token substitution behave as expected, now including all check definition attributes in the check request payload (including custom definition attributes, but excluding interval and subscribers which are only used by the server to publish the request).

@calebhailey

This comment has been minimized.

Copy link
Member Author

commented Jul 7, 2016

It should also be noted that it is possible to "resolve" this issue by deploying check definitions to all Sensu clients, thus making them available to be merged with incoming check requests (making custom attributes available for token substitution). This workflow requires the use of some configuration management or other automation tooling to ensure that changes are kept in sync between the client and server, which is a popular workflow for many Sensu users, but not ideal for every user.

@calebhailey

This comment has been minimized.

Copy link
Member Author

commented Jul 7, 2016

Fixed by #1360

@calebhailey calebhailey closed this Jul 7, 2016

@calebhailey calebhailey modified the milestones: 0.25, 0.25.5 Jul 12, 2016

@Gillingham

This comment has been minimized.

Copy link

commented Jul 13, 2016

Is this supposed to get into 25.5? Running the official RPM, sensu-0.25.5-1.x86_64 and I'm getting Unmatched client token(s): checker.user, checker.password
With a config of:

 "checker_sample": {
      "source": "foobar",
      "handlers": ["mailer"],
      "command": "foo -user :::checker.user::: -password :::checker.password:::",
      "subscribers": [
        "foobar"
      ],
      "checker": {
          "user": "foo",
          "password": "bar"
      },
      "interval": 60
    }

or am I mis-understanding what this issue was/was fixed?

@cwjohnston

This comment has been minimized.

Copy link
Member

commented Jul 13, 2016

@Gillingham The change in question here is to allow custom check attributes other than command to make use of token substitution to be passed along in a published check request.

Edit: allowing token substitution in check attributes other than command was added in 0.24, see #1294 and #1296

In your case, the token substitution routine is looking for checker.user and checker.password attributes under the client scope and not finding them there as they are actually under checks.checker_sample.checker.

Try placing the custom checker attributes under the client scope, i.e. outside the check definition. See https://sensuapp.org/docs/latest/reference/checks.html#check-token-substitution for more details.

@Gillingham

This comment has been minimized.

Copy link

commented Jul 13, 2016

@cwjohnston aw dang, these checks all run on proxy clients, so getting the client substitution is a bit harder, since a client may be running checks that have differing values for the proxy.
Unfortunately using the HTTP API to set the proxy client values is a bit harder, since those values tend to get reset/lost when the sensu server restarts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.