Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fleet] Support preconfigured output secrets (scrypt edition) #172041

Conversation

jillguyonnet
Copy link
Contributor

Summary

Support for outputs with secrets preconfigured in kibana.yml. This was already implemented in #170259, which had to be reverted as the argon2 package was causing Windows tests to test (cf. this comment).

The present implementation follows option 2 in Infosec's recommendations with the Scrypt algorithm using the following params: N=2^14 (16 MiB), r=8 (1024 bytes), p=5. Note that Scrypt is built-in within the crypto module (see here for documentation).

Closes #166360

Testing

  1. Ensure the outputSecretsStorage experimental feature is enabled.
  2. Add the following to your kibana config:
    xpack.fleet.outputs:
      - id: my-logstash-output-with-a-secret
        name: preconfigured logstash output with a secret
        type:  logstash
        hosts: ["localhost:9999"]
        ssl:
          certificate: xxxxxxxxxx
        secrets:
          ssl:
            key: thisissecret
    
  3. Verify the secret has been correctly created, e.g. by issuing a GET .fleet-secrets/_search request in Dev Tools: the secret should be listed there.
  4. Change the preconfigured value and wait for kibana to restart: the secret should be updated with the new value.

@apmmachine
Copy link
Contributor

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • /oblt-deploy-serverless : Deploy a serverless Kibana instance using the Observability test environments.
  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@jillguyonnet jillguyonnet added Team:Fleet Team label for Observability Data Collection Fleet team v8.12.0 release_note:feature Makes this part of the condensed release notes backport:skip This commit does not require backporting labels Nov 28, 2023
@jillguyonnet jillguyonnet self-assigned this Nov 28, 2023
@jillguyonnet jillguyonnet changed the title Fleet/166360 preconfigured output secrets scrypt [Fleet] Support preconfigured output secrets (scrypt edition) Nov 28, 2023
export async function hashSecret(secret: string) {
return new Promise((resolve, reject) => {
const salt = crypto.randomBytes(16).toString('hex');
crypto.scrypt(secret, salt, 64, { p: 5 }, (err, derivedKey) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: should the integer values be moved to constants?

Copy link
Contributor

@juliaElastic juliaElastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jillguyonnet jillguyonnet marked this pull request as ready for review November 28, 2023 16:22
@jillguyonnet jillguyonnet requested a review from a team as a code owner November 28, 2023 16:22
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

hop-dev and others added 4 commits November 28, 2023 17:26
This PR adds support for outputs with secrets preconfigured in the
`kibana.yml` config file.

As Kibana needs to compare the value of the secret to manage updates, a
hash of the value is stored in the output's saved object. The
implementation follows [option 2 in Infosec's
recommendations](elastic/infosec#14853 (comment))
with the Argon2id algorithm.

See [here](https://www.npmjs.com/package/argon2) for information about
the `argon2` Node package and
[here](https://github.com/ranisalt/node-argon2/wiki/Options) for the
config options. Here, `argon2` was configured with the recommended
`m=19456 (19 MiB), t=2, p=1 ` (for some reason, `timeCost` cannot be set
to less than 2).

Closes elastic#166360

1. Ensure the [`outputSecretsStorage` experimental feature
](https://github.com/elastic/kibana/blob/fd4fdb01bcb011c0968a4ccab7ba739c3195016a/x-pack/plugins/fleet/common/experimental_features.ts#L26)is
enabled.
2. Add the following to your kibana config:
   ```
   xpack.fleet.outputs:
     - id: my-logstash-output-with-a-secret
       name: preconfigured logstash output with a secret
       type:  logstash
       hosts: ["localhost:9999"]
       ssl:
         certificate: xxxxxxxxxx
       secrets:
         ssl:
           key: thisissecret
   ```
3. Verify the secret has been correctly created, e.g. by issuing a `GET
.fleet-secrets/_search` request in Dev Tools: the secret should be
listed there.
4. Change the preconfigured value and wait for kibana to restart: the
secret should be updated with the new value.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: jillguyonnet <jill.guyonnet@gmail.com>
@jillguyonnet jillguyonnet force-pushed the fleet/166360-preconfigured-output-secrets-scrypt branch from fa9fae0 to 0184a8d Compare November 28, 2023 16:28
@kpollich
Copy link
Member

@elasticmachine merge upstream

| {
id: string;
};
service_token?: OutputSecret;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@juliaElastic Does this look OK? I made this change after #171875 was merged.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, it's good

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should test if a preconfigured remote_elasticsearch output works though

@jillguyonnet
Copy link
Contributor Author

@elasticmachine merge upstream

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jillguyonnet

@jillguyonnet jillguyonnet merged commit 905dad1 into elastic:main Nov 30, 2023
31 checks passed
@jillguyonnet jillguyonnet deleted the fleet/166360-preconfigured-output-secrets-scrypt branch November 30, 2023 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting release_note:feature Makes this part of the condensed release notes Team:Fleet Team label for Observability Data Collection Fleet team v8.12.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Fleet] Support preconfigured output secrets
8 participants