Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrate secret store with Logstash core (part 4) #8905

Merged
merged 4 commits into from
Jan 12, 2018

Conversation

jakelandis
Copy link
Contributor

This change introduces the command line tooling and hooks needed to allow Logstash to use the secret store. This change hooks into the same logic that the does the environment variable substitution. The command line mirrors the Elasticsearch command line, and is implemented primarily in Java.

Part of #8353

Documentation and full workflow integration tests are still coming ...but this should be the final change set to implement the secret store.

Once this review passes, I will squash this feature branch and create PR, ensure all tests pass, allow for one final review and merge this master. (doc and itests can come independently)

This change introduces the command line tooling and hooks needed to allow Logstash to use the secret store. This change hooks into the same logic that the does the environment variable substitution. The commnad line mirrors the Elasticsearch command line, and is implemented primarily in Java.

Part of elastic#8353
@jakelandis jakelandis changed the title Integrate secret store with Logstash core Integrate secret store with Logstash core (part 4) Jan 2, 2018
@jakelandis
Copy link
Contributor Author

To manually test follow the instructions at https://www.elastic.co/guide/en/elasticsearch/reference/current/secure-settings.html , just replace bin/elasticsearch-keystore with bin/logstash-keystore

To use the secret use ${key} in either the configuration or settings (logstash.yml)

@jordansissel jordansissel self-assigned this Jan 2, 2018
@jordansissel
Copy link
Contributor

Jenkins doesn't run on this feature branch, so here's the rake artifact:tar failure I get (javadoc):

/home/jls/projects/logstash/logstash-core/src/main/java/org/logstash/secret/store/SecretStoreFactory.java:42: error: unexpected text
     * @throws {@link SecretStoreException} if errors occur while loading, or if store already exists
       ^
/home/jls/projects/logstash/logstash-core/src/main/java/org/logstash/secret/store/SecretStoreFactory.java:52: error: unexpected text
     * @throws {@link SecretStoreException} if errors occur
       ^
/home/jls/projects/logstash/logstash-core/src/main/java/org/logstash/secret/store/SecretStoreFactory.java:63: error: unexpected text
     * @throws {@link SecretStoreException} if errors occur while loading, or if store does not exist
       ^
/home/jls/projects/logstash/logstash-core/src/main/java/org/logstash/secret/store/SecureConfig.java:8: error: bad use of '>'
 * A String -> char[] container that holds a referenced char[] obfuscated in memory and allows for easy clearing all values from memory.
             ^

@jakelandis
Copy link
Contributor Author

jakelandis commented Jan 3, 2018

@jordansissel - The artifact:tar build is now fixed.

@jordansissel
Copy link
Contributor

A fresh keystore contains one key. This may be confusing to users?

% bin/logstash-keystore list

keystore.seed

  • ✔️ Adding an entry
  • ✔️ Using the ${foo} syntax in the config.

For my test, I added an entry fancypants and used it like this:

% bin/logstash -e 'input { generator { count => 1 message => "${fancypants}"  } }'
{
  "message"    => "hello world",
...


private void create(SecureConfig config) {
if (System.getenv(SecretStoreFactory.ENVIRONMENT_PASS_KEY) == null) {
terminal.write(String.format("WARNING: The keystore password is not set. Please set the environment variable `%s`. Failure to do so will result in" +
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: The keystore password is not set. Please set the environment variable LOGSTASH_KEYSTORE_PASS. Failure to do so will result in reduced security. Continue anyway ? [y/N]

It's unclear what behavior the user should expect by pressing y here. Maybe "Continue without password protection on the keystore?"

Copy link
Contributor Author

@jakelandis jakelandis Jan 3, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Continue without password protection on the keystore?

+ 1 , will update

I struggled a bit on the wording of this.

@jordansissel
Copy link
Contributor

I didn't remember what the precedence order is, and testing it shows that the keystore has priority over environment variables. Noting this so when we write the docs, this order is documented.


Testing what happens when an invalid entry is given (no env, no keystore entry):

"Cannot evaluate ${env}. Replacement variable env is not defined in a Logstash secret store or as an Environment entry and there is no default value given."

👍

@jordansissel
Copy link
Contributor

Testing both env and keystore:

# fancypants comes from keystore.
% env="foo" bin/logstash -e 'input { generator { count => 1 message => "${fancypants} ${env}"  } }'
...
  "message"    => "hello world foo"

@jakelandis
Copy link
Contributor Author

re : keystore.seed

This mirrors Elasticsearch's keystore in both name and function.

index = ARGV.find_index("--path.settings")
# strip out any path.settings from the command line
unless index.nil?
path_settings_value = ARGV.slice!(index, 2)[1]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels weird modifying the value of a constant (ARGV) here. No change necessarily required, but something to think on.

include LogStash::Util::Loggable

begin
index = ARGV.find_index("--path.settings")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This flag would benefit from users being able to discover it. I didn't see it listed in the --help output:

% bin/logstash-keystore --help


Commands
--------
create - Creates a new Logstash keystore
list   - List entries in the keystore
add    - Add a value to the keystore
remove - Remove a value from the keystore

And doing --help on a subcommand results in possibly confusing behavior:

% bin/logstash-keystore add --help

Enter value for --help: 

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add --help as a sub command

@@ -74,6 +75,9 @@ module Environment
# Compute the default dead_letter_queue path based on `path.data`
default_dlq_file_path = ::File.join(SETTINGS.get("path.data"), "dead_letter_queue")
SETTINGS.register Setting::WritableDirectory.new("path.dead_letter_queue", default_dlq_file_path)
# Compute the default secret store path based on `path.data`
default_secret_store_file_path = ::File.join(SETTINGS.get("path.data"), "logstash.keystore")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Elasticsearch docs say:

To create the elasticsearch.keystore, use the create command: ...
The file elasticsearch.keystore will be created alongside elasticsearch.yml.

However, bin/logstash-keystore create creates this file in the path.data directory:

% bin/logstash-keystore create
...
Created Logstash keystore at /home/jls/projects/logstash/data/logstash.keystore

Should this be path.config instead? (It's user-configuration, so my vote is config, not data directory)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will update.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to path.settings (next to logstash.yml), not path.config (next to my_input.conf) ...seems like a better fit, and assume that is what you meant.

ls config
jvm.options       log4j2.properties logstash.keystore logstash.yml      pipelines.yml     startup.options

@jordansissel
Copy link
Contributor

This mirrors Elasticsearch's keystore in both name and function.

% bin/elasticsearch-keystore remove keystore.seed
% bin/elasticsearch-keystore list
<no output>

It's unclear what purpose keystore.seed has for users. I grepped around the Elasticsearch code and it's not obvious from the source code. You can remove it (shown above) and neither elasticsearch-keystore nor elasticsearch seem to notice (no warnings/errors).

The two questions I have are: Will a user ever set this manually? Will a user ever need to reference it in their logstash configuration? I think both answers are no, right? If it's not a thing users need to interact with, we should not show it to them.

@jakelandis
Copy link
Contributor Author

Will a user ever set this manually?

No.

Will a user ever need to reference it in their logstash configuration?

No

I think both answers are no, right?

You clearly said you two questions and this is three, so I can't answer this one.

If it's not a thing users need to interact with, we should not show it to them.

Will update to no longer show it. I only added as visible to match ES and some minor testing benefits.

@jordansissel
Copy link
Contributor

I only added as visible to match ES and some minor testing benefits.

Noted! Consistency is a good approach. Maybe we can convince ES to remove this from their listing also.

@jakelandis
Copy link
Contributor Author

jakelandis commented Jan 5, 2018

@jordansissel - Could you take another look ? All changes have been made as requested (with just tiny bit extra) on 6600bb0

The command line help is quite a bit more verbose now, please let me know if you think we should trim it down any.

Ninja Edit: updated diff hash link

@jordansissel
Copy link
Contributor

I like the new help output.

% bin/logstash-keystore --path.settings /tmp create
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults

WARNING: The keystore password is not set. Please set the environment variable `LOGSTASH_KEYSTORE_PASS`. Failure to do so will result in reduced security. Continue without password protection on the keystore? [y/N] y
Created Logstash keystore at /tmp/logstash.keystore

The first warning warning doesn't seem to have enough information to take action. I believe I get this warning because I specify path.settings of /tmp and there is no /tmp/logstash.yml ?

@jakelandis
Copy link
Contributor Author

The first warning warning doesn't seem to have enough information to take action

That warning is actually not new, and comes from the core code that checks for this. I agree is kinda of confusing. Can we consider that one outside the scope of this review ?

@jakelandis
Copy link
Contributor Author

Maybe call this 'argument' or 'parameter' ?

Agreed, will change. I think i like argument slightly better.

@jakelandis
Copy link
Contributor Author

Where is LS_SETTINGS_DIR referenced ?

@jordansissel
Copy link
Contributor

I ran the tests via gradle and many failures:

% ./gradlew test
    Finished in 4 minutes 55.9 seconds (files took 8.88 seconds to load)
    3161 examples, 33 failures, 4 pending

    Failed examples:

    rspec ./logstash-core/spec/logstash/config/mixin_spec.rb:421 # LogStash::Config::Mixin environment variable evaluation when an environment variable is set should use the value in the variable
    rspec ./logstash-core/spec/logstash/config/mixin_spec.rb:431 # LogStash::Config::Mixin environment variable evaluation when an environment variable is set should validate settings after interpolating ENV variables
    rspec ./logstash-core/spec/logstash/config/mixin_spec.rb:465 # LogStash::Config::Mixin environment variable evaluation should support $ in values should not support $ in environment variable name
    rspec ./logstash-core/spec/logstash/config/mixin_spec.rb:461 # LogStash::Config::Mixin environment variable evaluation should support $ in values should support $ in values
    rspec ./logstash-core/spec/logstash/config/mixin_spec.rb:387 # LogStash::Config::Mixin environment variable evaluation when an environment variable is not set and a default is given should use the default
    rspec ./logstash-core/spec/logstash/config/mixin_spec.rb:370 # LogStash::Config::Mixin environment variable evaluation when an environment variable is not set and no default is given should raise a configuration error
    rspec ./logstash-core/spec/logstash/runner_spec.rb:296 # LogStash::Runner pipeline settings config.debug should set 'config.debug' to false by default
    rspec ./logstash-core/spec/logstash/runner_spec.rb:159 # LogStash::Runner pipeline settings when :path.data is defined by the user should set data paths
    rspec ./logstash-core/spec/logstash/runner_spec.rb:191 # LogStash::Runner pipeline settings when :path.data is defined by the user and path.dead_letter_queue is manually set should set data paths
    rspec ./logstash-core/spec/logstash/runner_spec.rb:173 # LogStash::Runner pipeline settings when :path.data is defined by the user and path.queue is manually set should set data paths
    rspec ./logstash-core/spec/logstash/runner_spec.rb:206 # LogStash::Runner pipeline settings when :http.host is defined by the user should pass the value to the webserver
    rspec ./logstash-core/spec/logstash/runner_spec.rb:252 # LogStash::Runner pipeline settings when no :http.port is not defined by the user should use the default settings
    rspec ./logstash-core/spec/logstash/runner_spec.rb:218 # LogStash::Runner pipeline settings when :http.host is not defined by the user should pass the value to the webserver
    rspec ./logstash-core/spec/logstash/runner_spec.rb:281 # LogStash::Runner pipeline settings when :pipeline_workers is defined by the user should pass the value to the pipeline
    rspec ./logstash-core/spec/logstash/runner_spec.rb:264 # LogStash::Runner pipeline settings when :pipeline_workers is not defined by the user should not pass the value to the pipeline
    rspec ./logstash-core/spec/logstash/runner_spec.rb:240 # LogStash::Runner pipeline settings when :http.port is defined by the user should pass a range value to the webserver
    rspec ./logstash-core/spec/logstash/runner_spec.rb:230 # LogStash::Runner pipeline settings when :http.port is defined by the user should pass a single value to the webserver
    rspec ./logstash-core/spec/logstash/runner_spec.rb:398 # LogStash::Runner logstash modules --modules with an available module specified and a mocked connection to elasticsearch should not terminate logstash
    rspec ./logstash-core/spec/logstash/runner_spec.rb:358 # LogStash::Runner logstash modules --modules with an available module specified but no connection to elasticsearch should log fatally and return a bad exit code
    rspec ./logstash-core/spec/logstash/runner_spec.rb:417 # LogStash::Runner logstash modules --modules with an unavailable module specified should log fatally and return a bad exit code
    rspec ./logstash-core/spec/logstash/runner_spec.rb:326 # LogStash::Runner logstash modules --config.test_and_exit with a good configuration should exit successfully
    rspec ./logstash-core/spec/logstash/runner_spec.rb:443 # LogStash::Runner --log.level when setting to verbose should set log level to info
    rspec ./logstash-core/spec/logstash/runner_spec.rb:436 # LogStash::Runner --log.level when setting to debug should set log level to debug
    rspec ./logstash-core/spec/logstash/runner_spec.rb:429 # LogStash::Runner --log.level when not set should set log level to warn
    rspec ./logstash-core/spec/logstash/runner_spec.rb:450 # LogStash::Runner --log.level when setting to quiet should set log level to error
    rspec ./logstash-core/spec/logstash/runner_spec.rb:478 # LogStash::Runner --log.level deprecated flags when using --debug should still set the log level accordingly
    rspec ./logstash-core/spec/logstash/runner_spec.rb:491 # LogStash::Runner --log.level deprecated flags when using --verbose should still set the log level accordingly
    rspec ./logstash-core/spec/logstash/runner_spec.rb:465 # LogStash::Runner --log.level deprecated flags when using --quiet should still set the log level accordingly
    rspec ./logstash-core/spec/logstash/runner_spec.rb:505 # LogStash::Runner path.settings if does not exist should not terminate logstash
    rspec ./logstash-core/spec/logstash/runner_spec.rb:76 # LogStash::Runner argument parsing when -e is given should execute the agent
    rspec ./logstash-core/spec/logstash/runner_spec.rb:114 # LogStash::Runner --auto-reload when -e is given should exit immediately
    rspec ./logstash-core/spec/logstash/runner_spec.rb:55 # LogStash::Runner argument precedence favors the last occurence of an option
    rspec ./logstash-core/spec/logstash/runner_spec.rb:128 # LogStash::Runner --config.test_and_exit with a good configuration should exit successfully

Most (all? I haven't evaluated) of the failures are basically this:

          # ArgumentError:
          #   Setting "keystore.file" has already been registered as #<LogStash::Setting::String:0x602b9f2e @possible_strings=[], @name="keystore.file", @value=nil, @value_is_set=false, @klass=String, @strict=true, @default="/home/jls/projects/logstash/config/logstash.keystore", @validator_proc=nil>
          #   ./logstash-core/lib/logstash/settings.rb:25:in `register'

terminal.writeLine("");
terminal.writeLine("Options:");
terminal.writeLine("--------");
terminal.writeLine("--path.settings - Set the directory for the keystore. This is should be the same directory as the logstash.yml settings file. " +
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is where LS_SETTINGS_DIR is referenced.

@jordansissel
Copy link
Contributor

Can we consider that one outside the scope of this review ?

I'm ok with this, but I'm certain the first customers who experience this will file an issue which will be escalated to us. I'd prefer to avoid that human cost.

It's likely a user providing --path.settings is probably rare, do you think? If expected to be rare, we can defer to another improvement of the message to another PR.

@jordansissel
Copy link
Contributor

this is looking very good!

I'm not sure why the (ruby) test suite fails for me. Does it pass for you?

@jordansissel
Copy link
Contributor

@jakelandis let me know what else you need.

Conditional LGTM if:

  1. the tests pass
  2. the LS_SETTINGS_DIR mention is removed from --help
  3. the Could not find logstash.yml message concern is resolved (which can be resolved as defer-until-future-pr, at your option)

If all 3 above are resolved, this is good to merge.

@jakelandis
Copy link
Contributor Author

jakelandis commented Jan 12, 2018

@jordansissel - Changes : 135819a

the tests pass

  • Had a minor bug in error handling of mis-configuration (only possible via tests)
  • Had issues with the global state of the SETTINGS object bleeding global state between tests when run in a certain order.
  • I have them passing locally, but will let Jenkins be the authoritative source with a PR to master.

the LS_SETTINGS_DIR mention is removed from --help

done.

the Could not find logstash.yml message concern is resolved (which can be resolved as defer-until-future-pr, at your option)

Logged #8934 to address

@jakelandis
Copy link
Contributor Author

merging to feature branch.

@jakelandis jakelandis merged commit fb4d082 into elastic:secret-store Jan 12, 2018
@jakelandis
Copy link
Contributor Author

grr.... just found another issue with the global state/registration settings mess...

@jakelandis
Copy link
Contributor Author

jakelandis commented Jan 12, 2018

GH won't let me add more commits here... so I pushed directly to feature branch 8bb47d9

I will have one last PR before this hits master, so if changes are requested I can make them there.

jakelandis added a commit to jakelandis/logstash that referenced this pull request Jan 12, 2018
…ta store.

Fixes elastic#8657

Part 1: API and JavaKeyStore implementation (elastic#8657)

Introduces the API to read/write/delete sensitive data from a secure store and includes a Java KeyStore implementation. Note - this commit does NOT integrate with the Logstash configuration or settings.

Part 2: Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support (elastic#8659)

*  Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support

* Introduce a SecretStoreFactory to allow runtime definition of SecretStore implementation.
* Introduce a SecureConfig to allow simple configuration of different SecretStore implementaiton.
* Introduce random default password plus obfuscation. Best attempt at security through obscurity.
* Corrections / better support for x-JVM modification.

Part 3: Secret Store: SecretStore, SecretStoreFactory, JavaKeystore - refacactor (elastic#8745)

* Adds more CRUD like operations for SecretStore API
* SecretStoreFactory Mirror API's CRUD operations
* Adds 'exists' to API to allow command line warning 'Overwrite ?'
* Minor readabiliy

Part 4: Integrate secret store with Logstash core (elastic#8905)

This change introduces the command line tooling and hooks needed to allow Logstash to use the secret store. This change hooks into the same logic that the does the environment variable substitution. The commnad line mirrors the Elasticsearch command line, and is implemented primarily in Java.
jakelandis added a commit to jakelandis/logstash that referenced this pull request Jan 12, 2018
…ta store.

Fixes elastic#8657

Part 1: API and JavaKeyStore implementation (elastic#8657)

Introduces the API to read/write/delete sensitive data from a secure store and includes a Java KeyStore implementation. Note - this commit does NOT integrate with the Logstash configuration or settings.

Part 2: Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support (elastic#8659)

*  Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support

* Introduce a SecretStoreFactory to allow runtime definition of SecretStore implementation.
* Introduce a SecureConfig to allow simple configuration of different SecretStore implementaiton.
* Introduce random default password plus obfuscation. Best attempt at security through obscurity.
* Corrections / better support for x-JVM modification.

Part 3: Secret Store: SecretStore, SecretStoreFactory, JavaKeystore - refacactor (elastic#8745)

* Adds more CRUD like operations for SecretStore API
* SecretStoreFactory Mirror API's CRUD operations
* Adds 'exists' to API to allow command line warning 'Overwrite ?'
* Minor readabiliy

Part 4: Integrate secret store with Logstash core (elastic#8905)

This change introduces the command line tooling and hooks needed to allow Logstash to use the secret store. This change hooks into the same logic that the does the environment variable substitution. The commnad line mirrors the Elasticsearch command line, and is implemented primarily in Java.
jakelandis added a commit to jakelandis/logstash that referenced this pull request Jan 12, 2018
…ta store.

Fixes elastic#8657

Part 1: API and JavaKeyStore implementation (elastic#8657)

Introduces the API to read/write/delete sensitive data from a secure store and includes a Java KeyStore implementation. Note - this commit does NOT integrate with the Logstash configuration or settings.

Part 2: Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support (elastic#8659)

*  Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support

* Introduce a SecretStoreFactory to allow runtime definition of SecretStore implementation.
* Introduce a SecureConfig to allow simple configuration of different SecretStore implementaiton.
* Introduce random default password plus obfuscation. Best attempt at security through obscurity.
* Corrections / better support for x-JVM modification.

Part 3: Secret Store: SecretStore, SecretStoreFactory, JavaKeystore - refacactor (elastic#8745)

* Adds more CRUD like operations for SecretStore API
* SecretStoreFactory Mirror API's CRUD operations
* Adds 'exists' to API to allow command line warning 'Overwrite ?'
* Minor readabiliy

Part 4: Integrate secret store with Logstash core (elastic#8905)

This change introduces the command line tooling and hooks needed to allow Logstash to use the secret store. This change hooks into the same logic that the does the environment variable substitution. The commnad line mirrors the Elasticsearch command line, and is implemented primarily in Java.
jakelandis added a commit to jakelandis/logstash that referenced this pull request Jan 12, 2018
…ta store.

Fixes elastic#8657

Part 1: API and JavaKeyStore implementation (elastic#8657)

Introduces the API to read/write/delete sensitive data from a secure store and includes a Java KeyStore implementation. Note - this commit does NOT integrate with the Logstash configuration or settings.

Part 2: Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support (elastic#8659)

*  Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support

* Introduce a SecretStoreFactory to allow runtime definition of SecretStore implementation.
* Introduce a SecureConfig to allow simple configuration of different SecretStore implementaiton.
* Introduce random default password plus obfuscation. Best attempt at security through obscurity.
* Corrections / better support for x-JVM modification.

Part 3: Secret Store: SecretStore, SecretStoreFactory, JavaKeystore - refacactor (elastic#8745)

* Adds more CRUD like operations for SecretStore API
* SecretStoreFactory Mirror API's CRUD operations
* Adds 'exists' to API to allow command line warning 'Overwrite ?'
* Minor readabiliy

Part 4: Integrate secret store with Logstash core (elastic#8905)

This change introduces the command line tooling and hooks needed to allow Logstash to use the secret store. This change hooks into the same logic that the does the environment variable substitution. The commnad line mirrors the Elasticsearch command line, and is implemented primarily in Java.
jakelandis added a commit to jakelandis/logstash that referenced this pull request Jan 13, 2018
…ta store.

Fixes elastic#8657

Part 1: API and JavaKeyStore implementation (elastic#8657)

Introduces the API to read/write/delete sensitive data from a secure store and includes a Java KeyStore implementation. Note - this commit does NOT integrate with the Logstash configuration or settings.

Part 2: Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support (elastic#8659)

*  Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support

* Introduce a SecretStoreFactory to allow runtime definition of SecretStore implementation.
* Introduce a SecureConfig to allow simple configuration of different SecretStore implementaiton.
* Introduce random default password plus obfuscation. Best attempt at security through obscurity.
* Corrections / better support for x-JVM modification.

Part 3: Secret Store: SecretStore, SecretStoreFactory, JavaKeystore - refacactor (elastic#8745)

* Adds more CRUD like operations for SecretStore API
* SecretStoreFactory Mirror API's CRUD operations
* Adds 'exists' to API to allow command line warning 'Overwrite ?'
* Minor readabiliy

Part 4: Integrate secret store with Logstash core (elastic#8905)

This change introduces the command line tooling and hooks needed to allow Logstash to use the secret store. This change hooks into the same logic that the does the environment variable substitution. The commnad line mirrors the Elasticsearch command line, and is implemented primarily in Java.

Part 5: Hardening and test fixes (this PR)
elasticsearch-bot pushed a commit that referenced this pull request Jan 13, 2018
…ta store.

Fixes #8657

Part 1: API and JavaKeyStore implementation (#8657)

Introduces the API to read/write/delete sensitive data from a secure store and includes a Java KeyStore implementation. Note - this commit does NOT integrate with the Logstash configuration or settings.

Part 2: Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support (#8659)

*  Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support

* Introduce a SecretStoreFactory to allow runtime definition of SecretStore implementation.
* Introduce a SecureConfig to allow simple configuration of different SecretStore implementaiton.
* Introduce random default password plus obfuscation. Best attempt at security through obscurity.
* Corrections / better support for x-JVM modification.

Part 3: Secret Store: SecretStore, SecretStoreFactory, JavaKeystore - refacactor (#8745)

* Adds more CRUD like operations for SecretStore API
* SecretStoreFactory Mirror API's CRUD operations
* Adds 'exists' to API to allow command line warning 'Overwrite ?'
* Minor readabiliy

Part 4: Integrate secret store with Logstash core (#8905)

This change introduces the command line tooling and hooks needed to allow Logstash to use the secret store. This change hooks into the same logic that the does the environment variable substitution. The commnad line mirrors the Elasticsearch command line, and is implemented primarily in Java.

Part 5: Hardening and test fixes (this PR)

Fixes #8935
elasticsearch-bot pushed a commit that referenced this pull request Jan 13, 2018
…ta store.

Fixes #8657

Part 1: API and JavaKeyStore implementation (#8657)

Introduces the API to read/write/delete sensitive data from a secure store and includes a Java KeyStore implementation. Note - this commit does NOT integrate with the Logstash configuration or settings.

Part 2: Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support (#8659)

*  Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support

* Introduce a SecretStoreFactory to allow runtime definition of SecretStore implementation.
* Introduce a SecureConfig to allow simple configuration of different SecretStore implementaiton.
* Introduce random default password plus obfuscation. Best attempt at security through obscurity.
* Corrections / better support for x-JVM modification.

Part 3: Secret Store: SecretStore, SecretStoreFactory, JavaKeystore - refacactor (#8745)

* Adds more CRUD like operations for SecretStore API
* SecretStoreFactory Mirror API's CRUD operations
* Adds 'exists' to API to allow command line warning 'Overwrite ?'
* Minor readabiliy

Part 4: Integrate secret store with Logstash core (#8905)

This change introduces the command line tooling and hooks needed to allow Logstash to use the secret store. This change hooks into the same logic that the does the environment variable substitution. The commnad line mirrors the Elasticsearch command line, and is implemented primarily in Java.

Part 5: Hardening and test fixes (this PR)

Fixes #8935
insukcho pushed a commit to insukcho/logstash that referenced this pull request Feb 1, 2018
…ta store.

Fixes elastic#8657

Part 1: API and JavaKeyStore implementation (elastic#8657)

Introduces the API to read/write/delete sensitive data from a secure store and includes a Java KeyStore implementation. Note - this commit does NOT integrate with the Logstash configuration or settings.

Part 2: Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support (elastic#8659)

*  Secret Store: SecretStoreFactory, SecureConfig, Obfuscation and X-JVM support

* Introduce a SecretStoreFactory to allow runtime definition of SecretStore implementation.
* Introduce a SecureConfig to allow simple configuration of different SecretStore implementaiton.
* Introduce random default password plus obfuscation. Best attempt at security through obscurity.
* Corrections / better support for x-JVM modification.

Part 3: Secret Store: SecretStore, SecretStoreFactory, JavaKeystore - refacactor (elastic#8745)

* Adds more CRUD like operations for SecretStore API
* SecretStoreFactory Mirror API's CRUD operations
* Adds 'exists' to API to allow command line warning 'Overwrite ?'
* Minor readabiliy

Part 4: Integrate secret store with Logstash core (elastic#8905)

This change introduces the command line tooling and hooks needed to allow Logstash to use the secret store. This change hooks into the same logic that the does the environment variable substitution. The commnad line mirrors the Elasticsearch command line, and is implemented primarily in Java.

Part 5: Hardening and test fixes (this PR)

Fixes elastic#8935
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants