Part 11/n - Add K/V Secrets Engine #109
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Start work on plugin * Update deps * More API work * Fix encrypted keys storage when listing folders * Fix issue with cache misses on the encrypted key storage * More work on API * Add passthrough factory and tests * More updates * Add salt and additional path files * Add an invalidate function * Fix deleting old versions and add tests * Remove ttl references for now * Downgrade passthrough requests that are using the new API * Add test for downgrade of passthrough requests * Add more tests * Add an upgrade function * Add upgrade check for API * Add test for upgrading * Cleanup code a bit * Update deps * Update deps * Use int arrays to archive/destroy multiple versions at once * Fix a few small things * Fix a few small things * Fix a few small things * Add test for config endpoint * Fix and test cleaning up older versions * Update comments and paths * Add test for reloading policy and salts * Add field descriptions * Update deps * Update deps * Use the keysutil LoadPolicy function * Fix upgrading a bit * Fix typo * Some review feedback * Some review feedback * update seal wrapped paths * update lock logic * Fix error message * Fix comment * Fix tests * remove unneeded call to current version * Update whitespace in proto file * review feedback * Rename archive to delete * Add created time and update time into newly created metadata keys * Move from uid in map to BackendUUID variable * Update package name and fix some review comments * Only update the changed values on config * Update deps * Use the http package to parse the headers * Fix listing downgrade * Update test
…ng (#5) * upgrade: use the upgrade canary as a means of kicking off the process * Fix tests and don't run while in metadata mode
* Make updates to upgrading on secondaries and version numbers * Fix test * Don't run upgrade on DR secondaries
* Cache global config * Use a mutex instead of an atomic value
* specify reader when calling policy.Rotate() * updates vault dep
Move tests to circle
Include delete_versions_after regardless of whether it has been overridden
* set proto go_package to github.com/hashicorp/kv * include paths=source_relative in proto make target * Add CustomMetadata to KeyMetadata proto message * compile proto changes with CustomMetadata field * change protoc-gen-go version from v1.27.1-devel->v1.26.0 * add read/write of custom_metadata field to metadata path * add custom metadata input validation logic * perform POST/PUT custom_metadata validation prior to fetch from storage * Fix custom_metadata not being written to storage after refactor * Add custom_metadata and metadata merge behavior unit tests * Only overwrite custom_metadata if key provided in request * use quoted character format for validation errors * simplify custom metadata check for writing custom metadata to storage * early return in validateCustomMetadata for too many keys * include unquoted key in unprintable key error string
* add initial PatchOperation support with cas validation * update go.mod, go.sum * Add PatchOperation case to invalid patch fallthrough * initial PatchOperation handler for data endpoint * move old version cleanup logic into its own function * respond with 404 for PATCH to deleted or destroyed version * go fmt * add test for put with cas=0 * additional patch tests for no data provided and entry/metadata not found * add better error messaging for data path tests * add comments * make patch resp when deleted/destroyed consistent with success resp * fixes for PR feedback * check for data in patch handler prior to locking * pass storage view and key to cleanupOldVersions directly * remove replace directive for vault/sdk * get vault/sdk @kv-patch branch * add cas_required check to TestVersionedKV_Patch_CASValidation * cleanupOldVersions now returns string instead of error * remove direct dependency on json-patch * require the json-patch library * fix TestVersionedKV_Patch_NoData * upgrade to go 1.16 Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
* update vault/sdk dependencies * change a few deps to be direct
* add custom_metadata field to data ReadOperation handler * add custom_metadata field to data CreateOperation handler * test for custom_metadata in data read and write responses * fix some typos * add custom_metadata field to data PatchOperation handler * test for custom_metadata in data patch responses * update help description to include patch
* deps: update dependencies for vault 1.9 release * ci: remove working_directory config
* add PatchOperation to metadata endpoint
* add custom_metadata validation to patch handler
* return 404 in metadata patch handler when entry does not exist
* add cas_required warning in metadata patch handler
* run HandlePatchOperation for metadata patch handler
* metadata patch validation tests
* convert custom_metadata from TypeKVPairs to TypeMap
TypeKVPairs results in using a map[string]string
whereas TypeMap results in using a map[string]interface{}.
Being able to accept null values for custom_metadata
fields is important for HTTP PATCH operations as it
signals to the handler to remove the field. A shared
parser has been added to ensure that the provided
non-nil values are indeed parsable as strings.
* adding custom_metadata validation and patch tests
* go get vault/sdk@patch-field-data-error-handling
* fix delete_version_after handling for metadata patch
* go fmt
* add versions and custom_metadata checks
* check for simpler substr in metadata validation test
* go get vault/sdk@patch-field-data-error-handling
* fix custom metadata parsing
* tests for ignored unknown metadata fields
* move lock prior to metadata read
* add explicit check for cas_required in test despite warning
* add godoc for kv metadata patch
* remove filter of nils as sdk will do so
* add metadata test for nils unsetting values
* go get vault/sdk@main; go mod tidy
* move some test helper funcs to helper file * add subkeys endpoint and associated ReadOperation handler * fix max depth handling to prevent leaking data * move getBackend test helper to common test helpers * adding more subkeys tests * go fmt * return metadata in subkeys resp if secret is destroyed * adding more unit tests * add depth param to subkeys endpoint * update subkeys help description * cleaning up some comments * add IsValid checks to prevent panics * fix test error message * change getBackend test helper to wait for upgrade to finish * move test helper functions back * gofmt * fix test error logging so it cannot panic * fix typo
Signed-off-by: Lars Lehtonen <lars.lehtonen@gmail.com>
correct go.mod module
Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
Signed-off-by: Alexander Scheel <alexander.m.scheel@gmail.com>
This was referenced Feb 3, 2024
naphelps
approved these changes
Feb 5, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This imports the K/V secrets plugin as discussed in #64, bringing it in-tree with history.
This is an update of #30, which now has conflicts, preserving attribution to @alrs. :-)
This is part of #68, broken up to make review easier.