Secrets Sync #23667

zofskeez · 2023-10-16T15:49:34Z

This PR adds support for secrets syncing in the UI. The new sync enterprise feature enables users to sync KVv2 secrets in Vault to external destinations such as AWS Secrets Manager, Azure Key Vault and more.

Screen.Recording.2023-12-12.at.8.50.55.PM.mov

* ember engine setup for secrets sync * Update ui/lib/sync/addon/routes.js Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com> --------- Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>

* adds mirage setup for sync endpoints * updates secret_name default in sync-association mirage factory

* add models * adapters * base model adapter * update test response * add sync destinations helper * finish renaming base destination model/adapter * add comment * add serializer * use normalizeItems instead * destination serializer test * add destination find method; * add conditional operand

* add models * adapters * base model adapter * update test response * add sync destinations helper * finish renaming base destination model/adapter * add comment * add serializer * doc-link helper * add version service * landing and overview component * overview page * add tests

* add models * adapters * base model adapter * update test response * add sync destinations helper * finish renaming base destination model/adapter * add comment * add serializer * doc-link helper * add version service * landing and overview component * overview page * build out serializer and adapters * update mirage * fix merge conflicts * one more conflict! * pull transformQueryResponse to separate method in adapter * move data transforming all to serializer and tests * add note to paginationd ocs docs * conditionally render CTA * add lazyPaginatedQuery method to destinations route * remove partial error

* add category to destinations * build select type page * refactor prompt config situation * routing for destinations * update select-type routing * make card width fixed * revert CTA routing change, keep shouldRenderOverview * add header for gif demo to form * cleanup scope * more scope cleanup * add test * add type selector * rename components * rename again * remove async * fix tests * fix select type rename in test * delete renamed test * fix import of general selectors * rename using component syntax

* add model attribute metadata * add form and save url, remove name and type from serializer * move checkbox list to form field helper * add styling to alert inline * use newly made class * fix cancel action and cleanup form * change quotes * remove checkbox action from form component * add tests * address feedback * add API error test * use create record method instead * adapter test for create record * return from find method if type is undefined

* refactor getter in base destination model * add getters back to model

* change labels to match params * add maskedParams to base model * add details route * add details view; * update mirage * fix secrets sync link; * delete parent destination route * add copyright header * add secrets route * move sync route outside of secrets/ route * upate mirage * export to-label * finish tests * make ternary * rename header tabs * fix selector in test

* remove destination header component, add headers/tabs to all routes * fix header padding * move tabs + toolbar back into component... * add copyright header * add delete modal * lol revert again * add extra line after copyright header

* adds route and page component for sync destinations list view * filters by type first for sync destinations * adds test for store.filterData method * Update ui/app/services/store.js Co-authored-by: Kianna <30884335+kiannaquach@users.noreply.github.com> * updates nav link label for secrets sync * moves sync destinations types out of app-types * moves loading-dropdown-option component to core addon and adds to destination list item menu * change true assertion to deepEqual in sync destinations test * adds copyright header to sync-destinations type file * clear store dataset on sync destination create --------- Co-authored-by: Kianna <30884335+kiannaquach@users.noreply.github.com>

* adds route and page component for sync destinations list view * filters by type first for sync destinations * adds test for store.filterData method * adds capabilities checks for sync destinations * removes canList from sync destinations capabilities * updates sync header tests * Update ui/tests/integration/components/sync/sync-header-test.js Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com> * updates sync destination response serialization * updates sync destination serializer test * updates sync destinations page test assertions * fixes mirage sync destinations payload issue * removes commented out method in sync destination adapter * fixes inconsistencies with url generation for sync destinations delete * fixes sync destinations page test --------- Co-authored-by: claire bontempo <68122737+hellobontempo@users.noreply.github.com>

* adds model, adapter and serializer for sync associations * updates sync association adapter save methods to use adapterOptions to determine action

* renames sync destination header component and adds tests * adds destination secrets route and page component * adds setup-models helper for sync testing * moves destination details test into subdir * adds destination secrets page component tests * adds controller for destination secrets route * fixes pagination route on destination secrets view * fixes sync association updated_at assertion based on timezone * updates kv secret details external route name

github-actions · 2023-11-16T22:14:24Z

Build Results:
All builds succeeded! ✅

* add error substate * add loading substates * delete loading from secrets route

* removes is-version helper and injects service into components * updates sync tests using version service to new API * adds comment back for tracked property in secret detials page component

…lector

…t destroy (#24450)

* updates filter-input component to conditionally show search icon * adds kv-suggestion-input component to core addon * updates destination sync page component to use KvSuggestionInput component * fixes issue in kv-suggestion-input where a partial search term was not replaced with the selected suggestion value * updates kv-suggestion-input to retain focus on suggestion click * fixes test * updates kv-suggestion-input to conditionally render label component * adds comments to kv-suggestion-input regarding trigger * moves alert banner in sync page below button set * moves inputId from getter to class property on kv-suggestion-input

* add form field groups to sync models * update create-and-edit form to use confirmLeave and enableInput component * enable input component * add more stars * update css comments * Update ui/app/styles/helper-classes/flexbox-and-grid.scss * make attrOptions optional * remove decorator * add env variables to subtexr * add subtext to textfile * fix overviwe transition bug * remove breadcrumbs to getter * WIP adapter update * update mirage response * add update method with PATCH * add patch to application adapter * fix typo * finish tests * remove validations because could use environment variables * use getter and setter in model * move update record business to serializer * rest of logic in serializer; gp ; gp * add model validation warnings * cleanup getters * pull create/update logic into method for mirage * add test for validation warning * update KV copy

* adds success banner to destination sync page * move submit disabled logic to getter in destination sync page * adds id and for attributes to kv mount input in sync page * hides sync success banner on submit

* use Sync secrets everywhere (remove new) * revert test name change

* fixes issues filtering destinations list * adds test

…ult into ui/VAULT-17968/secrets-sync

hashishaw

A couple minor nits and a11y change requests, but otherwise looks great!

hashishaw · 2023-12-13T17:43:51Z

ui/app/models/sync/destinations/aws-sm.js

+  accessKeyId; // obfuscated, never returned by API
+
+  @attr('string', {
+    label: 'Secret access key',


Should this and accessKeyId be masked inputs?

They weren't in the designs 🤔 But this is a good question - we have a meeting scheduled to do a design review/walk through in the new year. I'll make a note to ask this!

hashishaw · 2023-12-13T17:45:16Z

ui/app/models/sync/destinations/azure-kv.js

+    subText:
+      'Client secret of an Azure app registration. If empty, Vault will use the AZURE_CLIENT_SECRET environment variable if configured.',
+  })
+  clientSecret; // obfuscated, never returned by API


Same with this. Really any secrets should be masked, I would think.

We don't mask them for OIDC config...but maybe that was built before we used that component

hashishaw · 2023-12-13T17:45:28Z

ui/app/models/sync/destinations/gcp-sm.js

+  })
+  credentials; // obfuscated, never returned by API
+
+  // TODO - confirm if project_id is going to be added to READ response (not editable)


This is still a TODO 😞 - waiting for manually testing creds to confirm this

hashishaw · 2023-12-13T17:54:07Z

ui/lib/sync/addon/components/secrets/page/overview.hbs

+  <div class="tabs-container box is-bottomless is-marginless is-paddingless">
+    <nav class="tabs" aria-label="destination tabs">
+      <ul>
+        <LinkTo @route="secrets.overview" data-test-tab="Overview">Overview</LinkTo>


These need to be wrapped in <li>

Suggested change

<LinkTo @route="secrets.overview" data-test-tab="Overview">Overview</LinkTo>

<li><LinkTo @route="secrets.overview" data-test-tab="Overview">Overview</LinkTo></li>

good catch!

hashishaw · 2023-12-13T17:54:19Z

ui/lib/sync/addon/components/secrets/page/overview.hbs

+    <nav class="tabs" aria-label="destination tabs">
+      <ul>
+        <LinkTo @route="secrets.overview" data-test-tab="Overview">Overview</LinkTo>
+        <LinkTo @route="secrets.destinations" data-test-tab="Destinations">Destinations</LinkTo>


Suggested change

<LinkTo @route="secrets.destinations" data-test-tab="Destinations">Destinations</LinkTo>

<li><LinkTo @route="secrets.destinations" data-test-tab="Destinations">Destinations</LinkTo></li>

hashishaw · 2023-12-13T17:54:33Z

ui/lib/sync/addon/components/secrets/page/overview.hbs

+      </ul>
+    </nav>
+  </div>
+  <Toolbar>


This, ToolbarActions, and ToolbarLink needs aria-label

hashishaw · 2023-12-13T17:57:31Z

ui/tests/helpers/general-selectors.js

zofskeez added ui pr/no-changelog labels Oct 16, 2023

zofskeez added this to the 1.16.0-rc1 milestone Oct 16, 2023

github-actions bot added the hashicorp-contributed-pr If the PR is HashiCorp (i.e. not-community) contributed label Oct 16, 2023

zofskeez and others added 2 commits October 16, 2023 16:26

Sync Mirage Setup (#23683)

8338397

* adds mirage setup for sync endpoints * updates secret_name default in sync-association mirage factory

Merge branch 'main' into ui/VAULT-17968/secrets-sync

b0ca805

vercel bot deployed to Preview October 17, 2023 16:02 View deployment

hellobontempo and others added 4 commits October 17, 2023 10:27

Merge branch 'main' into ui/VAULT-17968/secrets-sync

32d2868

vercel bot deployed to Preview October 20, 2023 22:50 View deployment

hellobontempo and others added 5 commits October 23, 2023 14:32

cleanup test selectors

7884a13

secrets sync: refactor sync destinations helper (#23839)

f92c811

* refactor getter in base destination model * add getters back to model

Merge branch 'main' into ui/VAULT-17968/secrets-sync

b437785

vercel bot deployed to Preview October 26, 2023 22:34 View deployment

hellobontempo and others added 3 commits October 26, 2023 15:41

Secrets Sync UI: Cleanup headers + tabs (#23873)

ed05f84

* remove destination header component, add headers/tabs to all routes * fix header padding * move tabs + toolbar back into component... * add copyright header * add delete modal * lol revert again * add extra line after copyright header

Merge branch 'main' into ui/VAULT-17968/secrets-sync

4f8e1a9

vercel bot deployed to Preview October 31, 2023 01:41 View deployment

zofskeez and others added 5 commits November 2, 2023 15:15

Sync Associations Ember Data Setup (#24132)

3a948bd

* adds model, adapter and serializer for sync associations * updates sync association adapter save methods to use adapterOptions to determine action

Merge branch 'main' into ui/VAULT-17968/secrets-sync

681d540

vercel bot deployed to Preview November 16, 2023 22:17 View deployment

Merge branch 'main' into ui/VAULT-17968/secrets-sync

b37e040

zofskeez force-pushed the ui/VAULT-17968/secrets-sync branch from 2c92663 to b37e040 Compare December 5, 2023 23:10

vercel bot deployed to Preview December 5, 2023 23:17 View deployment

hellobontempo and others added 6 commits December 5, 2023 18:20

Secrets Sync UI: Add loading and error substates (#24353)

7e81adb

* add error substate * add loading substates * delete loading from secrets route

Remove is-version Helper (#24388)

39744bb

* removes is-version helper and injects service into components * updates sync tests using version service to new API * adds comment back for tracked property in secret detials page component

updates sync tests to use common selectors (#24397)

7eb7dff

Merge branch 'main' into ui/VAULT-17968/secrets-sync

d9e7003

update capitalization to consistently be titlecase, fix breadcrumb se…

a13c780

…lector

Merge branch 'main' into ui/VAULT-17968/secrets-sync

d3bf747

vercel bot deployed to Preview December 7, 2023 23:32 View deployment

zofskeez and others added 12 commits December 11, 2023 13:54

clears sync associations from store on destination sync page componen…

4d7a3cf

…t destroy (#24450)

Sync Success Banner (#24491)

5494d60

* adds success banner to destination sync page * move submit disabled logic to getter in destination sync page * adds id and for attributes to kv mount input in sync page * hides sync success banner on submit

use Sync secrets everywhere (remove new) (#24494)

b0fd407

* use Sync secrets everywhere (remove new) * revert test name change

Sync Destinations List Filter Bug (#24496)

b8c1d9d

* fixes issues filtering destinations list * adds test

fixes Sync now action text alignment in destination secrets list

4b712b7

UI Secrets sync: Add purge query param to delete endpoint (#24497)

5f98b44

adds updated_at to mirage set association handler

341a9c1

adds changelog entry

e4b364f

Merge branch 'ui/VAULT-17968/secrets-sync' of github.com:hashicorp/va…

9c6d28e

…ult into ui/VAULT-17968/secrets-sync

Merge branch 'main' into ui/VAULT-17968/secrets-sync

a181036

zofskeez marked this pull request as ready for review December 13, 2023 04:06

zofskeez removed the pr/no-changelog label Dec 13, 2023

vercel bot deployed to Preview December 13, 2023 04:11 View deployment

add enterprise in parenthesis for changelog

c1e8f76

hashishaw approved these changes Dec 13, 2023

View reviewed changes

addres a11y feedback

aaf532b

hellobontempo merged commit 99445db into main Dec 13, 2023
71 checks passed

hellobontempo deleted the ui/VAULT-17968/secrets-sync branch December 13, 2023 19:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Secrets Sync #23667

Secrets Sync #23667

zofskeez commented Oct 16, 2023 •

edited

github-actions bot commented Nov 16, 2023

hashishaw left a comment

hashishaw Dec 13, 2023

hellobontempo Dec 13, 2023

hashishaw Dec 13, 2023

hellobontempo Dec 13, 2023

hashishaw Dec 13, 2023

hellobontempo Dec 13, 2023

hashishaw Dec 13, 2023

hellobontempo Dec 13, 2023

hashishaw Dec 13, 2023

hashishaw Dec 13, 2023

hashishaw Dec 13, 2023

	<LinkTo @route="secrets.overview" data-test-tab="Overview">Overview</LinkTo>
	<li><LinkTo @route="secrets.overview" data-test-tab="Overview">Overview</LinkTo></li>

	<LinkTo @route="secrets.destinations" data-test-tab="Destinations">Destinations</LinkTo>
	<li><LinkTo @route="secrets.destinations" data-test-tab="Destinations">Destinations</LinkTo></li>

Secrets Sync #23667

Secrets Sync #23667

Conversation

zofskeez commented Oct 16, 2023 • edited

github-actions bot commented Nov 16, 2023

hashishaw left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zofskeez commented Oct 16, 2023 •

edited