docs: add secret management documentation [skip-ci] #542

yshyn-iohk · 2023-06-02T16:23:52Z

Overview

Documentation for secret management using Vault service

The following aspects are covered (clickable):

Fixes ATL-4173 and ATL-4876

Checklist

My PR contains...

No code changes (changes to documentation, CI, metadata, etc.)
Bug fixes (non-breaking change which fixes an issue)
Improvements (misc. changes to existing features)
Features (non-breaking change which adds functionality)

My changes...

are breaking changes
are not breaking changes
If yes to above: I have updated the documentation accordingly

Documentation

My changes do not require a change to the project documentation
My changes require a change to the project documentation
If yes to above: I have updated the documentation accordingly

Tests

My changes can not or do not need to be tested
My changes can and should be tested by unit and/or integration tests
If yes to above: I have added tests to cover my changes
If yes to above: I have taken care to cover edge cases in my tests

# Conflicts: # docs/decisions/20230518-data-isolation-for-multitenancy.md

github-actions · 2023-06-02T16:28:39Z

Atala PRISM Test Results

  52 files ±0   52 suites ±0 7m 31s ⏱️ -19s
412 tests ±0 412 ✔️ ±0 0 💤 ±0 0 ❌ ±0
413 runs ±0 413 ✔️ ±0 0 💤 ±0 0 ❌ ±0

Results for commit a3d7503. ± Comparison against base commit ee43feb.

♻️ This comment has been updated with latest results.

github-actions · 2023-06-02T16:28:42Z

Atala PRISM Code Coverage

File	Coverage
All files	`22%`	✅

Minimum allowed coverage is 12%

Generated by 🐒 cobertura-action against a3d7503

github-actions · 2023-06-02T16:41:37Z

E2E Test Results

11 files ±0 11 suites ±0 5m 36s ⏱️ -12s
20 tests ±0 20 ✔️ ±0 0 💤 ±0 0 ❌ ±0
31 runs ±0 31 ✔️ ±0 0 💤 ±0 0 ❌ ±0

Results for commit a3d7503. ± Comparison against base commit ee43feb.

♻️ This comment has been updated with latest results.

antonbaliasnikov · 2023-06-02T23:26:54Z

@yshyn-iohk , don't forget to update sidebar to include this correctly into the website.

yshyn-iohk · 2023-06-03T06:11:37Z

@antonbaliasnikov will do.
this docs are draft version, these will be corrected next week.

patlo-iog

Thanks Yurii. Looking good overall. I left comments below

docs/docusaurus/secrets/sequence-diagrams.md

docs/docusaurus/secrets/secret-storage.md

docs/docusaurus/secrets/account-management.md

patlo-iog · 2023-06-06T08:46:59Z

docs/docusaurus/secrets/secret-storage.md

+
+### PRISM Agent Logical Isolation
+Each instance of the PRISM agent needs to have access to the secrets but must be isolated from other agents at the same environment.
+Vault namespaces are used to achieve this goal, so each agent has its own namespace and can access only its own secrets.


Doesn't the namespace require Vault Enterprise? 🤔

namespaces look like its indeed Enterprise. Perhaps adding a policy limiting tenant-id/* for each agent?

@womfoo is right. My bad, I will correct the documentation.
https://developer.hashicorp.com/vault/tutorials/enterprise/namespaces

docs/docusaurus/secrets/secret-storage.md

EzequielPostan · 2023-06-08T02:52:38Z

docs/docusaurus/secrets/account-management.md

+- Wallet Account - application account used by the Wallet to authenticate itself to the Vault service
+- Tenant Account - user account used by the user to authenticate mobile or WEB application to the PRISM Platform.


I think I need a bit more context

is the wallet referring to a separate application than the agent? is it referring to edge wallets?

for the mobile/web apps, are they consumers/clients of the agent? is the PRISM platform something different than the agent API?

The Wallet refers to the logical entity inside the Agent.
Currently, the Prism Agent supports single-tenant mode only and instance-par-tenant logical and physical isolation.
In the multi-tenant mode, a single Agent can manage multiple tenants.
Each tenant must have a wallet with the assets.

So, the wallet in this documentation refers to the managed wallet. All secret materials and assets are stored in the cloud, and the cloud agent can access the wallet and all its assets. The edge wallet refers to the web, mobile, or desktop application. We don't support this now.

Currently, web/mobile applications are clients of the agent.

I think, it makes sense to remove the term Tennant Account
The Wallet account is clear enough for SSI domain.

docs/docusaurus/secrets/access-control.md

docs/docusaurus/secrets/account-management.md

bvoiturier · 2023-06-09T13:10:52Z

docs/docusaurus/secrets/secret-storage.md

+There are following types of the secrets in the PRISM Platform:
+- seed: a secret used to derive cryptographic keys
+- private key: a secret used to sign data
+- public key: a secret used to verify signatures


Is this the place where AnonCreds Link Secrets would be stored as well?

and it's also a place where we can store the private part of the credential definintion

docs/docusaurus/secrets/operation.md

docs/docusaurus/secrets/account-management.md

womfoo · 2023-06-15T07:52:45Z

docs/docusaurus/secrets/secret-storage.md

+In the single-tenant configuration, the PRISM Agent uses a single Vault namespace, a single Wallet and a single Vault account to authenticate itself to the Vault service.
+In the multi-tenant configuration, the PRISM Agent manages multiple Wallets, each Wallet is associated with a single tenant.
+Multi-tenant configuration is used to achieve logical data separation between tenants, so each Wallet can access only its own secrets.
+The Wallet is identified by the tenant ID and represented by the account in the Vault service.


Might be out-of-scope for this change, but where is the source of truth for tenants? Will this be somewhere in a env var/config file/database table etc?

I will remove the User and Tenant accounts and leave the Wallet account only.
The Wallet account will be configured in the Vault dynamically

docs/docusaurus/secrets/secret-storage.md

patlo-iog · 2023-06-16T10:37:37Z

docs/docusaurus/secrets/secret-storage.md

+The keys material of the DID peer can be stored by the following path:
+
+```mermaid
+<tenant-id>/dids/peer/<did-ref>/keys/<key-purpose> value=<base64-encoded-value> <metadata>


Ah, the dids/peer is slightly different from the merged PR. I guess we'll have to align this during multi-tenancy

…repo subject to the Developer Certificate of Origin (DCO), Version 1.1. 81a2ad9 docs: removed secret-storage from the Tutorials section. (#559) [skip-ci] 130abbd docs: fix secret storage documentation page (#556) 4f413a3 docs: add secret management documentation [skip-ci] (#542) 885870d Fix ATL-4830 verification policies update (#549) f6d1fd4 docs: compose the ADR for DID-linked resources. ATL-3186 [skip-ci] (#518) dc54af8 docs: ADR for data isolation for multi-tenancy (#531) 357273d docs: ADR for HD key derivation in the PRISM v2 (#529) 64b0a2a fix(prism-agent): Alight the error responses according to RFC7807. ATL-3962 (#480) 7fa2e1a fix(prism-agent) Align VerificationPolicy OAS ATL-3909 (#473) 8d42fb1 fix: update mercury to 0.21.0 979b609 chore: update mercury dependency to 0.21.0 (#458) d3a8d15 feat(prism-agent) update schema logic - agent part. ATL-3164 (#452) 6e22bfc feat(pollux): update credential schema logic (#450) ATL-3164 68b8e4a docs: credential schema introduction, creation, update, and delete. ATL-3548 (#443) 91902ce feat(prism-agent): Add OAS specification to the schema registry endpoint. ATL-3164 (#438) 32f9e83 feat(prism-agent): CredentialSchema DAL, model, service and repositor… (#425) 79352f0 feat(pollux): CredentialSchema DAL, model, service and repository #2 (#424) 6e941da docs(pollux): add credential-schema.md to docusaurus (#407) ffa5f7e feat(pollux): CredentialSchema service, repository and sql (#416) c96f804 doc(pollux): verification policy documentation (#384) 1e47ba1 doc(pollux): add schema registry documentation. ATL-1334 (#296) 142ff55 feat(prism-agent): integrate VerificationPolicy DAL into the agent, update OAS and implement REST API for verification policies (#369) b290a18 feat(pollux): implement the DAL for CRUD on the verifiable policy entity. ATL-2478 (#368) edaab33 docs: ADR Quill library for sql statement generation (#318) 3d0c642 feat(prism-agent): implement DAL for the credential schema. ATL-1334 e0831e8 feat(pollux): fix the lookup count in the credential schema DAL (#315) f43320f feat(pollux): add dal for the credential schema ATL-1342 (#298) 6903afa fix(prism-agent): switch datetime format to offsetdatetime. ATL-2723 (#243) e13a1bd fix(connect): bump mercury version to 1.10.1 and touch README.md ee27755 fix(pollux): upgrade mercury lib to 1.10.1 cdd4772 fix(castor): README.md is added to increase the version of the castor library 5ffb0cc fix(mercury): simple commit to increase the version of mercury library 403eb38 feat(prism-agent): verification policies pagination. ATL-1334 (#205) 726e2d9 feat(prism-agent): implement pagination with navigation for schema-registry (#195) 4528c57 feat(pollux): alight the OAS for schema registry (#189) 16d5fdb feat(pollux): cleanup the code of IssueCredentialApi 79170f8 feat(pollux): cleanup the OAS from Issue Credentials and other unused tags d75b36b feat(pollux): schema registry lookup and verification policies REST API ATL-1334 (#168) b3cf828 feat(apollo): add schema registry to the agent using Tapir library. ATL-1334 (#94) 8d8bf56 doc(adr): use Tapir library as a DLS for OAS (#51) 496337b ATL-1334 feat(pollux): add sandbox project to play with Tapir and schema-registry (#62) 95667ba doc(pollux): add JWT encoding and Present Proof endpoints (#37) 234bc06 [ATL-1388] doc(pollux): add Revocation Registry API, remove v1 from the path, apply changes after review (#14) e47242c [ATL-1001] doc: add examples of OpenAPI specification of well-known competitors (#23) 22e37cf doc(pollux): add Pollux open-api specification (schema and issue-credentials) (#8) Signed-off-by: Yurii Shynbuiev <yurii.shynbuiev@iohk.io>

yshyn-iohk added 7 commits May 22, 2023 17:13

docs: ADR for data isolation for multi-tenancy

c2e44ad

Merge remote-tracking branch 'origin/main' into main2

51d80ba

Merge remote-tracking branch 'origin/main' into main2

dfd66fb

# Conflicts: # docs/decisions/20230518-data-isolation-for-multitenancy.md

Merge remote-tracking branch 'origin/main' into main2

cfbf2a2

Merge remote-tracking branch 'origin/main' into main2

549dba1

Merge remote-tracking branch 'origin/main' into main2

e37fbf8

docs: documentation for secret management using Vault service [skip-ci]

09528b9

yshyn-iohk requested a review from antonbaliasnikov as a code owner June 2, 2023 16:23

yshyn-iohk requested a review from a user June 2, 2023 16:23

yshyn-iohk requested a review from BS-IO as a code owner June 2, 2023 16:23

atala-dev added the Type: Documentation label Jun 2, 2023

yshyn-iohk requested review from bvoiturier, mineme0110 and patlo-iog June 5, 2023 11:14

patlo-iog reviewed Jun 6, 2023

View reviewed changes

EzequielPostan reviewed Jun 8, 2023

View reviewed changes

docs/docusaurus/secrets/secret-storage.md Outdated Show resolved Hide resolved

EzequielPostan reviewed Jun 8, 2023

View reviewed changes

bvoiturier reviewed Jun 9, 2023

View reviewed changes

patlo-iog force-pushed the ATL-4173-secret-management-documentation branch from 6db48ec to 5945798 Compare June 14, 2023 13:44

docs: add doc on agent config guide [skip ci]

4bddef3

patlo-iog force-pushed the ATL-4173-secret-management-documentation branch from 5945798 to 4bddef3 Compare June 14, 2023 13:47

patlo-iog added 3 commits June 14, 2023 20:49

docs: fix type [skip ci]

6354446

docs: modify sidebar.js

f99f57b

docs: minor refinement [skip ci]

294b82e

womfoo reviewed Jun 15, 2023

View reviewed changes

docs/docusaurus/secrets/operation.md Outdated Show resolved Hide resolved

womfoo reviewed Jun 15, 2023

View reviewed changes

docs/docusaurus/secrets/account-management.md Outdated Show resolved Hide resolved

womfoo reviewed Jun 15, 2023

View reviewed changes

mkbreuningIOHK requested a review from lohanspies June 15, 2023 08:03

patlo-iog and others added 2 commits June 15, 2023 15:44

docs: update default DEV_MODE doc

19c2f4e

docs: update documentation for secret storage

ae75a0d

patlo-iog approved these changes Jun 16, 2023

View reviewed changes

patlo-iog changed the title ~~Atl 4173 secret management documentation~~ docs: add secret management documentation Jun 16, 2023

Merge branch 'main' into ATL-4173-secret-management-documentation

a3d7503

yshyn-iohk changed the title ~~docs: add secret management documentation~~ docs: add secret management documentation [skip-ci] Jun 16, 2023

yshyn-iohk merged commit 4f413a3 into main Jun 16, 2023
10 checks passed

yshyn-iohk deleted the ATL-4173-secret-management-documentation branch June 16, 2023 11:12

yshyn-iohk mentioned this pull request Aug 18, 2023

I, Yurii Shynbuiev hereby sign-off-by all of my past commits to this … #634

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: add secret management documentation [skip-ci] #542

docs: add secret management documentation [skip-ci] #542

yshyn-iohk commented Jun 2, 2023 •

edited by mkbreuningIOHK

github-actions bot commented Jun 2, 2023 •

edited

github-actions bot commented Jun 2, 2023 •

edited

github-actions bot commented Jun 2, 2023 •

edited

antonbaliasnikov commented Jun 2, 2023

yshyn-iohk commented Jun 3, 2023

patlo-iog left a comment

patlo-iog Jun 6, 2023

womfoo Jun 15, 2023

yshyn-iohk Jun 15, 2023

EzequielPostan Jun 8, 2023

yshyn-iohk Jun 8, 2023

yshyn-iohk Jun 15, 2023

bvoiturier Jun 9, 2023

yshyn-iohk Jun 15, 2023

womfoo Jun 15, 2023

yshyn-iohk Jun 15, 2023

patlo-iog Jun 16, 2023

		- Wallet Account - application account used by the Wallet to authenticate itself to the Vault service
		- Tenant Account - user account used by the user to authenticate mobile or WEB application to the PRISM Platform.

docs: add secret management documentation [skip-ci] #542

docs: add secret management documentation [skip-ci] #542

Conversation

yshyn-iohk commented Jun 2, 2023 • edited by mkbreuningIOHK

Overview

Checklist

My PR contains...

My changes...

Documentation

Tests

github-actions bot commented Jun 2, 2023 • edited

Atala PRISM Test Results

github-actions bot commented Jun 2, 2023 • edited

github-actions bot commented Jun 2, 2023 • edited

E2E Test Results

antonbaliasnikov commented Jun 2, 2023

yshyn-iohk commented Jun 3, 2023

patlo-iog left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yshyn-iohk commented Jun 2, 2023 •

edited by mkbreuningIOHK

github-actions bot commented Jun 2, 2023 •

edited

github-actions bot commented Jun 2, 2023 •

edited

github-actions bot commented Jun 2, 2023 •

edited