From 5acee3cc8ded8b93ccbf32de6a0d0554f49a5b5d Mon Sep 17 00:00:00 2001 From: Olly Levett Date: Fri, 13 Oct 2023 09:32:37 +0100 Subject: [PATCH] GPC-NONE: Update architecture summary (#2392) Update doc/architecture/architecture.md Co-authored-by: Ethan Mills --- doc/architecture/architecture.md | 22 ++++++++------------ doc/architecture/img/architecture.drawio.svg | 4 ++-- doc/architecture/img/event-flow.drawio.svg | 4 ---- 3 files changed, 11 insertions(+), 19 deletions(-) delete mode 100644 doc/architecture/img/event-flow.drawio.svg diff --git a/doc/architecture/architecture.md b/doc/architecture/architecture.md index db1a74a70..c941f0f6f 100644 --- a/doc/architecture/architecture.md +++ b/doc/architecture/architecture.md @@ -3,37 +3,33 @@ This is a summary of the ADRs across the service, trying to produce a single consolidated view of the as-is system. ## AWS/Infrastructure + A current state view of the overall system architecture and services used in AWS is documented below. -![](img/architecture.drawio.svg) +![Architecture Diagram](img/architecture.drawio.svg) ## Stubs/additional services We currently have a fake version of the supplier side (HMPO), running as -- a mock API to look up against in AWS AppRunner -- a mock event API calling the service in AWS Lambda -We also have a mock consumer running as -- a set of Step Functions calling the API +- a Lambda to fetch a file from the GRO SFTP Host +- Mapped using step functions into individual API calls into the CloudFront front end ## Load/Volumes We're currently working with a limited data set of notifications of death. Using ONS data, we've worked out what an "average" day looks like, and are targetting ~3,000 notifications a day. -We're currently working with one data acquirer, and one data supplier, as well as an internal mock acquirer, -so the total data volumes are relatively low currently. +We're currently working with one data acquirer, and one data supplier, so the total data volumes are relatively low currently. ## Event flow -Events flow through the system as shown in the diagram below. - -![flow.drawio.svg](img/event-flow.drawio.svg) -Each supplier event may result in several acquirer events as the system fans out. These are decoupled from each other early -in the process, and queues are used to provide resiliency. Each supplier event is given an ID, which can be linked to -audit information. +The data model and event process is described in [TechDocs](https://docs.life-events.dev.account.gov.uk/data-model.html). ## Data flow + The data flow is illustrated below. ![data-flow.drawio.svg](img/data-flow.drawio.svg) + +More detailed data retention policies can be found in our [Data Retention Policy](https://docs.life-events.dev.account.gov.uk/data-retention-policy.html). diff --git a/doc/architecture/img/architecture.drawio.svg b/doc/architecture/img/architecture.drawio.svg index 8cb058c10..cb442911f 100644 --- a/doc/architecture/img/architecture.drawio.svg +++ b/doc/architecture/img/architecture.drawio.svg @@ -1,4 +1,4 @@ - + -
AWS Cloud
AWS Cloud
eu-west-2
eu-west-2
Supplier (e.g. HMPO)
Suppl...
Acquirer (API)
Acqui...
Internal SQS Queue
Internal SQS...
RDS (Audit, events)
RDS (Audit, e...
GDX DataShare POC
GDX DataShare...

Load Balancer
Load Balanc...
CloudFront
CloudFront
Cognito (Client authentication)
Cognito (Clie...
Acquirer Queue
Acquirer Queue
Acquirer (SQS)
Acqui...
Parameter
Store
Parameter...
Logging, monitoring
Logging, moni...
Performance, metrics
Performance,...
Graphing
GraphingText is not SVG - cannot display \ No newline at end of file +
AWS Cloud
AWS Cloud
eu-west-2
eu-west-2
Supplier (e.g. HMPO)
Suppl...
CloudFront
CloudFront
Cognito (Client authentication)
Cognito (Clie...
Acquirer (SQS)
Acqui...
Logging, monitoring
Logging, moni...
Performance, metrics
Performance,...
Validation
Validation
Enrichment and mapping to SET
Enrichment an...
Enrichment Queue
Enrichment Qu...
Fan out
Fan out
Minimisation Queue
Minimisation...
Minimsation
Minimsation
Delivery
Delivery
DynaTrace
DynaTraceText is not SVG - cannot display \ No newline at end of file diff --git a/doc/architecture/img/event-flow.drawio.svg b/doc/architecture/img/event-flow.drawio.svg deleted file mode 100644 index b22ef6e1d..000000000 --- a/doc/architecture/img/event-flow.drawio.svg +++ /dev/null @@ -1,4 +0,0 @@ - - - -Platform
AWS Account
AWS Account
External event supplier (HMPO)
External event supplier (HMPO)
EventAcceptorService
EventAcceptorService
SupplierEventProcessor
SupplierEventProcessor
AcquirerEventQueue
AcquirerEventQueue
One per acquirer subscription
One per acquirer subscription
SupplierEvent
SupplierEvent
AcquirerEventQueue
AcquirerEventQueue
SupplierEventQueue
SupplierEventQueue
AWS Cloud
AWS CloudTemporary LEN
AcquirerEvent
AcquirerEvent
OutboundEventQueue
OutboundEventQueue
Inbound event API
Inbound event API
AcquirerEventProcessor
AcquirerEventProcessor
metadata
metadata
metadata
metadata
metadata plus hashed
payload
metadata plus hashed...
AcquirerEventAudit
AcquirerEventAudit
GRO data bucket
GRO data bucket
Callback API
Callback API
Notification
Notification
AWS Account
AWS Account
External event acquirer (DWP)
External event acquirer (DWP)
Sensitive data flows outside core of the system
Sensitive...Text is not SVG - cannot display \ No newline at end of file