Architecture Overview

Architecture Overview

phoss-ap is an open-source Peppol Access Point built on phase4 and Spring Boot.

Key Components

Component	Technology	Purpose
AS4 Engine	phase4	AS4 message sending and receiving
Database	PostgreSQL or MySQL	Persistent storage of transaction metadata, file path references, and reporting data
API	Spring Boot REST	Inbound API for triggering sends and reporting
Document Storage	IDocumentPayloadManager	Reading, writing, and deleting document files (filesystem or S3, selected via storage.mode)
Verification	Pluggable SPI (optional)	Validation of outbound and inbound documents before processing
Forwarding	IDocumentForwarder (forwarding.mode)	Delivering received documents to the Receiver Backend
Retry	Customizable Java interfaces	Automatic retry of failed sends and failed forwarding
Notifications	Generic Java interface	Alerting on permanent failures
Reporting	peppol-reporting	Peppol TSR/EUSR generation

Peppol 4-Corner Model Context

C1 (Sender) --> C2 (Sender AP) --> C3 (Receiver AP) --> C4 (Receiver)

phoss-ap operates exclusively on the C2-to-C3 boundary. It does not implement C1 or C4 functionality. The systems that interact with the AP on either side are:

• Sender Backend: An internal system within C2 that submits documents to the AP for outbound transmission.
• Receiver Backend: An internal system within C3 that receives documents from the AP after inbound reception.

[Sender Backend (C2)] --> [phoss-ap (C2)] --AS4--> [Remote AP (C3)]
[Remote AP (C2)] --AS4--> [phoss-ap (C3)] --> [Receiver Backend (C3)]

Sending Input Modes

The AP accepts outbound documents from the Sender Backend via its API in three forms:

1. Raw business document + metadata: The AP creates the SBDH envelope. The Sender Backend provides sender ID, receiver ID, document type ID, process ID, and C1 country code as API parameters.
2. Pre-built SBD: The Sender Backend provides a complete Standard Business Document (with SBDH). All metadata (including C1 country code) is extracted from the SBDH.
3. S3 reference + metadata: The Sender Backend uploads the document to an S3 bucket and calls the AP with a JSON body containing Peppol identifiers and the S3 bucket/key reference. The AP fetches the document from S3 and processes it like a raw document submission. Requires outbound.s3.enabled=true.

Receiving Forwarding Modes

Received documents are forwarded to the Receiver Backend. The forwarding mode is selected at runtime via the forwarding.mode configuration property. All implementations are included in a single phoss-ap-forwarding module. Valid modes:

1. HTTP POST (async reporting): Document is POSTed to the Receiver Backend. The Receiver Backend calls the reporting API asynchronously later.
2. HTTP POST (sync reporting): Document is POSTed to the Receiver Backend. The Receiver Backend responds synchronously with the C4 country code, and the AP triggers reporting immediately.
3. S3 + link forwarding: Document is stored in S3, and a link/reference is forwarded to the Receiver Backend. Reporting is asynchronous.
4. SFTP: Document is uploaded via SFTP. Reporting is asynchronous.

Receiver Verification

Before processing an inbound document, the AP verifies that it actually services the addressed receiver participant for the given document type and process. This check is implemented via a pluggable SPI (IReceiverCheck or similar) defined in phoss-ap-api.

Possible implementations include:

• Querying the AP's own SMP registration
• Checking a local configuration or database

If the receiver is not serviced, the AP returns an AS4 error with code EBMS:0004 / PEPPOL:NOT_SERVICED to the sending AP (C2). No database entry is created for rejected messages.

This check occurs before database storage and document verification, ensuring that unserviced messages are rejected as early as possible with the correct Peppol-defined error code.

Document Verification

Both outbound and inbound documents can optionally be verified before further processing. Verification is implemented via a generic plugin-like SPI (IDocumentVerifier or similar) defined in phoss-ap-api.

• Outbound: After storing the document to disk and recording its path in the database, the verification plugin is invoked before AS4 sending. If verification fails, the transaction is marked as rejected, a notification is triggered, and no sending attempt is made.
• Inbound: After storing the received document to disk and recording its path in the database, the verification plugin is invoked before forwarding to the Receiver Backend. If verification fails, the transaction is marked as rejected, a notification is triggered, an MLS response with code RE (Rejection) with business rule violation issues is sent back to C2, and no forwarding attempt is made.

The verification step is optional — if no plugin is configured, processing continues without verification. Multiple verification plugins can be chained (all must pass).

Built-in Validation Service Integration

The phoss-ap-validation module provides a built-in verification implementation that calls the "phorm Validation Service". It uses the /api/dd_and_validate/ endpoint which auto-detects the document type and validates it against the appropriate Peppol BIS rules. The validation response is parsed via PhiveJsonHelper.getAsValidationResultList from the phive library. If the overall conformance is INVALID, the document is rejected.

This implementation is shared for both inbound and outbound verification — both SPI interfaces are implemented by the same class (ValidationServiceDocumentVerifier).

Duplicate Detection

Duplicate detection operates on two independent levels:

1. AS4 Message ID: Detects retransmission of the exact same AS4 message.
2. SBDH Instance Identifier: Detects the same Peppol business envelope sent again (potentially with a different AS4 Message ID).

Each level is tracked independently in the database. The behavior on detection is configurable:

• Reject: Duplicate messages are rejected at the AS4 level.
• Store and flag: Duplicates are stored in the DB but flagged per level.

Peppol Message Level Status (MLS)

MLS is an asynchronous response mechanism defined by Peppol that allows the receiving AP (C3) to inform the sending AP (C2) about the processing outcome of a received document.

MLS Response Codes

Code	Meaning	When sent
RE	Rejection	Document failed verification (status reason BV) or permanent forwarding failure to C4 (status reason FD)
AP	Approved	Successful delivery with confirmation — used when forwarding mode is HTTP (http_post_sync, http_post_async)
AB	Accepted Blind	Document forwarded without delivery confirmation — used when forwarding mode is SFTP or S3

MLS Flow

1. C2 sends a business document to C3 via AS4.
2. C3 receives the document, forwards it to the Receiver Backend, and determines the outcome.
3. C3 generates an MLS response and sends it back to C2 as a separate AS4 message.
4. C2 receives the MLS and correlates it with the original outbound transaction via the SBDH Instance Identifier.

MLS messages are themselves regular Peppol documents sent via AS4, so they reuse the existing outbound transaction infrastructure (including retry logic).

MLS Configuration

• MLS_TO: An alternative Peppol participant ID where C3 should send the MLS response. Configurable per outbound transaction (set by the Sender Backend via the API). Useful when C2 operates multiple access points.
• MLS_TYPE: Controls when MLS is sent — FAILURE_ONLY (only on rejection) or ALWAYS_SEND (for all outcomes). Configurable per AP instance. The active value is captured per inbound transaction at reception time.

MLS and the AP

• As C2 (sending): The AP receives MLS responses asynchronously via AS4, correlates them with outbound transactions, and updates the MLS status.
• As C3 (receiving): After determining the forwarding outcome, the AP generates and sends an MLS response back to C2. The MLS sending is tracked as an outbound_transaction with transaction_type = mls_response.

MLS Timing and SLA Measurement

The Peppol Network Policy defines three MLS milestones based on AS4 MessageInfo/Timestamp values (all UTC):

Milestone	Definition	Where captured
M1	AS4 Timestamp when the original business document transmission is initiated at C2	outbound_sending_attempt.as4_timestamp (outbound) or inbound_transaction.as4_timestamp (inbound)
M2	AS4 Timestamp when the MLS message transmission is initiated at C3	outbound_sending_attempt.as4_timestamp on the MLS response's sending attempt
M3	Date/time when the MLS message is received at C2	outbound_transaction.mls_received_dt

Service Level Requirements (per Peppol Network Policy, monthly measurement, for payloads < 10 MB):

SLR	Constraint	Measurement
SLR MLS-1 (Latest MLS Sending)	99.5% of MLS messages: M2 - M1 <= 20 minutes	inbound_transaction.as4_timestamp (M1) vs MLS outbound_sending_attempt.as4_timestamp (M2)
SLR MLS-2 (Latest MLS Reception)	99.5% of MLS messages: M3 - M1 <= 25 minutes	inbound_transaction.as4_timestamp (M1) vs outbound_transaction.mls_received_dt (M3)

Multi-Instance Deployment

phoss-ap supports running multiple instances against the same database (PostgreSQL or MySQL) for high availability and horizontal scaling.

Work Distribution via SKIP LOCKED

All DB-based schedulers and checks use SELECT ... FOR UPDATE SKIP LOCKED (supported by both PostgreSQL and MySQL 8.0+) to safely distribute work across instances without requiring leader election:

• Retry scheduler (outbound sending and inbound forwarding): Each instance picks up eligible transactions independently. SKIP LOCKED ensures no two instances process the same transaction concurrently.
• Archival scheduler: Each instance picks up completed transactions to archive. Same SKIP LOCKED approach prevents double-moves.
• Duplicate detection: Protected by unique constraints on as4_message_id and sbdh_instance_id. Concurrent inserts are safe — the second insert fails and is handled gracefully.
• MLS correlation: DB-based lookup by SBDH Instance Identifier. No instance-affinity required.

Reporting Submission (Leader-Based)

Peppol reporting (TSR/EUSR) submission must happen exactly once per reporting period. This uses a leader-based approach controlled via configuration:

• reporting.leader.enabled: When true, this instance acts as the reporting leader and submits Peppol reports on schedule. Default is false.

Exactly one instance should have this set to true. All other instances set it to false.

Circuit Breaker (Per-Instance, In-Memory)

The per-C3 circuit breaker is held in-memory per instance. Each instance independently tracks the health of C3 endpoints it communicates with. This is acceptable — each instance protects itself, and a restart clears all breaker state (which acts as a natural recovery).

Notifications

Notifications are triggered by the instance that performs the action (e.g., the instance that exhausts retries triggers the permanent failure notification). Since work distribution via SKIP LOCKED ensures only one instance handles a given transaction, duplicate notifications do not occur.

Archival

Transactions that are fully completed successfully are moved from the primary DB tables to backup/archive tables to prevent table bloat. The archive rows reference the same document files on disk via document_path. Failed transactions remain in the primary tables. File cleanup for archived documents is a separate future concern.

Graceful Shutdown

The AP uses Spring Boot's built-in graceful shutdown (server.shutdown=graceful) combined with Spring SmartLifecycle for phased, orderly termination.

Shutdown Phases

Shutdown proceeds in three phases:

1. Stop schedulers — The retry scheduler (outbound + inbound), archival scheduler, and reporting submission (if leader) are stopped. No new work is picked up from the database. Any work items not yet claimed remain in the DB with their next_retry_dt intact and will be picked up after restart (or by another instance).

2. Stop accepting new requests — Spring Boot's graceful shutdown stops the servlet container from accepting new HTTP connections. This covers both the REST API (new outbound submissions, reporting calls) and the phase4 AS4 servlet (new inbound AS4 messages). Requests already being processed continue to completion.

3. Await in-flight operations — The bounded ExecutorService thread pools used for outbound AS4 sending and inbound forwarding are shut down via shutdown() + awaitTermination(timeout). This gives in-flight operations time to complete within the configured timeout (shutdown.timeout.ms). If the timeout expires, shutdownNow() interrupts remaining tasks.

Stale Transaction Recovery at Startup

If a shutdown is not clean (e.g., process kill, timeout exceeded, crash), some transactions may be left in transient states (sending or forwarding) that no instance is actively processing. On startup, the AP runs a recovery step:

• outbound_transaction rows with status = 'sending' are reset to failed with next_retry_dt = NOW() so they become immediately eligible for retry.
• inbound_transaction rows with status = 'forwarding' are reset to forward_failed with next_retry_dt = NOW().

This query runs once at boot time, before schedulers are started. In a multi-instance deployment, each instance runs this recovery — SKIP LOCKED is not needed here because only one instance will encounter the stale rows of its own previous incarnation (other instances' in-flight work is still in progress and locked by their transactions). However, the recovery uses UPDATE ... WHERE status = 'sending' which is idempotent, so concurrent execution by multiple instances at startup is safe.

Configuration

Property	Description	Default
server.shutdown	Spring Boot shutdown mode. Set to graceful.	immediate
spring.lifecycle.timeout-per-shutdown-phase	Spring Boot timeout for the servlet container drain phase.	30s
shutdown.timeout.ms	Maximum time in milliseconds to wait for in-flight sending/forwarding operations to complete.	30000

Forwarding Payload

The AP forwards the complete SBD (Standard Business Document = SBDH + business document) to the Receiver Backend. The Receiver Backend is responsible for parsing the SBDH and extracting the business document if needed.

Notifications

A generic Java interface is provided for failure notifications. Concrete implementations are deployment-specific (e.g., email, Slack, monitoring system webhook). Notification triggers include:

• Outbound document verification rejection
• Inbound document verification rejection
• Permanent sending failure (max retries exhausted)
• Permanent forwarding failure on the receiving side