Skip to content

Support Zero-Peristence Mode #4622

@brandonjackson

Description

@brandonjackson
  • Channels must support the project's zero-persistence setting.
  • Apply the same principles about what's retained / what's not as with workflows (ie users can see the status information but not the request/response body)

Scope

Update policy language

Input/Output Data Storage Policy
The input and output data associated with workflow runs and channel requests is useful for debugging. However it can contain sensitive PII. Should OpenFn store this data? Learn more
[ ] Retain all input/output data
[ ] Never retain input/output data (zero-persistence)

Update Request Details Page

  • Hide request headers / response columns

Only Persist Status Info

ChannelRequest

Field Tier Reasoning
id Must keep PK — FKs from ChannelEvent break without it
channel_id Must keep Cannot aggregate "is channel X healthy?" without it
state Must keep The single most important health signal
started_at Must keep Billing, rate-limit accounting, throughput
client_webhook_auth_method_id Must keep Security audit — proves auth was applied to each request
channel_snapshot_id Should keep Useful when "everything broke after the customer changed config"
destination_credential_id Should keep Audit / "which credential failed"
client_auth_type Should keep Cheap operational metadata
completed_at Should keep Cleanest aggregate latency signal
request_id Don't keep Customer correlation, not ours
client_identity Definitely don't keep PII (IPs, end-user identifiers)

ChannelEvent

Field Tier Reasoning
id, channel_request_id Must keep FKs
type Must keep Distinguishes destination-failure from non-2xx — different alerting paths
response_status Must keep Core ops signal
inserted_at Must keep Timestamp
latency_us Must keep Headline SLO metric
ttfb_us, response_duration_us Should keep Distinguishes "slow destination" from "slow network"
request_send_us, queue_us, connect_us Should keep Granular timing breakdown
request_body_size, response_body_size Should keep Egress billing, capacity planning
reused_connection Should keep Connection-pool optimization signal
error_message (as classified category only) Should keep Replace free text with :timeout/:connection_refused/:tls_error/:destination_4xx/:destination_5xx/:parse_error enum
request_method Don't keep Useful for dashboards but not essential
error_message (free text) Definitely don't keep Destinations echo request data in error bodies
request_path Definitely don't keep REST paths embed customer record ids
request_query_string Definitely don't keep Frequently carries emails, tokens, filters
request_headers Definitely don't keep Auth tokens, cookies, custom X-User-*
response_headers Definitely don't keep Set-Cookie, identifying response headers
request_body_preview, response_body_preview Definitely don't keep Main PII surface
request_body_hash, response_body_hash Definitely don't keep Reversible for short/low-entropy values

Metadata

Metadata

Assignees

Labels

channelsneeds detailMore information on the problem/business requirement is needed

Type

No type
No fields configured for issues without a type.

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions