Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,14 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

### Changed

- **EQL v3 (searchable encryption)**: Proxy now targets EQL v3. Encrypted columns are declared with self-configuring, typed `jsonb` domains (for example `eql_v3_text_search`, `eql_v3_integer_ord`, `eql_v3_json_search`) that encode both the scalar type and the column's searchable capabilities in the column type itself, replacing EQL v2's opaque `eql_v2_encrypted` composite type and its separate `eql_v2_configuration` table. The bundled `cipherstash-client` is upgraded to 0.42.0 and EQL to 3.0.2. Existing v2-encrypted data and schemas must be migrated to v3.
- **EQL v3 (searchable encryption)**: Proxy now targets EQL v3. Encrypted columns are declared with self-configuring, typed `jsonb` domains (for example `eql_v3_text_search`, `eql_v3_integer_ord`, `eql_v3_json_search`) that encode both the scalar type and the column's searchable capabilities in the column type itself, replacing EQL v2's opaque `eql_v2_encrypted` composite type and its separate `eql_v2_configuration` table. The bundled `cipherstash-client` is upgraded to 0.42.0 and EQL to 3.0.3. Existing v2-encrypted data and schemas must be migrated to v3.

### Added

- **Encrypted full-text match with `@@`**: The `@@` operator is now supported on encrypted text columns whose domain carries a match (bloom-filter) term, rewritten to the EQL v3 `eql_v3.match_term` form.

- **Equality on encrypted JSON fields**: `WHERE col -> 'field' = 'value'` now works on encrypted JSON columns, in both the simple and extended query protocols, and in the `->>` and `jsonb_path_query_first(col, path) = value` spellings. `<>` is supported as the negation. The field and the value are combined into a single encrypted value-selector needle and matched by containment, so a query never reveals the field and value separately. Matching is exact and case-sensitive; the value must be a JSON scalar (comparing a whole object or array to a field is rejected β€” use containment with `@>` instead).

### Fixed

- **`LIKE`/`ILIKE` capability checking**: `LIKE` and `ILIKE` on an encrypted column are now gated by the column's token-match capability. Previously these predicates bypassed capability checking and were silently accepted on columns that do not support fuzzy match; they are now rejected with a capability error.
Expand Down
16 changes: 8 additions & 8 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ cts-common = { version = "=0.42.0" }
# EQL v3 domain catalog: maps Postgres domain names to (token type, SEM terms).
# The authoritative source for a column's domain identity (ADR-0002); only the
# SchemaManager depends on it, keeping eql-mapper wire-format-agnostic.
eql-bindings = { version = "=3.0.1" }
eql-bindings = { version = "=3.0.3" }

thiserror = "2.0.9"
tokio = { version = "1.44.2", features = ["full"] }
Expand Down
2 changes: 1 addition & 1 deletion mise.toml
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ CS_PROXY__HOST = "host.docker.internal"
# Misc
DOCKER_CLI_HINTS = "false" # Please don't show us What's Next.

CS_EQL_VERSION = "eql-3.0.2"
CS_EQL_VERSION = "eql-3.0.3"


[tools]
Expand Down
3 changes: 3 additions & 0 deletions packages/cipherstash-proxy/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -440,6 +440,9 @@ pub enum ProtocolError {
#[error("Expected {expected} parameter format codes, received {received}")]
ParameterResultFormatCodesMismatch { expected: usize, received: usize },

#[error("Rewritten statement binds parameter {param}, but only {received} were provided")]
MissingBoundParameter { param: usize, received: usize },

#[error("Expected a {expected} message, received message code {received}")]
UnexpectedAuthenticationResponse { expected: String, received: i32 },

Expand Down
26 changes: 20 additions & 6 deletions packages/cipherstash-proxy/src/postgresql/backend.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ use super::error_handler::PostgreSqlErrorHandler;
use super::message_buffer::MessageBuffer;
use super::messages::error_response::ErrorResponse;
use super::messages::row_description::RowDescription;
use super::messages::BackendCode;
use super::messages::{BackendCode, UNSPECIFIED_TYPE_OID};
use super::Column;
use crate::connect::Sender;
use crate::error::{EncryptError, Error};
Expand Down Expand Up @@ -572,20 +572,34 @@ where
debug!(target: PROTOCOL, client_id = self.context.client_id, ParamDescription = ?description);

if let Some(statement) = self.context.get_statement_from_describe() {
// Describe the params the CLIENT wrote, not the ones PostgreSQL was
// sent. A rewrite may have fused or dropped params, in which case
// the server's description is both shorter than and shifted from
// what the client needs in order to bind.
let param_types = statement
.param_columns
.iter()
.map(|col| {
col.as_ref().map(|col| {
.enumerate()
.map(|(idx, col)| match col {
Some(col) => {
debug!(target: MAPPER, client_id = self.context.client_id, ColumnConfig = ?col);
col.postgres_type.clone()
})
col.postgres_type.oid() as i32
}
// A native param is never fused, so it reaches PostgreSQL
// as some output param; take the type the server inferred
// for it.
None => statement
.output_params
.iter()
.position(|output| output.source.primary_input() == idx)
.and_then(|output_idx| description.types.get(output_idx).copied())
.unwrap_or(UNSPECIFIED_TYPE_OID),
})
.collect::<Vec<_>>();

debug!(target: MAPPER, client_id = self.context.client_id, param_types = ?param_types);

description.map_types(&param_types);
description.set_types(param_types);
}

if description.requires_rewrite() {
Expand Down
36 changes: 35 additions & 1 deletion packages/cipherstash-proxy/src/postgresql/column_mapper.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ use crate::{
proxy::EncryptConfig,
};
use cipherstash_client::eql::Identifier;
use eql_mapper::{EqlTerm, TableColumn, TypeCheckedStatement};
use eql_mapper::{EqlTerm, ParamPlan, TableColumn, TypeCheckedStatement};
use postgres_types::Type;
use std::sync::Arc;
use tracing::{debug, warn};
Expand Down Expand Up @@ -95,6 +95,40 @@ impl ColumnMapper {
Ok(param_columns)
}

/// Maps the params of the *rewritten* statement to an Encrypt column
/// configuration, positionally over [`ParamPlan::outputs`].
///
/// These are the values actually sent to PostgreSQL, which after a fusion
/// are not the values the client bound β€” hence a separate mapping from
/// [`Self::get_param_columns`].
pub fn get_output_param_columns(&self, plan: &ParamPlan) -> Result<Vec<Option<Column>>, Error> {
let mut output_columns = vec![];

for output in plan.outputs() {
let configured_column = match &output.value {
eql_mapper::Value::Eql(eql_term) => {
let TableColumn { table, column } = eql_term.table_column();
let identifier =
Identifier::new(table.value.to_string(), column.value.to_string());

debug!(
target: MAPPER,
msg = "Encrypted output parameter",
param = %output.param,
column = ?identifier,
?eql_term,
);

self.get_column(identifier, eql_term)?
}
_ => None,
};
output_columns.push(configured_column);
}

Ok(output_columns)
}

/// Maps typed statement literal columns to an Encrypt column configuration
pub fn get_literal_columns(
&self,
Expand Down
8 changes: 8 additions & 0 deletions packages/cipherstash-proxy/src/postgresql/context/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -814,6 +814,13 @@ where
self.column_mapper.get_param_columns(typed_statement)
}

pub fn get_output_param_columns(
&self,
plan: &eql_mapper::ParamPlan,
) -> Result<Vec<Option<Column>>, Error> {
self.column_mapper.get_output_param_columns(plan)
}

pub fn get_literal_columns(
&self,
typed_statement: &eql_mapper::TypeCheckedStatement<'_>,
Expand Down Expand Up @@ -1116,6 +1123,7 @@ mod tests {
projection_columns: vec![],
literal_columns: vec![],
postgres_param_types: vec![],
output_params: vec![],
}
}

Expand Down
121 changes: 121 additions & 0 deletions packages/cipherstash-proxy/src/postgresql/context/statement.rs
Original file line number Diff line number Diff line change
@@ -1,11 +1,77 @@
use super::Column;
use eql_mapper::{JsonSelectorSource, ParamPlan};

/// Where the path half of a fused JSON value selector comes from.
///
/// The proxy's copy of [`eql_mapper::JsonSelectorSource`], with param numbers
/// converted to 0-based bind indexes.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum JsonSelectorPath {
/// A literal path in the SQL, known at Parse time.
Literal(String),

/// A placeholder path, arriving in this (0-based) input param.
Param(usize),
}

/// How the value bound to one output param is built from the input params.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum OutputParamSource {
/// Taken from this (0-based) input param.
Input(usize),

/// Fused from a JSON path and a value into one value-selector needle.
JsonValueSelector {
path: JsonSelectorPath,
value: usize,
},
}

impl OutputParamSource {
/// The input param this output is built *around* β€” the one whose wire
/// format and (for a passthrough) whose bytes it inherits. For a fusion
/// that is the value operand; the path only contributes to the needle.
pub fn primary_input(&self) -> usize {
match self {
OutputParamSource::Input(idx) => *idx,
OutputParamSource::JsonValueSelector { value, .. } => *value,
}
}
}

/// One param of the *rewritten* statement β€” what PostgreSQL will be sent.
#[derive(Debug, Clone, PartialEq)]
pub struct OutputParam {
/// The column configuration, when this param must be encrypted. `None` for
/// a native param, which is forwarded byte-for-byte.
pub column: Option<Column>,

/// The input param(s) its value is built from.
pub source: OutputParamSource,

/// Whether this param is a query operand, whose payload must be projected
/// to carry search terms without a ciphertext.
pub query_operand: bool,
}

///
/// Type Analysed parameters and projection
///
/// Params have **two** shapes, and they are not guaranteed to correspond:
/// `param_columns` describes what the *client* binds, `output_params` describes
/// what PostgreSQL receives. Encrypted JSON equality fuses two input params into
/// one output param, so any code that assumes `$n` in equals `$n` out is wrong.
///
#[derive(Debug, Clone, PartialEq)]
pub struct Statement {
/// The params as the client sees them β€” used to decode bound values and to
/// answer `Describe`.
pub param_columns: Vec<Option<Column>>,

/// The params of the rewritten statement, in the order PostgreSQL sees
/// them, each naming the input it is built from.
pub output_params: Vec<OutputParam>,

pub projection_columns: Vec<Option<Column>>,
pub literal_columns: Vec<Option<Column>>,
pub postgres_param_types: Vec<i32>,
Expand All @@ -14,12 +80,14 @@ pub struct Statement {
impl Statement {
pub fn new(
param_columns: Vec<Option<Column>>,
output_params: Vec<OutputParam>,
projection_columns: Vec<Option<Column>>,
literal_columns: Vec<Option<Column>>,
postgres_param_types: Vec<i32>,
) -> Statement {
Statement {
param_columns,
output_params,
projection_columns,
literal_columns,
postgres_param_types,
Expand All @@ -38,3 +106,56 @@ impl Statement {
!self.projection_columns.is_empty()
}
}

/// `true` when `output_params` are the first `output_params.len()` input params
/// in order, unchanged β€” the ordinary case, where the rewrite reshaped nothing.
///
/// Callers that hold the bound values must also check the count matches: a
/// prefix match alone would silently drop trailing params.
pub fn params_are_positional(output_params: &[OutputParam]) -> bool {
output_params
.iter()
.enumerate()
.all(|(idx, output)| output.source == OutputParamSource::Input(idx))
}

/// Converts a mapper [`ParamPlan`] to the proxy's 0-based form, pairing each
/// output param with the column configuration that says how to encrypt it.
///
/// `output_columns` is positional over the plan's outputs.
pub fn output_params_from_plan(
plan: &ParamPlan,
output_columns: Vec<Option<Column>>,
) -> Vec<OutputParam> {
plan.outputs()
.iter()
.zip(output_columns)
.map(|(output, column)| OutputParam {
column,
query_operand: output.query_operand,
source: match &output.source {
eql_mapper::OutputParamSource::Input(param) => {
OutputParamSource::Input(to_index(param.0))
}
eql_mapper::OutputParamSource::JsonValueSelector { path, value } => {
OutputParamSource::JsonValueSelector {
path: match path {
JsonSelectorSource::Literal(path) => {
JsonSelectorPath::Literal(path.to_owned())
}
JsonSelectorSource::Param(param) => {
JsonSelectorPath::Param(to_index(param.0))
}
},
value: to_index(value.0),
}
}
},
})
.collect()
}

/// Mapper params are 1-based (`$1`), bind params are 0-based.
fn to_index(param: u16) -> usize {
param.saturating_sub(1) as usize
}
Loading
Loading