Cannot use custom constraint transforms for certain columns (inconsistent ordering in forward vs. reverse)

[npatki]

Environment Details

• SDV version: 1.2.0 (latest)
• Python version: Any
• Operating System: Any

Error Description

Sometimes, I may have a primary key or PII column that is involved in a constraint. If I try to apply a custom constraint for it, it always falls back to the reject sampling strategy (using is_valid). It never uses the transform strategy (using transform and reverse_transform).

Root Cause

During the Data Processor's reverse transform, we are handling constraints first -- even before the primary keys and anonymized columns are created. This means that the constraints cannot access primary keys or PII columns.

SDV/sdv/data_processing/data_processor.py

Lines 713 to 732 in 50392fa

	for constraint in reversed(self._constraints_to_reverse):
	reversed_data = constraint.reverse_transform(reversed_data)
	num_rows = len(reversed_data)
	sampled_columns = list(reversed_data.columns)
	missing_columns = [
	column
	for column in self.metadata.columns.keys() - set(sampled_columns + self._keys)
	if self._hyper_transformer.field_transformers.get(column)
	]
	if missing_columns and num_rows:
	anonymized_data = self._hyper_transformer.create_anonymized_columns(
	num_rows=num_rows,
	column_names=missing_columns
	)
	sampled_columns.extend(missing_columns)
	if self._keys and num_rows:
	generated_keys = self.generate_keys(num_rows, reset_keys)
	sampled_columns.extend(self._keys)

The expected order is to do the constraints last.

Note that the reverse should always be in the opposite order as the forward transform. The forward happens in this order: (constraints -> primary key -> PII columns). See code