Skip to content

Field Engine

themarioga edited this page Nov 8, 2026 · 1 revision

Field Engine

The field engine turns the raw values of one line into typed values, or into the list of every error of the line. It is a pure function: the same fields, the same line and the same answers of your SPIs always give the same result.

Import: the ten steps

For each field, in the order of its copies (a copy source always comes first):

# Step Notes
1 Read If inInput; a missing column or JSON key is an empty value
2 Copy If copyFrom: the final text of the source field replaces the read value
3 Trim String.strip(); a value that is only spaces becomes empty
4 Text transforms UPPER, LOWER, STRIP_ACCENTS (ñ → n), PAD_LEFT/RIGHT (never truncates)
5 Translate Rows IN and BOTH; case-sensitive; unmatched → onUnmatched
6 Custom transformer FieldTransformer.apply(value, recordSoFar, Direction.IMPORT); never called with an empty value
7 Convert to the type Failure → INVALID_TYPE
8 Validations On the transformed text; every failing validation is reported
9 Resolve empty onEmpty
10 Drop KEEP fields At the end of the line

A failing step stops that field; the other fields are still computed, so a line reports all its errors at once. A line with at least one field error never reaches the consumer.

Empty values

onEmpty Result
EMPTY (default without defaultValue) The field is present with null: "empty it"
KEEP The field is absent from the record: "do not touch it"
ERROR The line is rejected: REQUIRED_EMPTY
DEFAULT (default with defaultValue) The default value, converted to the field type

Absent and null mean different things to your consumer: record.has("f") is false for an absent field and true for an empty one. A copy of an empty KEEP field receives empty.

A defaultValue with ${…} is a template for your DefaultValueResolver, which receives the template and the record computed so far. Without a resolver the line is rejected (DEFAULT_UNRESOLVABLE); an exception of the resolver rejects it too (DEFAULT_RESOLVER_FAILED). A literal default never calls the resolver.

Types

Type Java Default format (no typeFormat) With typeFormat
STRING String — —
INTEGER Long digits with an optional sign; out of range is invalid {groupingSeparator}
DECIMAL BigDecimal (exact) . as decimal separator, no grouping {decimalSeparator, groupingSeparator}
DATE LocalDate yyyy-MM-dd a DateTimeFormatter pattern, strict (31/02/2026 is invalid)
DATETIME LocalDateTime yyyy-MM-dd'T'HH:mm:ss a DateTimeFormatter pattern
BOOLEAN Boolean true / false, ignoring case {trueLiteral, falseLiteral}, case-sensitive

When a grouping separator is configured and appears in a number, the groups must be of three digits: 1.234.567,8 is valid, 1234.5 with . as grouping separator is INVALID_TYPE (never 12345).

Values read from XLSX cells also accept the canonical form (2026-12-31, 1234.5) whatever the typeFormat, because cells hold typed values. See File Formats.

Translations

FieldDefinition.builder("sex")
        .translation("M", "Male")                                      // BOTH
        .translation(new Translation("X", null, TranslationDirection.IN)) // X → empty, import only
        .onUnmatched(OnUnmatched.ERROR)

Import uses IN and BOTH rows (from → to); export uses OUT rows as they are and BOTH rows inverted (to → from). Two rows with the same key for the same direction are a configuration error.

Custom transformers

Tie tie = Tie.builder()
        .transformer("normalizeNif", (value, record, direction) -> value.replace("-", ""))
        .build();

The transformer receives the value after the text steps, a read-only view of the fields computed so far (typed), and the Direction, so one transformer can behave differently on import and export. Returning null empties the value; an exception rejects the line with TRANSFORMER_FAILED. With VALIDATE_THEN_DELIVER it runs once per pass, so keep it deterministic.

A layout without fields

A layout with no fields passes every value through as STRING, keyed by its header name (or column1, column2… without header, or its path in JSON/XML).

Export: the eight steps

# Step Notes
1 Read The value under the field name in your Map; a blank string is empty
2 Copy The supplied value of the source field
3 Validations On the canonical text of the value
4 Convert to text With typeFormat; skipped with an effective outputAs NATIVE
5 Text transforms
6 Translate OUT rows, BOTH rows inverted
7 Custom transformer With Direction.EXPORT
8 Resolve empty KEEP behaves as EMPTY

An exported value may be the Java type of the field, any number for INTEGER/DECIMAL (an INTEGER cannot have decimals), or its canonical text ("42", "2026-12-31"). Anything else is INVALID_TYPE. See Exports and Delivery for outputAs.

Error codes

Every rejected line carries LineError{field, code, args, message}. The code and the names of its args are a stable contract; the English message may change. See Errors and Warnings.

Clone this wiki locally