Skip to content

Integration Definitions

themarioga edited this page Nov 8, 2026 · 1 revision

Integration Definitions

An IntegrationDefinition is an immutable description of one import or one export. Build it with the Java API (IntegrationDefinition.importing(id) / exporting(id)) or load it from a declarative document. Building never validates: Tie.validate(definition) returns every problem at once, and importFiles / exportRecords throw a ConfigurationException with the same list before touching any file.

IntegrationDefinition

Property Applies to Default Meaning
id both required Name of the integration, used in logs
direction both required IMPORT or EXPORT
layouts both ≥ 1 required Structures of the files and of their nested records
inputChannels import ≥ 1 required Where files are listed; see Channels
outputChannel export required Where the exported file is delivered
outputNamePattern export required Name of the exported file, without extension
deliveryStrategy import STREAMING STREAMING or VALIDATE_THEN_DELIVER; see Running Imports
rejectionThreshold both no limit maxRejected (count) and maxRejectedPercent (needs VALIDATE_THEN_DELIVER)
consumerErrorPolicy import REJECT_LINE What an exception of the consumer does: REJECT_LINE or ABORT_FILE
duplicateProtection import none watermark (boolean) and postSuccessAction (NONE, MOVE(folder), DELETE)
resultReport import NONE See Result Reports

Settings that do not apply to the direction are configuration errors, not silently ignored (for example an outputChannel in an import, or VALIDATE_THEN_DELIVER in an export).

Layout

A root layout describes a file; a child layout (with parent and nestedKey) describes the records nested inside a record of its parent, in JSON and XML only.

Property Default Meaning
name required Unique in the integration; the key of its watermark
fileNamePattern required for import roots See name patterns; not used by child layouts or exports
allowedFormats ≥ 1 for roots CSV, XLSX, XML, JSON; an export layout allows exactly one
formatOptions defaults below Options per format: csv, xlsx, xml, json
columnLocation HEADER How CSV/XLSX columns map to fields: POSITION, HEADER, LETTER
fields empty The fields; an empty list passes every value through as text
parent, nestedKey — Child layouts: the parent layout and the path of the array (JSON) or container (XML)
onParentRejected CASCADE What children of a rejected parent do: CASCADE (rejected too) or PROCESS

Format options

Format Options and defaults
CSV separator=',', quote='"', encoding=UTF-8, hasHeader=true, firstDataRow=2, skipBlankRecords=false
XLSX sheet=null (the first one), hasHeader=true, firstDataRow=2
XML recordElement (required), rootElement="records" (written by exports), encoding=UTF-8
JSON recordProperty=null (records are the root array), encoding=UTF-8

HEADER with hasHeader=false is a configuration error, except in a layout without fields. firstDataRow must leave room for the header: with a header, the header row is firstDataRow - 1 and every earlier row is a preamble that is skipped without being looked at.

FieldDefinition

Property Default Meaning
name required Unique in the layout, ignoring case
column see below Header name (HEADER), letter A–ZZZ (LETTER), or dotted path in JSON/XML (customer.nif, @id)
inInput true false: the field is not read and consumes no column; it is fed by copyFrom or defaultValue
optional false HEADER only: the column may be missing from the header; the field is then empty
copyFrom — Another field of the layout whose value this field starts from
textTransforms none UPPER, LOWER, STRIP_ACCENTS, PAD_LEFT(length, char), PAD_RIGHT(length, char), in order
translations none Rows {from, to, direction}; direction defaults to BOTH
onUnmatched ERROR A value without translation: PASS, EMPTY or ERROR
transformer — Name of a registered FieldTransformer
type STRING STRING, INTEGER (Long), DECIMAL (BigDecimal), DATE (LocalDate), DATETIME (LocalDateTime), BOOLEAN
typeFormat see Field Engine A date pattern, {decimalSeparator, groupingSeparator} or {trueLiteral, falseLiteral}
validations none minLength, maxLength, regex (whole value), allowedValues
onEmpty DEFAULT if there is a defaultValue, else EMPTY EMPTY, KEEP, ERROR, DEFAULT
defaultValue — A literal, or a template with ${…} resolved by your DefaultValueResolver
outputAs derived Exports to JSON/XLSX: NATIVE or TEXT; see Exports and Delivery

With POSITION, the n-th input field (fields with inInput = true, in declaration order) reads the n-th column and column is not used. With HEADER, column defaults to the field name. With LETTER, every input field needs a letter. In JSON and XML, column is the path and defaults to the field name.

Copies form an acyclic graph: a cycle (A → B → A) is a configuration error, and the order of computation follows the copies whatever the declaration order.

Name patterns

fileNamePattern, outputNamePattern and the report namePattern share one syntax: literal text, {variable} markers, and at most one {timestamp:<pattern>}.

alta_{tenant}_{timestamp:yyyyMMddHHmmss}
  • No wildcards and no path separators. The pattern has no extension: on import it is matched against the whole file name without its last extension (alta_ACME_20261026120000.csv → alta_ACME_20261026120000); on export the extension comes from the format.
  • The extension is compared ignoring case (.CSV is CSV). The rest of the name is case-sensitive.
  • Every {variable} must be in the context's variables; a missing one is a ConfigurationException.
  • {timestamp:…} accepts numeric fields only: yyyy/yy, M/MM, d/dd, H/HH, m/mm, s/ss, S…, quoted text and non-letter literals. Year, month and day are required; missing time fields are 0 ({timestamp:yyyyMMdd} is that day at 00:00:00). Dates are strict: 20260230 does not match.
  • The timestamp is a LocalDateTime with no zone: the clock or system zone never shifts it.

A name that matches the patterns of two layouts fails the whole run (AMBIGUOUS_LAYOUT). Without a timestamp, a pattern matches one base name only, so the files of a layout can only differ by extension.

Channels

See Channels: SftpChannel, LocalDirectoryChannel and ProvidedStreamChannel.

Validation

tie.validate(definition) checks the definition and the SPIs it needs (registered transformers, a SecretProvider when there are ${secret:…} values). Problems carry their location:

layout 'alta', field 'nif': copyFrom 'ghost' does not exist
layout 'alta': copy cycle A -> B -> A
inputChannels[0] (SFTP): host key verification requires knownHostsFile or hostKeyFingerprint (or an explicit insecureAcceptAnyHostKey)
rejectionThreshold: percentage threshold requires VALIDATE_THEN_DELIVER

Variables are checked when the integration runs, because only the context has them.

Clone this wiki locally