Skip to content

Getting Started

themarioga edited this page Nov 8, 2026 · 2 revisions

Getting Started

Add the dependency

TIE needs Java 25. Use tie-core on its own, or the Spring Boot starter (see Spring Boot).

<dependency>
    <groupId>io.github.tie-integration-engine</groupId>
    <artifactId>tie-core</artifactId>
    <version>0.1.0</version>
</dependency>

tie-core depends on Commons CSV, Apache POI, Jackson 3, JSch (the maintained com.github.mwiede fork) and the SLF4J API. It has no Spring dependency. It ships as the automatic module org.tie.

A first import

Files named people_20261026.csv in /data/in, separated by ;, with a nif and a born column:

nif;born
12345678z;31/12/1990
IntegrationDefinition definition = IntegrationDefinition.importing("people")
        .inputChannel(LocalDirectoryChannel.of(Path.of("/data/in")))
        .layout(Layout.builder("person")
                .fileNamePattern("people_{timestamp:yyyyMMdd}")
                .allowedFormats(FileFormat.CSV)
                .csv(CsvOptions.builder().separator(';').build())
                .field(FieldDefinition.builder("taxId").column("nif").textTransform(TextTransform.UPPER))
                .field(FieldDefinition.builder("born").type(FieldType.DATE)
                        .typeFormat(TypeFormat.date("dd/MM/yyyy"))))
        .duplicateProtection(new DuplicateProtection(true, PostSuccessAction.move("done")))
        .build();

Tie tie = Tie.builder().build();

ImportResult result = tie.importFiles(definition, ImportContext.builder(record -> {
            String taxId = record.get("taxId", String.class);     // "12345678Z"
            LocalDate born = record.get("born", LocalDate.class); // 1990-12-31
            people.save(taxId, born);
            return Outcome.APPLIED;                               // or Outcome.rejected("reason")
        })
        .watermarks(store.loadWatermarks())
        .build());

store.saveWatermarks(result.watermarks());

What happened:

  1. TIE validated the whole definition and context before touching any file.
  2. It listed /data/in and kept the files whose name matches people_{timestamp:yyyyMMdd} and whose timestamp is after the stored watermark of the person layout.
  3. It read each file in streaming, ran every line through the field engine and called your consumer with each valid record.
  4. After a successful file it moved it to /data/in/done and advanced the watermark.
  5. It returned an ImportResult with one FileResult per file, the new watermarks and the warnings.

TIE keeps no state: store result.watermarks() yourself, or the next run will see the same files again (unless they were moved away).

Checking the result

for (FileResult file : result.files()) {
    System.out.printf("%s %s: %d applied, %d rejected%n",
            file.name(), file.status(), file.applied(), file.rejected());
    if (file.error() != null) {
        System.out.println(file.error().type() + ": " + file.error().message());
    }
}

Rejected lines do not stop a file. To see them, register a RunListener, or ask for a result report:

ImportContext.builder(consumer)
        .listener(new RunListener() {
            @Override
            public void onLineRejected(LineRejection rejection) {
                log.warn("{}:{} {}", rejection.sourceFileName(), rejection.lineNumber(), rejection.errors());
            }
        })
        .build();

A first export

IntegrationDefinition export = IntegrationDefinition.exporting("people-out")
        .outputChannel(LocalDirectoryChannel.of(Path.of("/data/out")))
        .outputNamePattern("people_{timestamp:yyyyMMddHHmmss}")
        .layout(Layout.builder("person")
                .allowedFormats(FileFormat.JSON)
                .field(FieldDefinition.builder("taxId"))
                .field(FieldDefinition.builder("born").type(FieldType.DATE)))
        .build();

Iterator<Map<String, Object>> records = List.<Map<String, Object>>of(
        Map.of("taxId", "12345678Z", "born", LocalDate.of(1990, 12, 31))).iterator();

ExportResult exported = tie.exportRecords(export, records, ExportContext.builder().build());
// exported.fileName() == "people_20261108153000.json"
// [{"taxId":"12345678Z","born":"1990-12-31"}]

The file is written to the working directory first and only delivered when the export ends without exceeding its rejection threshold and without being cancelled.

The same import in YAML

version: 1
id: people
direction: IMPORT
inputChannels:
  - { type: LOCAL_DIRECTORY, directory: /data/in }
duplicateProtection:
  watermark: true
  postSuccessAction: { type: MOVE, remoteFolder: done }
layouts:
  - name: person
    fileNamePattern: people_{timestamp:yyyyMMdd}
    allowedFormats: [CSV]
    formatOptions:
      csv: { separator: ";" }
    fields:
      - { name: taxId, column: nif, textTransforms: [UPPER] }
      - { name: born, type: DATE, typeFormat: dd/MM/yyyy }
IntegrationDefinition definition = DefinitionLoader.fromYaml(Files.newInputStream(path));

See Declarative Documents.

Next

Clone this wiki locally