Skip to content
This repository has been archived by the owner on Jan 21, 2022. It is now read-only.

Latest commit

 

History

History
108 lines (74 loc) · 4.14 KB

README.adoc

File metadata and controls

108 lines (74 loc) · 4.14 KB
Raw

File Source

This application polls a directory and sends new files or their contents to the output channel. The file source provides the contents of a File as a byte array by default. However, this can be customized using the --mode option:

  • ref Provides a java.io.File reference

  • lines Will split files line-by-line and emit a new message for each line

  • contents The default. Provides the contents of a file as a byte array

When using --mode=lines, you can also provide the additional option --withMarkers=true. If set to true, the underlying FileSplitter will emit additional start-of-file and end-of-file marker messages before and after the actual data. The payload of these 2 additional marker messages is of type FileSplitter.FileMarker. The option withMarkers defaults to false if not explicitly set.

Input

N/A (Reads from a file system directory).

Output

mode = contents

Headers:

  • Content-Type: application/octet-stream

  • file_originalFile: <java.io.File>

  • file_name: <file name>

Payload:

A byte[] filled with the file contents.

mode = lines

Headers:

  • Content-Type: text/plain

  • file_originalFile: <java.io.File>

  • file_name: <file name>

  • correlationId: <UUID> (same for each line)

  • sequenceNumber: <n>

  • sequenceSize: 0 (number of lines is not know until the file is read)

Payload:

A String for each line.

The first line is optionally preceded by a message with a START marker payload. The last line is optionally followed by a message with an END marker payload.

Marker presence and format are determined by the with-markers and markers-json properties.

mode = ref

Headers:

None.

Payload:

A java.io.File object.

Options

The file source has the following options:

file.consumer.markers-json

When 'fileMarkers == true', specify if they should be produced as FileSplitter.FileMarker objects or JSON. (Boolean, default: true)

file.consumer.mode

The FileReadingMode to use for file reading sources. Values are 'ref' - The File object, 'lines' - a message per line, or 'contents' - the contents as bytes. (FileReadingMode, default: <none>, possible values: ref,lines,contents)

file.consumer.with-markers

Set to true to emit start of file/end of file marker messages before/after the data. Only valid with FileReadingMode 'lines'. (Boolean, default: <none>)

file.directory

The directory to poll for new files. (String, default: <none>)

file.filename-pattern

A simple ant pattern to match files. (String, default: <none>)

file.filename-regex

A regex pattern to match files. (Pattern, default: <none>)

file.prevent-duplicates

Set to true to include an AcceptOnceFileListFilter which prevents duplicates. (Boolean, default: true)

trigger.cron

Cron expression value for the Cron Trigger. (String, default: <none>)

trigger.date-format

Format for the date value. (String, default: <none>)

trigger.fixed-delay

Fixed delay for periodic triggers. (Integer, default: 1)

trigger.initial-delay

Initial delay for periodic triggers. (Integer, default: 0)

trigger.max-messages

Maximum messages per poll, -1 means infinity. (Long, default: -1)

trigger.time-unit

The TimeUnit to apply to delay values. (TimeUnit, default: SECONDS, possible values: NANOSECONDS,MICROSECONDS,MILLISECONDS,SECONDS,MINUTES,HOURS,DAYS)

The ref option is useful in some cases in which the file contents are large and it would be more efficient to send the file path.

Build

$ ./mvnw clean install -PgenerateApps
$ cd apps

You can find the corresponding binder based projects here. You can then cd into one of the folders and build it:

$ ./mvnw clean package

Examples

java -jar file_source.jar --file.directory=/tmp/foo --file.consumer.mode=lines --trigger.fixed-delay=60