Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions components/camel-file/src/main/docs/file-component.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,13 @@ with the following path and query parameters:
| *startingDirectoryMustExist* (consumer) | Whether the starting directory must exist. Mind that the autoCreate option is default enabled, which means the starting directory is normally auto created if it doesn't exist. You can disable autoCreate and enable this to ensure the starting directory must exist. Will thrown an exception if the directory doesn't exist. | false | boolean
| *startingDirectoryMustHave Access* (consumer) | Whether the starting directory has access permissions. Mind that the startingDirectoryMustExist parameter must be set to true in order to verify that the directory exists. Will thrown an exception if the directory doesn't have read and write permissions. | false | boolean
| *appendChars* (producer) | Used to append characters (text) after writing files. This can for example be used to add new lines or other separators when writing and appending to existing files. To specify new-line (slash-n or slash-r) or tab (slash-t) characters then escape with an extra slash, eg slash-slash-n | | String
| *fileExist* (producer) | What to do if a file already exists with the same name. Override, which is the default, replaces the existing file. Append - adds content to the existing file. Fail - throws a GenericFileOperationException, indicating that there is already an existing file. Ignore - silently ignores the problem and does not override the existing file, but assumes everything is okay. Move - option requires to use the moveExisting option to be configured as well. The option eagerDeleteTargetFile can be used to control what to do if an moving the file, and there exists already an existing file, otherwise causing the move operation to fail. The Move option will move any existing files, before writing the target file. TryRename is only applicable if tempFileName option is in use. This allows to try renaming the file from the temporary name to the actual name, without doing any exists check. This check may be faster on some file systems and especially FTP servers. | Override | GenericFileExist
| *fileExist* (producer) a| What to do if a file already exists with the same name. Override, which is the default, replaces the existing file.

- Append - adds content to the existing file.
- Fail - throws a GenericFileOperationException, indicating that there is already an existing file.
- Ignore - silently ignores the problem and does not override the existing file, but assumes everything is okay.
- Move - option requires to use the moveExisting option to be configured as well. The option eagerDeleteTargetFile can be used to control what to do if an moving the file, and there exists already an existing file, otherwise causing the move operation to fail. The Move option will move any existing files, before writing the target file.
- TryRename is only applicable if tempFileName option is in use. This allows to try renaming the file from the temporary name to the actual name, without doing any exists check. This check may be faster on some file systems and especially FTP servers. | Override | GenericFileExist
| *flatten* (producer) | Flatten is used to flatten the file name path to strip any leading paths, so it's just the file name. This allows you to consume recursively into sub-directories, but when you eg write the files to another directory they will be written in a single directory. Setting this to true on the producer enforces that any file name in CamelFileName header will be stripped for any leading paths. | false | boolean
| *jailStartingDirectory* (producer) | Used for jailing (restricting) writing files to the starting directory (and sub) only. This is enabled by default to not allow Camel to write files to outside directories (to be more secured out of the box). You can turn this off to allow writing files to directories outside the starting directory, such as parent or root folders. | true | boolean
| *lazyStartProducer* (producer) | Whether the producer should be started lazy (on the first message). By starting lazy you can use this to allow CamelContext and routes to startup in situations where a producer may otherwise fail during starting and cause the route to fail being started. By deferring this startup to be lazy then the startup failure can be handled during routing messages via Camel's routing error handlers. Beware that when the first message is processed then creating and starting the producer may take a little time and prolong the total processing time of the processing. | false | boolean
Expand Down Expand Up @@ -151,7 +157,18 @@ with the following path and query parameters:
| *minDepth* (filter) | The minimum depth to start processing when recursively processing a directory. Using minDepth=1 means the base directory. Using minDepth=2 means the first sub directory. | | int
| *move* (filter) | Expression (such as Simple Language) used to dynamically set the filename when moving it after processing. To move files into a .done subdirectory just enter .done. | | String
| *exclusiveReadLockStrategy* (lock) | Pluggable read-lock as a org.apache.camel.component.file.GenericFileExclusiveReadLockStrategy implementation. | | GenericFileExclusive ReadLockStrategy
| *readLock* (lock) | Used by consumer, to only poll the files if it has exclusive read-lock on the file (i.e. the file is not in-progress or being written). Camel will wait until the file lock is granted. This option provides the build in strategies: none - No read lock is in use markerFile - Camel creates a marker file (fileName.camelLock) and then holds a lock on it. This option is not available for the FTP component changed - Changed is using file length/modification timestamp to detect whether the file is currently being copied or not. Will at least use 1 sec to determine this, so this option cannot consume files as fast as the others, but can be more reliable as the JDK IO API cannot always determine whether a file is currently being used by another process. The option readLockCheckInterval can be used to set the check frequency. fileLock - is for using java.nio.channels.FileLock. This option is not avail for Windows OS and the FTP component. This approach should be avoided when accessing a remote file system via a mount/share unless that file system supports distributed file locks. rename - rename is for using a try to rename the file as a test if we can get exclusive read-lock. idempotent - (only for file component) idempotent is for using a idempotentRepository as the read-lock. This allows to use read locks that supports clustering if the idempotent repository implementation supports that. idempotent-changed - (only for file component) idempotent-changed is for using a idempotentRepository and changed as the combined read-lock. This allows to use read locks that supports clustering if the idempotent repository implementation supports that. idempotent-rename - (only for file component) idempotent-rename is for using a idempotentRepository and rename as the combined read-lock. This allows to use read locks that supports clustering if the idempotent repository implementation supports that. Notice: The various read locks is not all suited to work in clustered mode, where concurrent consumers on different nodes is competing for the same files on a shared file system. The markerFile using a close to atomic operation to create the empty marker file, but its not guaranteed to work in a cluster. The fileLock may work better but then the file system need to support distributed file locks, and so on. Using the idempotent read lock can support clustering if the idempotent repository supports clustering, such as Hazelcast Component or Infinispan. | none | String
| *readLock* (lock) a| Used by consumer, to only poll the files if it has exclusive read-lock on the file (i.e. the file is not in-progress or being written). Camel will wait until the file lock is granted. This option provides the build in strategies:

- none - No read lock is in use
- markerFile - Camel creates a marker file (fileName.camelLock) and then holds a lock on it. This option is not available for the FTP component
- changed - Changed is using file length/modification timestamp to detect whether the file is currently being copied or not. Will at least use 1 sec to determine this, so this option cannot consume files as fast as the others, but can be more reliable as the JDK IO API cannot always determine whether a file is currently being used by another process. The option readLockCheckInterval can be used to set the check frequency.
- fileLock - is for using java.nio.channels.FileLock. This option is not avail for Windows OS and the FTP component. This approach should be avoided when accessing a remote file system via a mount/share unless that file system supports distributed file locks.
- rename - rename is for using a try to rename the file as a test if we can get exclusive read-lock.
- idempotent - (only for file component) idempotent is for using a idempotentRepository as the read-lock. This allows to use read locks that supports clustering if the idempotent repository implementation supports that.
- idempotent-changed - (only for file component) idempotent-changed is for using a idempotentRepository and changed as the combined read-lock. This allows to use read locks that supports clustering if the idempotent repository implementation supports that.
- idempotent-rename - (only for file component) idempotent-rename is for using a idempotentRepository and rename as the combined read-lock. This allows to use read locks that supports clustering if the idempotent repository implementation supports that.

Notice: The various read locks is not all suited to work in clustered mode, where concurrent consumers on different nodes is competing for the same files on a shared file system. The markerFile using a close to atomic operation to create the empty marker file, but its not guaranteed to work in a cluster. The fileLock may work better but then the file system need to support distributed file locks, and so on. Using the idempotent read lock can support clustering if the idempotent repository supports clustering, such as Hazelcast Component or Infinispan. | none | String
| *readLockCheckInterval* (lock) | Interval in millis for the read-lock, if supported by the read lock. This interval is used for sleeping between attempts to acquire the read lock. For example when using the changed read lock, you can set a higher interval period to cater for slow writes. The default of 1 sec. may be too fast if the producer is very slow writing the file. Notice: For FTP the default readLockCheckInterval is 5000. The readLockTimeout value must be higher than readLockCheckInterval, but a rule of thumb is to have a timeout that is at least 2 or more times higher than the readLockCheckInterval. This is needed to ensure that amble time is allowed for the read lock process to try to grab the lock before the timeout was hit. | 1000 | long
| *readLockDeleteOrphanLock Files* (lock) | Whether or not read lock with marker files should upon startup delete any orphan read lock files, which may have been left on the file system, if Camel was not properly shutdown (such as a JVM crash). If turning this option to false then any orphaned lock file will cause Camel to not attempt to pickup that file, this could also be due another node is concurrently reading files from the same shared directory. | true | boolean
| *readLockIdempotentRelease Async* (lock) | Whether the delayed release task should be synchronous or asynchronous. See more details at the readLockIdempotentReleaseDelay option. | false | boolean
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -837,25 +837,23 @@ public String getReadLock() {
/**
* Used by consumer, to only poll the files if it has exclusive read-lock on the file (i.e. the file is not in-progress or being written).
* Camel will wait until the file lock is granted.
* <p/>
* This option provides the build in strategies:
* <ul>
* <li>none - No read lock is in use
* <li>markerFile - Camel creates a marker file (fileName.camelLock) and then holds a lock on it. This option is not available for the FTP component
* <li>changed - Changed is using file length/modification timestamp to detect whether the file is currently being copied or not. Will at least use 1 sec
*
* This option provides the build in strategies:\n\n
* - none - No read lock is in use\n
* - markerFile - Camel creates a marker file (fileName.camelLock) and then holds a lock on it. This option is not available for the FTP component\n
* - changed - Changed is using file length/modification timestamp to detect whether the file is currently being copied or not. Will at least use 1 sec
* to determine this, so this option cannot consume files as fast as the others, but can be more reliable as the JDK IO API cannot
* always determine whether a file is currently being used by another process. The option readLockCheckInterval can be used to set the check frequency.</li>
* <li>fileLock - is for using java.nio.channels.FileLock. This option is not avail for Windows OS and the FTP component. This approach should be avoided when accessing
* a remote file system via a mount/share unless that file system supports distributed file locks.</li>
* <li>rename - rename is for using a try to rename the file as a test if we can get exclusive read-lock.</li>
* <li>idempotent - (only for file component) idempotent is for using a idempotentRepository as the read-lock.
* This allows to use read locks that supports clustering if the idempotent repository implementation supports that.</li>
* <li>idempotent-changed - (only for file component) idempotent-changed is for using a idempotentRepository and changed as the combined read-lock.
* This allows to use read locks that supports clustering if the idempotent repository implementation supports that.</li>
* <li>idempotent-rename - (only for file component) idempotent-rename is for using a idempotentRepository and rename as the combined read-lock.
* This allows to use read locks that supports clustering if the idempotent repository implementation supports that.</li>
* </ul>
* Notice: The various read locks is not all suited to work in clustered mode, where concurrent consumers on different nodes is competing
* always determine whether a file is currently being used by another process. The option readLockCheckInterval can be used to set the check frequency.\n
* - fileLock - is for using java.nio.channels.FileLock. This option is not avail for Windows OS and the FTP component. This approach should be avoided when accessing
* a remote file system via a mount/share unless that file system supports distributed file locks.\n
* - rename - rename is for using a try to rename the file as a test if we can get exclusive read-lock.\n
* - idempotent - (only for file component) idempotent is for using a idempotentRepository as the read-lock.
* This allows to use read locks that supports clustering if the idempotent repository implementation supports that.\n
* - idempotent-changed - (only for file component) idempotent-changed is for using a idempotentRepository and changed as the combined read-lock.
* This allows to use read locks that supports clustering if the idempotent repository implementation supports that.\n
* - idempotent-rename - (only for file component) idempotent-rename is for using a idempotentRepository and rename as the combined read-lock.
* This allows to use read locks that supports clustering if the idempotent repository implementation supports that.\n
* \nNotice: The various read locks is not all suited to work in clustered mode, where concurrent consumers on different nodes is competing
* for the same files on a shared file system. The markerFile using a close to atomic operation to create the empty marker file,
* but its not guaranteed to work in a cluster. The fileLock may work better but then the file system need to support distributed file locks, and so on.
* Using the idempotent read lock can support clustering if the idempotent repository supports clustering, such as Hazelcast Component or Infinispan.
Expand Down Expand Up @@ -1087,17 +1085,16 @@ public GenericFileExist getFileExist() {
* What to do if a file already exists with the same name.
*
* Override, which is the default, replaces the existing file.
* <ul>
* <li>Append - adds content to the existing file.</li>
* <li>Fail - throws a GenericFileOperationException, indicating that there is already an existing file.</li>
* <li>Ignore - silently ignores the problem and does not override the existing file, but assumes everything is okay.</li>
* <li>Move - option requires to use the moveExisting option to be configured as well.
* \n\n
* - Append - adds content to the existing file.\n
* - Fail - throws a GenericFileOperationException, indicating that there is already an existing file.\n
* - Ignore - silently ignores the problem and does not override the existing file, but assumes everything is okay.\n
* - Move - option requires to use the moveExisting option to be configured as well.
* The option eagerDeleteTargetFile can be used to control what to do if an moving the file, and there exists already an existing file,
* otherwise causing the move operation to fail.
* The Move option will move any existing files, before writing the target file.</li>
* <li>TryRename is only applicable if tempFileName option is in use. This allows to try renaming the file from the temporary name to the actual name,
* without doing any exists check. This check may be faster on some file systems and especially FTP servers.</li>
* </ul>
* The Move option will move any existing files, before writing the target file.\n
* - TryRename is only applicable if tempFileName option is in use. This allows to try renaming the file from the temporary name to the actual name,
* without doing any exists check. This check may be faster on some file systems and especially FTP servers.
*/
public void setFileExist(GenericFileExist fileExist) {
this.fileExist = fileExist;
Expand Down
Loading