[FLINK-11302] FlinkS3FileSystem uses an incorrect path for temporary files.

[art4ul]

What is the purpose of the change

This pull request fixes the bug FLINK-11302
AbstractS3FileSystemFactory uses config value CoreOptions.TMP_DIRS.
This value represented as a list of paths separated by the separator.
Instead of using the config value "as is" I split it by separator and use the first path as a temporary path for FlinkS3FileSystem

Brief change log

• • split TMP_DIRS config value by separator and use the first path as a temporary path for FlinkS3FileSystem*

Verifying this change

Added test that validates that produced by AbstractS3FileSystemFactory object will contain an only first path from multiple paths in config link

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): ( no)
• The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
• The serializers: ( no )
• The runtime per-record code paths (performance sensitive): (no)
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
• The S3 file system connector: (yes)

Documentation

• Does this pull request introduce a new feature? ( no)
• If yes, how is the feature documented? (not applicable)

[kl0u]

Thanks for the work @art4ul ! I will merge later today.

[jonpollock]

I know this is already closed and merged, but I'd just like to point out that the constructor of the FlinkS3FileSystem calls RefCountedTmpFileCreator.inDirectories, passing it a single File object. This method takes multiple files and each will be used as a way of spreading the temp files over multiple directories.

The change you applied, limits this to just the first directory in the LOCAL_DIRS. I understand that you probably didn't want to change the Constructor signature, but I think that is only used in AbstractS3FileSystemFactory.

In any case, thank you for your work and this solve a problem I've had for a few hours now.