Skip to content

S3 driver creates temporary files in current directory #1519

@0x0badc0de

Description

@0x0badc0de

As a result if a process is executed on an environment where whole FS is read-only except /tmp directory - uploading to S3 fails with Could not append to the internal temporary file.

That's not the case with GCS which tries to find tmp dir:

char* raw_temp_file_name = TF_GetTempFileName(extension.c_str());
=>
https://github.com/tensorflow/tensorflow/blob/17774a11b576186e439174298af82b096dcd5582/tensorflow/core/platform/path.cc#L329

Compare with S3 which just passes null prefix downstream:

kS3FileSystemAllocationTag, nullptr, "_s3_filesystem_XXXXXX",

TF S3 implementation just uses /tmp:
https://github.com/tensorflow/tensorflow/blob/17774a11b576186e439174298af82b096dcd5582/tensorflow/core/platform/s3/s3_file_system.cc#L57

Quick workaround can be to os.chdir("/tmp") if acceptable.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions