Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[C++] Reviewing the rename call in LocalFileSystem::Move. #39385

Closed
anjakefala opened this issue Dec 29, 2023 · 6 comments · Fixed by #39481
Closed

[C++] Reviewing the rename call in LocalFileSystem::Move. #39385

anjakefala opened this issue Dec 29, 2023 · 6 comments · Fixed by #39481

Comments

@anjakefala
Copy link
Collaborator

Describe the enhancement requested

LocalFileSystem::Move on non-Windows, calls rename: https://github.com/apache/arrow/blob/main/cpp/src/arrow/filesystem/localfs.cc#L585. It then does a check for -1, and prints the error.

While the rename system call and Posix standard do specify that a return value of -1 is expected for error calls, the C++ reference specifies that a "non-zero" is returned upon error.

I have two questions:

  1. Is the rename that is being called in LocalFileSystem::Move the one in std::? Should it be prefaced with std::?
  2. Should the check be changed to a non-zero check, as opposed to a -1 check? Is there is a possibility that std::rename returns a different number other than -1 upon error?

Component(s)

C++

@anjakefala
Copy link
Collaborator Author

For reference, the LocalFileSystem::Move tests are here: https://github.com/apache/arrow/blob/main/cpp/src/arrow/filesystem/filesystem_test.cc#L701

@mapleFU
Copy link
Member

mapleFU commented Dec 29, 2023

rename(2) ( https://man7.org/linux/man-pages/man2/rename.2.html ) is a Posix api defined in here ( https://github.com/apache/arrow/blob/main/cpp/src/arrow/filesystem/localfs.cc#L28 ). We also having a std::filesystem in C++17 ( https://en.cppreference.com/w/cpp/filesystem/rename )

On success, zero is returned. On error, -1 is returned, and errno is set to indicate the error.

So generally I think it's ok here

@anjakefala
Copy link
Collaborator Author

rename(2) ( https://man7.org/linux/man-pages/man2/rename.2.html ) is a Posix api defined in here ( https://github.com/apache/arrow/blob/main/cpp/src/arrow/filesystem/localfs.cc#L28 ).

Out of curiousity, how did you figure out it was defined in fcntl.h? fcntl.h seems to contain file descriptor notifications?

@mapleFU
Copy link
Member

mapleFU commented Dec 30, 2023

Oh it might in stdio.h, I think rename here should be an POSIX api, so I search it in https://man7.org/linux/man-pages/

After find the api, we can know where we should include

@felipecrv
Copy link
Contributor

libc++ simply re-exports the global rename defined in stdio.h to the std:: namespace [1].

I think the C++ standard, when defining rename didn't want to guarantee -1 is always returned even though the actual implementation guarantees that.

I wouldn't oppose changing the code from == -1 to < 0, but since we are using #ifdef _WIN32 and not trusting the rename provided by Windows via its cstdio, it's more idiomatic to keep the rename without the std:: for POSIX systems. This means we are relying on POSIX and guarding against C++ changing std::rename in the future (even though the chances of that happening are basically 0).

[1] https://github.com/llvm/llvm-project/blob/main/libcxx/include/cstdio#L157

@anjakefala
Copy link
Collaborator Author

it's more idiomatic to keep the rename without the std:: for POSIX systems. This means we are relying on POSIX and guarding against C++ changing std::rename in the future

Oh, thank you for explaining why it does not have the std::! =))

anjakefala added a commit to anjakefala/arrow that referenced this issue Jan 5, 2024
anjakefala added a commit to anjakefala/arrow that referenced this issue Jan 8, 2024
@kou kou closed this as completed in #39481 Jan 9, 2024
kou pushed a commit that referenced this issue Jan 9, 2024
### Rationale for this change

While the `rename` [system call](https://man7.org/linux/man-pages/man2/rename.2.html) and [Posix standard](https://pubs.opengroup.org/onlinepubs/9699919799/functions/rename.html) do specify that a return value of -1 is expected for error calls, the [C++ reference](https://en.cppreference.com/w/cpp/io/c/rename) specifies that a "non-zero" is returned upon error.

This PR proposes changing to the more encompassing "non-zero" check for `std::rename`.

### Are these changes tested?

There are existing tests: https://github.com/apache/arrow/blob/afb40a9f5a33802897e1d5bae8305c81da7beee1/cpp/src/arrow/filesystem/filesystem_test.cc#L701C3-L701C3
* Closes: #39385

Authored-by: anjakefala <anja@voltrondata.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@kou kou added this to the 16.0.0 milestone Jan 9, 2024
clayburn pushed a commit to clayburn/arrow that referenced this issue Jan 23, 2024
…ache#39481)

### Rationale for this change

While the `rename` [system call](https://man7.org/linux/man-pages/man2/rename.2.html) and [Posix standard](https://pubs.opengroup.org/onlinepubs/9699919799/functions/rename.html) do specify that a return value of -1 is expected for error calls, the [C++ reference](https://en.cppreference.com/w/cpp/io/c/rename) specifies that a "non-zero" is returned upon error.

This PR proposes changing to the more encompassing "non-zero" check for `std::rename`.

### Are these changes tested?

There are existing tests: https://github.com/apache/arrow/blob/afb40a9f5a33802897e1d5bae8305c81da7beee1/cpp/src/arrow/filesystem/filesystem_test.cc#L701C3-L701C3
* Closes: apache#39385

Authored-by: anjakefala <anja@voltrondata.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
dgreiss pushed a commit to dgreiss/arrow that referenced this issue Feb 19, 2024
…ache#39481)

### Rationale for this change

While the `rename` [system call](https://man7.org/linux/man-pages/man2/rename.2.html) and [Posix standard](https://pubs.opengroup.org/onlinepubs/9699919799/functions/rename.html) do specify that a return value of -1 is expected for error calls, the [C++ reference](https://en.cppreference.com/w/cpp/io/c/rename) specifies that a "non-zero" is returned upon error.

This PR proposes changing to the more encompassing "non-zero" check for `std::rename`.

### Are these changes tested?

There are existing tests: https://github.com/apache/arrow/blob/afb40a9f5a33802897e1d5bae8305c81da7beee1/cpp/src/arrow/filesystem/filesystem_test.cc#L701C3-L701C3
* Closes: apache#39385

Authored-by: anjakefala <anja@voltrondata.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
zanmato1984 pushed a commit to zanmato1984/arrow that referenced this issue Feb 28, 2024
…ache#39481)

### Rationale for this change

While the `rename` [system call](https://man7.org/linux/man-pages/man2/rename.2.html) and [Posix standard](https://pubs.opengroup.org/onlinepubs/9699919799/functions/rename.html) do specify that a return value of -1 is expected for error calls, the [C++ reference](https://en.cppreference.com/w/cpp/io/c/rename) specifies that a "non-zero" is returned upon error.

This PR proposes changing to the more encompassing "non-zero" check for `std::rename`.

### Are these changes tested?

There are existing tests: https://github.com/apache/arrow/blob/afb40a9f5a33802897e1d5bae8305c81da7beee1/cpp/src/arrow/filesystem/filesystem_test.cc#L701C3-L701C3
* Closes: apache#39385

Authored-by: anjakefala <anja@voltrondata.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants