Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

command: skip nonregular files #618

Merged
merged 53 commits into from
Aug 4, 2023

Conversation

ahmethakanbesel
Copy link
Contributor

@ahmethakanbesel ahmethakanbesel commented Jul 26, 2023

Resolves #498

Changes are made:

  • cp command checks file mode bits before trying to copy. If the file is a special file, cp do not copy that file. It neither returns an error nor prints a message. The copy process continues as usual.
  • As sync uses cp  in the background, it behaves like cp, so it does not sync special files.

Copy link
Member

@seruman seruman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not remember if we ever discussed this but what about outputting some warnings to users?

For example; a cp to a single fifo file exits with 0 without any output which kinda indicates like if it would successfully copied;

~/src/github.com/peak/s5cmd fix-special-files*
; file fifoo
fifoo: fifo (named pipe)

~/src/github.com/peak/s5cmd fix-special-files*
; s5cmd cp fifoo s3://some-bucket/selman/test/; echo $?
0

~/src/github.com/peak/s5cmd fix-special-files*
; s5cmd ls 's3://some-bucket/selman/test/*'
ERROR "ls s3://some-bucket/selman/test/*": no object found

What do you think @ilkinulas?

e2e/cp_test.go Outdated Show resolved Hide resolved
e2e/sync_test.go Outdated Show resolved Hide resolved
command/cp.go Outdated Show resolved Hide resolved
e2e/cp_test.go Outdated Show resolved Hide resolved
@ahmethakanbesel
Copy link
Contributor Author

ahmethakanbesel commented Aug 1, 2023

I do not remember if we ever discussed this but what about outputting some warnings to users?

For example; a cp to a single fifo file exits with 0 without any output which kinda indicates like if it would successfully copied;

~/src/github.com/peak/s5cmd fix-special-files*
; file fifoo
fifoo: fifo (named pipe)

~/src/github.com/peak/s5cmd fix-special-files*
; s5cmd cp fifoo s3://some-bucket/selman/test/; echo $?
0

~/src/github.com/peak/s5cmd fix-special-files*
; s5cmd ls 's3://some-bucket/selman/test/*'
ERROR "ls s3://some-bucket/selman/test/*": no object found

What do you think @ilkinulas?

I added error messages for skipped objects. It can be good to show information messages. I can revert if it is unnecessary.
da3ccd4

e2e/cp_test.go Outdated Show resolved Hide resolved
command/cp.go Outdated Show resolved Hide resolved
command/cp.go Outdated Show resolved Hide resolved
e2e/cp_test.go Outdated Show resolved Hide resolved
e2e/sync_test.go Outdated Show resolved Hide resolved
e2e/cp_test.go Outdated Show resolved Hide resolved
e2e/sync_test.go Outdated Show resolved Hide resolved
e2e/cp_test.go Outdated Show resolved Hide resolved
e2e/sync_test.go Outdated Show resolved Hide resolved
@ahmethakanbesel ahmethakanbesel changed the title command: skip special files command: skip nonregular files Aug 4, 2023
@ilkinulas ilkinulas merged commit 393e8dd into peak:master Aug 4, 2023
13 checks passed
@ahmethakanbesel ahmethakanbesel deleted the fix-special-files branch August 4, 2023 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

s5cmd cp hangs on named pipes/socketfiles
3 participants