Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cp fails on linux for large files #39868

Open
JonasIsensee opened this issue Mar 1, 2021 · 2 comments
Open

cp fails on linux for large files #39868

JonasIsensee opened this issue Mar 1, 2021 · 2 comments
Labels
domain:io Involving the I/O subsystem: libuv, read, write, etc. system:linux Affects only Linux

Comments

@JonasIsensee
Copy link
Contributor

I just noticed that on linux using julia1.6.0-rc1 cp fails with

IOError: sendfile: Unknown system error -167499847 (Unknown system error -167499847)
Stacktrace:
 [1] uv_error
   @ ./libuv.jl:97 [inlined]
 [2] sendfile(dst::Base.Filesystem.File, src::Base.Filesystem.File, src_offset::Int64, bytes::Int64)
   @ Base.Filesystem ./filesystem.jl:119
 [3] sendfile(src::String, dst::String)
   @ Base.Filesystem ./file.jl:960
 [4] cp(src::String, dst::String; force::Bool, follow_symlinks::Bool)
   @ Base.Filesystem ./file.jl:355
 [5] top-level scope
   @ REPL[1308]:1

when copying large files.
E.g.

julia> stat("file")
StatStruct(mode=0o100664, size=2619968817)

Using previous julia version cp behaves as expected.

julia> versioninfo()
Julia Version 1.5.2
Commit 539f3ce943 (2020-09-23 23:17 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake-avx512)
@mgkuhn
Copy link
Contributor

mgkuhn commented Mar 1, 2021

I was unable to reproduce this on Ubuntu 20.04 x86_64

julia> versioninfo()
Julia Version 1.6.0-rc1
Commit a58bdd9010 (2021-02-06 15:49 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, ivybridge)

with

size=2619968817
src="/tmp/big"
dst="$(src)2"
open(src, "w") do f
  seek(f, size-1)
  write(f, UInt8(0))
end
cp(src, dst)
run(`ls -ld $(src) $(dst)`)
run(`cmp $(src) $(dst)`)
cp(dst, src; force=true)
run(`ls -ld $(src) $(dst)`)
run(`cmp $(src) $(dst)`)
rm.((src, dst))

and

$ df /tmp ; findmnt /
Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/sdb2       26573796 18600576   6776852  74% /
TARGET SOURCE    FSTYPE OPTIONS
/      /dev/sdb2 ext4   rw,relatime,errors=remount-ro

@vtjnash
Copy link
Sponsor Member

vtjnash commented Mar 1, 2021

Possibly a duplicate of #30723

@JeffBezanson JeffBezanson added the domain:io Involving the I/O subsystem: libuv, read, write, etc. label Mar 1, 2021
@ViralBShah ViralBShah added the system:linux Affects only Linux label Mar 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
domain:io Involving the I/O subsystem: libuv, read, write, etc. system:linux Affects only Linux
Projects
None yet
Development

No branches or pull requests

5 participants