New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File protocol does not work for files on procfs or debugfs #681

Closed
jessetan opened this Issue Feb 25, 2016 · 3 comments

Comments

Projects
None yet
3 participants
@jessetan

Reading files from the local file system using the file:// protocol works fine:

# curl -v "file://localhost/etc/centos-release"
CentOS release 6.5 (Final)
* Closing connection #0

However if the file is located on a procfs or debugfs, no result is returned:

curl -v "file://localhost/proc/cpuinfo"
* Closing connection #0

curl probably does a check for size, since fstat shows size 0. See strace excerpt:

open("/proc/cpuinfo", O_RDONLY)         = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
close(3)                                = 0

Compare that to cat which probably ignores size and does a while buff != EOL in this strace excerpt:

open("/proc/cpuinfo", O_RDONLY)         = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(3, "processor\t: 0\nvendor_id\t: Genuin"..., 32768) = 3876
write(1, "processor\t: 0\nvendor_id\t: Genuin"..., 3876processor    : 0

Is there a way to read these files using curl, perhaps with a special command line option to ignore size? Is this considered a bug or intended behavior?

curl 7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
Protocols: tftp ftp telnet dict ldap ldaps http file https ftps scp sftp
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz
@jay

This comment has been minimized.

Show comment
Hide comment
@jay

jay Feb 25, 2016

Member

Yes, if the file size stats to zero then the function does not attempt to read.

owner@ubuntu1404-x64-vm:~/curl$ curl -I "file://localhost/proc/cpuinfo"
Content-Length: 0
Accept-ranges: bytes
Last-Modified: Thu, 25 Feb 2016 20:15:14 GMT

We could piggyback on --ignore-content-length (which internally maps to data->set.ignorecl) and just continue reading I suppose.

Member

jay commented Feb 25, 2016

Yes, if the file size stats to zero then the function does not attempt to read.

owner@ubuntu1404-x64-vm:~/curl$ curl -I "file://localhost/proc/cpuinfo"
Content-Length: 0
Accept-ranges: bytes
Last-Modified: Thu, 25 Feb 2016 20:15:14 GMT

We could piggyback on --ignore-content-length (which internally maps to data->set.ignorecl) and just continue reading I suppose.

@jay jay added the enhancement label Feb 25, 2016

@bagder

This comment has been minimized.

Show comment
Hide comment
@bagder

bagder Feb 25, 2016

Member

Alternatively, we unconditionally take the zero size as "size unknown" and just try to continue and see what happens. The size is mostly used for making a more accurate progress meter anyway.

Member

bagder commented Feb 25, 2016

Alternatively, we unconditionally take the zero size as "size unknown" and just try to continue and see what happens. The size is mostly used for making a more accurate progress meter anyway.

@bagder bagder closed this in 1e486db Feb 25, 2016

@bagder

This comment has been minimized.

Show comment
Hide comment
@bagder

bagder Feb 25, 2016

Member

Thanks!

I decided to do it like that. Existing functionality remains and I there should be very little risk with this approach. I think.

Member

bagder commented Feb 25, 2016

Thanks!

I decided to do it like that. Existing functionality remains and I there should be very little risk with this approach. I think.

@lock lock bot locked as resolved and limited conversation to collaborators May 7, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.