File protocol does not work for files on procfs or debugfs #681

Closed
jessetan opened this Issue Feb 25, 2016 · 3 comments

Projects

None yet

3 participants

@jessetan

Reading files from the local file system using the file:// protocol works fine:

# curl -v "file://localhost/etc/centos-release"
CentOS release 6.5 (Final)
* Closing connection #0

However if the file is located on a procfs or debugfs, no result is returned:

curl -v "file://localhost/proc/cpuinfo"
* Closing connection #0

curl probably does a check for size, since fstat shows size 0. See strace excerpt:

open("/proc/cpuinfo", O_RDONLY)         = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
close(3)                                = 0

Compare that to cat which probably ignores size and does a while buff != EOL in this strace excerpt:

open("/proc/cpuinfo", O_RDONLY)         = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(3, "processor\t: 0\nvendor_id\t: Genuin"..., 32768) = 3876
write(1, "processor\t: 0\nvendor_id\t: Genuin"..., 3876processor    : 0

Is there a way to read these files using curl, perhaps with a special command line option to ignore size? Is this considered a bug or intended behavior?

curl 7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.15.3 zlib/1.2.3 libidn/1.18 libssh2/1.4.2
Protocols: tftp ftp telnet dict ldap ldaps http file https ftps scp sftp
Features: GSS-Negotiate IDN IPv6 Largefile NTLM SSL libz
@jay
Member
jay commented Feb 25, 2016

Yes, if the file size stats to zero then the function does not attempt to read.

owner@ubuntu1404-x64-vm:~/curl$ curl -I "file://localhost/proc/cpuinfo"
Content-Length: 0
Accept-ranges: bytes
Last-Modified: Thu, 25 Feb 2016 20:15:14 GMT

We could piggyback on --ignore-content-length (which internally maps to data->set.ignorecl) and just continue reading I suppose.

@jay jay added the enhancement label Feb 25, 2016
@bagder
Member
bagder commented Feb 25, 2016

Alternatively, we unconditionally take the zero size as "size unknown" and just try to continue and see what happens. The size is mostly used for making a more accurate progress meter anyway.

@bagder bagder added a commit that closed this issue Feb 25, 2016
@bagder bagder file: try reading from files with no size
Some systems have special files that report as 0 bytes big, but still
contain data that can be read (for example /proc/cpuinfo on
Linux). Starting now, a zero byte size is considered "unknown" size and
will be read as far as possible anyway.

Reported-by: Jesse Tan

Closes #681
1e486db
@bagder bagder closed this in 1e486db Feb 25, 2016
@bagder
Member
bagder commented Feb 25, 2016

Thanks!

I decided to do it like that. Existing functionality remains and I there should be very little risk with this approach. I think.

@jessetan jessetan referenced this issue in collectd/collectd Feb 29, 2016
Open

Create "read file" plugin #1192

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment