-T/--file-upload . does not read from STDIN on 8.14

[denandz]

I did this

curl --file-upload . --trace - https://pipeto.me/asdf should read from stdin in a non-blocking way, writing data to the server as it's read from STDIN. This does not happen on 8.14. Versions 8.8 to 8.13 work; however, the reading from STDIN is delayed. Version 8.7.1 functions as expected. The following was performed on a Fedora linux machine after compiling each release with ./configure --with-openssl --without-libpsl ; make -j$(nproc). In each case, I started the command and typed asdf<ENTER> as fast as I could.

Version 8.14 - this does not read from STDIN at all and wont send data to the server:

doi@koplje:~/tmp/curls/curl-8.14.0/src$ ./curl --trace - -T . -N https://pipeto.me/asdf
== Info: Host pipeto.me:443 was resolved.
== Info: IPv6: (none)
== Info: IPv4: 146.190.74.113
== Info:   Trying 146.190.74.113:443...
== Info: ALPN: curl offers http/1.1
=> Send SSL data, 5 bytes (0x5)
0000: 16 03 01 02 00                                  .....
== Info: TLSv1.3 (OUT), TLS handshake, Client hello (1):
=> Send SSL data, 512 bytes (0x200)
...yoink...
=> Send header, 127 bytes (0x7f)
0000: 50 55 54 20 2f 61 73 64 66 20 48 54 54 50 2f 31 PUT /asdf HTTP/1
0010: 2e 31 0d 0a 48 6f 73 74 3a 20 70 69 70 65 74 6f .1..Host: pipeto
0020: 2e 6d 65 0d 0a 55 73 65 72 2d 41 67 65 6e 74 3a .me..User-Agent:
0030: 20 63 75 72 6c 2f 38 2e 31 34 2e 30 0d 0a 41 63  curl/8.14.0..Ac
0040: 63 65 70 74 3a 20 2a 2f 2a 0d 0a 54 72 61 6e 73 cept: */*..Trans
0050: 66 65 72 2d 45 6e 63 6f 64 69 6e 67 3a 20 63 68 fer-Encoding: ch
0060: 75 6e 6b 65 64 0d 0a 45 78 70 65 63 74 3a 20 31 unked..Expect: 1
0070: 30 30 2d 63 6f 6e 74 69 6e 75 65 0d 0a 0d 0a    00-continue....
<= Recv SSL data, 5 bytes (0x5)
0000: 17 03 03 00 4a                                  ....J
<= Recv SSL data, 1 bytes (0x1)
0000: 16                                              .
== Info: TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
<= Recv SSL data, 57 bytes (0x39)
0000: 04 00 00 35 00 01 51 80 72 96 16 24 08 00 00 00 ...5..Q.r..$....
0010: 00 00 00 00 00 00 20 ea a6 79 7d 29 2f fe 49 7a ...... ..y})/.Iz
0020: d8 7e f0 19 01 6a 72 76 d7 c0 29 fd 09 e4 dd 81 .~...jrv..).....
0030: a2 1c 56 c1 20 dc 76 00 00                      ..V. .v..
<= Recv SSL data, 5 bytes (0x5)
0000: 17 03 03 00 4a                                  ....J
<= Recv SSL data, 1 bytes (0x1)
0000: 16                                              .
== Info: TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
<= Recv SSL data, 57 bytes (0x39)
0000: 04 00 00 35 00 01 51 80 92 a3 66 6b 08 00 00 00 ...5..Q...fk....
0010: 00 00 00 00 01 00 20 e8 01 76 78 f5 8e d7 a6 b8 ...... ..vx.....
0020: c3 30 00 7d 49 d4 96 db 68 6a 2c c1 5e 9c f0 86 .0.}I...hj,.^...
0030: 57 41 55 6f 4c d1 ca 00 00                      WAUoL....
<= Recv SSL data, 5 bytes (0x5)
0000: 17 03 03 00 2a                                  ....*
<= Recv SSL data, 1 bytes (0x1)
0000: 17                                              .
<= Recv header, 23 bytes (0x17)
0000: 48 54 54 50 2f 31 2e 31 20 31 30 30 20 43 6f 6e HTTP/1.1 100 Con
0010: 74 69 6e 75 65 0d 0a                            tinue..
<= Recv header, 2 bytes (0x2)
0000: 0d 0a                                           ..
asdf
asd
fasdf
asd
fasdf
asdf
asdf
asdf
asdf
asdf
asdf
asdf
asdf
^C

Curl 8.13 below, note the delay and multiple entries before data being sent. Have confirmed this behavior down to version 8.8, too.

doi@koplje:~/tmp/curls/curl-8.13.0/src$ ./curl --trace - -T . -N https://pipeto.me/asdf
== Info: Host pipeto.me:443 was resolved.
== Info: IPv6: (none)
== Info: IPv4: 146.190.74.113
...yoink...
<= Recv header, 23 bytes (0x17)
0000: 48 54 54 50 2f 31 2e 31 20 31 30 30 20 43 6f 6e HTTP/1.1 100 Con
0010: 74 69 6e 75 65 0d 0a                            tinue..
<= Recv header, 2 bytes (0x2)
0000: 0d 0a                                           ..
asd
asdf
asdf
asdf
asdf
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1a                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 9 bytes (0x9)
0000: 34 0d 0a 61 73 64 0a 0d 0a                      4..asd...
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
asdf
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
asdf
asdf=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...

asdf
asdf
asdf
asd
fasd=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1a                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 9 bytes (0x9)
0000: 34 0d 0a 61 73 64 0a 0d 0a                      4..asd...

And 8.7.1 - working as expected. Data is read on STDIN immediately and sent to the server:

doi@koplje:~/tmp/curls/curl-8.7.1/src$ ./curl --trace - -T . -N https://pipeto.me/asdf
== Info: Host pipeto.me:443 was resolved.
== Info: IPv6: (none)
== Info: IPv4: 146.190.74.113
...yoink...
<= Recv header, 23 bytes (0x17)
0000: 48 54 54 50 2f 31 2e 31 20 31 30 30 20 43 6f 6e HTTP/1.1 100 Con
0010: 74 69 6e 75 65 0d 0a                            tinue..
<= Recv header, 2 bytes (0x2)
0000: 0d 0a                                           ..
asdf
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
asdf
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
asdf
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
asdf
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
asdf
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
asdf
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
asdf
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
asdf
=> Send SSL data, 5 bytes (0x5)
0000: 17 03 03 00 1b                                  .....
=> Send SSL data, 1 bytes (0x1)
0000: 17                                              .
=> Send data, 10 bytes (0xa)
0000: 35 0d 0a 61 73 64 66 0a 0d 0a                   5..asdf...
^C

I expected the following

Expected all versions to behave like 8.7.1, reading data from STDIN promptly and sending this to the server.

curl/libcurl version

Not working at all - 8.14:

$ ./curl -V
curl 8.14.0 (aarch64-unknown-linux-gnu) libcurl/8.14.0 OpenSSL/3.2.4 zlib/1.3.1.zlib-ng brotli/1.1.0 zstd/1.5.7
Release-Date: 2025-05-28
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTPS-proxy IPv6 Largefile libz NTLM SSL threadsafe TLS-SRP UnixSockets zstd

Working, but slow - 8.8 to 8.13

curl 8.13.0 (aarch64-unknown-linux-gnu) libcurl/8.13.0 OpenSSL/3.2.4 zlib/1.3.1.zlib-ng brotli/1.1.0 zstd/1.5.7
Release-Date: 2025-04-02
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS brotli HSTS HTTPS-proxy IPv6 Largefile libz NTLM SSL threadsafe TLS-SRP UnixSockets zstd

working as expected - 8.7.1 and below:

curl 8.7.1 (aarch64-unknown-linux-gnu) libcurl/8.7.1 OpenSSL/3.2.4 zlib/1.3.1.zlib-ng brotli/1.1.0 zstd/1.5.7
Release-Date: 2024-03-27
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli HSTS HTTPS-proxy IPv6 Largefile libz NTLM SSL threadsafe TLS-SRP UnixSockets zstd

operating system

Fedora Linux 40

[icing]

@denandz I propose #17517 as a fix. Care to verify on your end?

[denandz]

I'm not sure if this is correctly closed. Using Curl with https://github.com/denandz/httpstream2tcp and current master fails. Performance issues also remain - 8.13.1 works, but is slow. Unfortunately I don't have a cleaner POC for this issue right now.

@bagder did you want to re-open this issue or shall I open a new one for the performace issues and further investigation? I don't have permissions to reopen.

Current master (ccdcf40)

$ ssh -v -o ProxyCommand="./curl -s -N 127.0.0.1:3000/stream -T . --no-progress-meter --expect100-timeout 0.01" 127.0.0.1
OpenSSH_9.6p1, OpenSSL 3.2.4 11 Feb 2025
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug1: configuration requests final Match pass
debug1: re-parsing configuration
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug1: Executing proxy command: exec ./curl -s -N 127.0.0.1:3000/stream -T . --no-progress-meter --expect100-timeout 0.01
...yoink...
debug1: Local version string SSH-2.0-OpenSSH_9.6
...stalls...

8.13.1

$ ssh -v -o ProxyCommand="~/tmp/curls/curl-8.13.0/src/curl -s -N 127.0.0.1:3000/stream -T . --no-progress-meter --expect100-timeout 0.01" 127.0.0.1
OpenSSH_9.6p1, OpenSSL 3.2.4 11 Feb 2025
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug1: configuration requests final Match pass
debug1: re-parsing configuration
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Reading configuration data /etc/ssh/ssh_config.d/50-redhat.conf
debug1: Reading configuration data /etc/crypto-policies/back-ends/openssh.config
debug1: Executing proxy command: exec ~/tmp/curls/curl-8.13.0/src/curl -s -N 127.0.0.1:3000/stream -T . --no-progress-meter --expect100-timeout 0.01
...yoink...
debug1: Local version string SSH-2.0-OpenSSH_9.6
debug1: Remote protocol version 2.0, remote software version OpenSSH_9.6
debug1: compat_banner: match: OpenSSH_9.6 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 127.0.0.1:22 as 'doi'
debug1: load_hostkeys: fopen /home/doi/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: aes256-gcm@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: aes256-gcm@openssh.com MAC: <implicit> compression: none
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug1: kex: curve25519-sha256 need=32 dh_need=32
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
...ssh continues as expected...
debug1: pledge: fork
Last login: Mon Jun  2 23:34:54 2025 from 127.0.0.1
doi@koplje:~$ 

Similarly, 8.7.1 performance versus 8.13.1 performance below:

doi@koplje:~/tmp/curls/curl/src$ scp -o ProxyCommand="~/tmp/curls/curl-8.13.0/src/curl -s -N 127.0.0.1:3000/stream -T . --no-progress-meter --expect100-timeout 0.01" ~/tmp/test.dat 127.0.0.1:weh
test.dat                                                                                                                     2%  173MB  10.1MB/s   13:11 ETA^C
doi@koplje:~/tmp/curls/curl/src$ scp -o ProxyCommand="~/tmp/curls/curl-8.7.1/src/curl -s -N 127.0.0.1:3000/stream -T . --no-progress-meter --expect100-timeout 0.01" ~/tmp/test.dat 127.0.0.1:weh
test.dat                                                                                                                    38% 3142MB 486.4MB/s   00:10 ETA^C

[icing]

I tested with

>  ((echo -n 123; sleep 2; echo -n 456) |./src/curl -vvvv  -T . -N https://pipeto.me/asdf 2>inpoint.log)

to reproduce and verify the fix. Does this work for you as well?

[denandz]

Yup, that works for me, along with while true; do echo ABCD1234; sleep 1; done | ./curl --trace - -T . -N https://pipeto.me/asdf on current master; however, the SSH POC above doesn't work on current master. There's something still going on with STDIN/STDOUT processing.

I tried ((echo -n 123; sleep 2; echo -n 456) |./src/curl --trace -T . -N http://127.0.0.1:3000/stream) with current master, running https://github.com/denandz/httpstream2tcp pointed to a netcat listener running on localhost (cargo run -- -c 127.0.0.1:1234 and nc -v -k -l -p 1234) to rule out issues with httpstream2tcp, and that worked as expected too

[denandz]

Here is the log from httpstream2tcp with the current curl master, only one packet is received from curl. Lines starting with -> come from curl, <- come from the TCP socket:

doi@koplje:~/src/httpstream2tcp$ cargo run -- -v
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.03s
     Running `target/debug/httpstream2tcp -v`
[.] Listening on 0.0.0.0:3000
[.] Will connect to 127.0.0.1:22
[+] PUT to /stream from 127.0.0.1:43034
-> Ok(b"SSH-2.0-OpenSSH_9.6\r\n")
<- b"SSH-2.0-OpenSSH_9.6\r\n"
<- b"\0\0\x03\xcc\x08...yoink...0\0\0\0\0\0\0\0\0"
^C

Versus 8.13.1 below, which is very slow but does work:

~/src/httpstream2tcp$ cargo run -- -v
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.02s
     Running `target/debug/httpstream2tcp -v`
[.] Listening on 0.0.0.0:3000
[.] Will connect to 127.0.0.1:22
[+] PUT to /stream from 127.0.0.1:52666
-> Ok(b"SSH-2.0-OpenSSH_9.6\r\n")
<- b"SSH-2.0-OpenSSH_9.6\r\n"
<- b"\0\0\x03\xcc\x08...yoink...\0\0\0\0\0\0\0\0\0\0\0\0\0"
-> Ok(b"\0\0\x05l\x0...yoink...\0\0")
<- b"\0\0\...yoink...a3+\xf8=G\xa9"
-> Ok(b"\0\0...yoink...d3c\xa4")
<- b"\0\0\0...yoink...\x8f"
-> Ok(b"\0\0\00...yoink...\x96")
...etcetera...

[icing]

Not to say "have you tried switching it off and on again", but are you sure that cargo command picks up the correct curl?

[denandz]

Always a fair thing to check! In this case cargo doesn't run curl, it's running an HTTP server. The curl command is specified manually in the SSH invocation as in #17513 (comment)

EG: ssh -v -o ProxyCommand="./curl -s -N 127.0.0.1:3000/stream -T . --no-progress-meter --expect100-timeout 0.01" 127.0.0.1

When the current master branch curl is used in the above it only reads the first STDIN message from SSH. 8.13.1 works, but is very slow, 8.7.1 works perfectly.