UDT not working

[pr3d]

Hello there,

I have been trying to use UDT with GridFTP for several days now and it doesn't work.

Both servers running Ubuntu 18.04.2 LTS.

---

1. Downloaded GCT 6.2.20190226
2. Configured with:
   ./configure --prefix=/home/USER/tools/gct62_selfcompiled --enable-udt --disable-gsi-openssh --disable-gram5 --disable-gsi
3. Make:
   make ccommonlibs gridftp udt -j 16
4. Install:
   make install
5. Follow documentation -> Enabling Threading in GridFTP
6. Start server:

~/tools/gct62_selfcompiled/sbin$ ./globus-gridftp-server -debug -dc-whitelist udt,gsi,tcp -threads 1 -allow-udt

Server listening at SERVER-IP:40899

7. Start UDT transfer like mentioned here on Client:

~/tools/gct62_selfcompiled/bin$ ./globus-url-copy -dbg -udt file:/dev/zero sshftp://SERVER-IP/dev/null
debug: starting to put sshftp://SERVER-IP/dev/null
debug: connecting to sshftp://SERVER-IP/dev/null
debug: response from sshftp://SERVER-IP/dev/null:
220 SERVER-IP GridFTP Server 12.17 (gcc64, 1558548600-85) [Globus Toolkit 6.0.1558548600] ready.

debug: authenticating with sshftp://SERVER-IP/dev/null
debug: response from sshftp://SERVER-IP/dev/null:
230 User anonymous logged in.

debug: sending command to sshftp://SERVER-IP/dev/null:
SITE HELP

debug: response from sshftp://SERVER-IP/dev/null:
214-The following commands are recognized:
    ALLO    APPE    REST    CWD     CDUP    DCAU    EPSV    FEAT
    ERET    MDTM    STAT    ESTO    HELP    LIST    MODE    NLST
    MLSC    MLSD    PASV    RNFR    MLSR    MLST    NOOP    OPTS
    STOR    PASS    PBSZ    PORT    PROT    SITE    EPRT    RETR
    SPOR    MFMT    SCKS    TREV    PWD     QUIT    SBUF    SIZE
    SPAS    STRU    SYST    RNTO    TYPE    USER    LANG    MKD
    RMD     DELE    CKSM    DCSC
214 End

debug: sending command to sshftp://SERVER-IP/dev/null:
FEAT

debug: response from sshftp://SERVER-IP/dev/null:
211-Extensions supported
 DSI file-12.17
 STORATTR
 UPAS
 HTTP
 DCSC P,D
 MFMT
 WHOAMI
 AUTHZ_ASSERT
 MLSR
 MLSC
 UTF8
 LANG EN
 DCAU
 PARALLEL
 SIZE
 MLST Type*;Size*;Modify*;Perm*;Charset;UNIX.mode*;UNIX.owner*;UNIX.uid*;UNIX.group*;UNIX.gid*;Unique*;UNIX.slink*;X.count;
 ERET
 ESTO
 SPAS
 SPOR
 REST STREAM
 MDTM
 PASV AllowDelayed;
211 End.

debug: sending command to sshftp://SERVER-IP/dev/null:
SITE CLIENTINFO scheme=sshftp;appname="globus-url-copy";appver="10.4 (gcc64, 1550490409-0) [Globus Toolkit 6.0.1517984806]";
debug: response from sshftp://SERVER-IP/dev/null:
250 OK.

debug: sending command to sshftp://SERVER-IP/dev/null:
TYPE I
debug: response from sshftp://SERVER-IP/dev/null:
200 Type set to I.

debug: sending command to sshftp://SERVER-IP/dev/null:
PASV

debug: response from sshftp://SERVER-IP/dev/null:
227 Entering Passive Mode (xxx,xxx,xxx,xxx,230,137)

debug: sending command to sshftp://SERVER-IP/dev/null:
STOR /dev/null

debug: response from sshftp://SERVER-IP/dev/null:
150 Beginning transfer.

debug: writing buffer 0x7fe3828b4010, length 1048576, offset=0, eof=false
[...]

---

Netstat shows:

Every 1.0s: netstat -tupn

(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp   1623720      0 SERVER-IP:43839     CLIENT-IP:50134     ESTABLISHED 30567/globus-gridft
[...]

---

ps-aux:

/home/USER/tools/gct62_selfcompiled/sbin/globus-gridftp-server -ssh -dc-whitelist udt,gsi,tcp -threads 1 -allow-udt -debug

---

Also tried configuring the files sshftp and gridftp.conf on the Server, still not successful.
Unfortunately it is always using TCP..

[pr3d]

Running strace server side shows that UDT lib is loaded:

openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libglobus_xio_udt_driver.la", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/libglobus_xio_udt_driver.la", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/libglobus_xio_udt_driver.la", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libfakeroot/libglobus_xio_udt_driver.la", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/local/lib/libglobus_xio_udt_driver.la", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/lib/x86_64-linux-gnu/libglobus_xio_udt_driver.la", O_RDONLY) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libglobus_xio_udt_driver.la", O_RDONLY) = -1 ENOENT (No such file or directory)
access("/usr/lib/x86_64-linux-gnu/libglobus_xio_udt_driver.so", R_OK) = 0
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libglobus_xio_udt_driver.so", O_RDONLY|O_CLOEXEC) = 12
openat(AT_FDCWD, "/usr/lib/libudt.so.0", O_RDONLY|O_CLOEXEC) = 12

Running strace client side shows that the "-udt" argument isn't used/processed/...:

execve("~/tools/gct62_selfcompiled/bin/globus-url-copy", ["globus-url-copy", "-dbg", "-udt", "file:/dev/zero", "sshftp://SERVER-IP/dev/null"], 0x7ffc07bc3470 /* 24 vars */) = 0

[fscheiner]

I know that this worked in the past with earlier Globus Toolkit versions (i.e. 5.x or so). But I wonder if this works with sshftp:// and/or using file:/// URLs (which performs a two-party transfer directly between client and server instead of a 3rd party transfer between two servers orchestrated by the client).

Could you try using UDT in a 3rd-party transfer and using gsiftp:// instead?

I think it could be enough to use one GridFTP service/server only, because you can do transfers between the same server and these are not transformed into local copies then AFAIK and you should be also able to have server and client on the same host to simplify things.

You'll need host and client credentials for such a configuration, please follow the Basic Security Configuration to obtain these.

UPDATE: I just recognized that you were using --disable-gsi when configuring the toolkit build. You'll have to rebuild the toolkit without this option during the config run to be able to test with gsiftp://.