FUSE: Upload vs Download speed

[DePingus]

TLDR:
Is 116 MiB/s up, but only 44 MiB/s down normal for weed mount? What are the bottlenecks?

Hi. I'm testing SeaweedFS with some medium sized video files (1-20GB). Uploading a 3GB file from my workstation to the server transfers at a nice 116 MiB/s. But downloading the same file back to the workstation (immediately after upload) is only transferring at 44 MiB/s; Not so great. For comparison, a smb share I have on the same workstation (different server) does a consistent 72 MiB/s in both directions.

The LXD host running the container is a dual Xeon X5650, 12 cores @ 2.66 GHz with 32GB RAM. 10K SAS drives in RAID. The LXD volume is BTRFS. There is nothing else running on the server. The workstation and server are on the same gig network switch. Iperf3 tests from the same container to the same workstation are 941 Mbits/sec (117 MiB) both directions. I tried setting up the containers with the default LXD bridged network and also with macvlan to expose the whole thing to the LAN. It made no difference.

To reproduce on a LXD host running on Alpine Linux

# create a container and add proxy devices for the filer ports
lxc launch images:alpine/3.15
lxc config device add swmaster filer8888 proxy listen=tcp:0.0.0.0:8888 connect=tcp:127.0.0.1:8888
lxc config device add swmaster filer18888 proxy listen=tcp:0.0.0.0:18888 connect=tcp:127.0.0.1:18888

# enter container
lxc exec swmaster -- /bin/sh

# enable Alpine edge and install seaweedfs 
echo http://dl-cdn.alpinelinux.org/alpine/edge/main > /etc/apk/repositories
echo http://dl-cdn.alpinelinux.org/alpine/edge/community >> /etc/apk/repositories
echo http://dl-cdn.alpinelinux.org/alpine/edge/testing >> /etc/apk/repositories
apk update
apk upgrade
apk add seaweedfs

# setup seaweedfs directories
mkdir /data
mkdir /data/seaweedfs
mkdir /data/seaweedfs/master
mkdir /data/seaweedfs/volume
mkdir /data/seaweedfs/filer

# run seaweedfs
weed master -port=9333 -defaultReplication=000 -mdir="/data/seaweedfs/master" &
weed volume -port=8080 -mserver=localhost:9333 -dir="/data/seaweedfs/volume1" &
weed filer -port=8888 -master=localhost:9333 -defaultStoreDir=/data/seaweedfs/filer &

Fedora 35 workstation

# mount seaweedfs
weed mount -filer=10.66.99.110:8888 -volumeServerAccess=filerProxy -dir=/home/depingus/mntsw -filer.path=/

[chrislusf]

Could you please try "weed mount2" in this version https://github.com/chrislusf/seaweedfs/releases/tag/dev ?

The "weed mount2" is going to replace "weed mount" soon. Please let me know the performance difference if any.

[DePingus]

Thanks for getting back to me. The new mount2 made a huge difference! On the same setup with the bridged container, I used the linux amd64 weed-normal-disk binary from your link on both the server container and the workstation mount client. Downloads are now up to 96 MiB/s. Wow! Upload is still 117MiB/s (which, according to iperf, is maxing out my gig network). Thanks again! This is a great project!