FAQ
Please keep the questions coming !
What does "multiple TCP paths" mean? TCP does not deal with paths (routing/forwarding). Is it just a typo for "connections" or does the library really use multiple paths where available?
Facebook internally use SDN which hashes source host/port destination host/port to pick paths. By using multiple connections on independent ports we do get multiple path utilized and can thus get better reliability and throughput if some are less healthy than others.
http://udt.sourceforge.net/ is a solution using UDP but we decided that TCP with multiple flow was a better tradeoff for us (not to reinvent most of the window, congestion control, etc... of TCP)
bittorrent is optimized for sending to many and across peers. for fastest data transfer the time it takes to read the data is the bottleneck once the network is optimized, so hashing before sending would be more costly (for a 1:1 copy) we are considering 1:many case for future development but there are tools (like bittorrent) which already excel at this
Make sure you looked at BUILDS.txt (and .travis.yml)
You might get an error with svn initially, if you get
Scanning dependencies of target gmock
[ 20%] Creating directories for 'gmock'
[ 23%] Performing download step (SVN checkout) for 'gmock'
CMake Error at gmock-stamp/gmock-download.cmake:16 (message):
Command failed: 1
'/usr/bin/svn' 'co' 'https://googlemock.googlecode.com/svn/trunk/' '--non-interactive' 'gmock'
See also
/Users/ldemailly/wdt_eclipse/gmock/src/gmock-stamp/gmock-download-*.log
make[2]: *** [gmock/src/gmock-stamp/gmock-download] Error 1
make[1]: *** [CMakeFiles/gmock.dir/all] Error 2
make: *** [all] Error 2
ldemailly-mbp:wdt_eclipse ldemailly$ cat /Users/ldemailly/wdt_eclipse/gmock/src/gmock-stamp/gmock-download-err.log
svn: E175002: Unable to connect to a repository at URL 'https://googlemock.googlecode.com/svn/trunk'
svn: E175002: OPTIONS of 'https://googlemock.googlecode.com/svn/trunk': Server certificate verification failed: issuer is not trusted (https://googlemock.googlecode.com)
Do it interactively and accept permanently e.g
svn list https://googlemock.googlecode.com
# type "p" to permanently accept
You get errors like
E1207 21:16:40.445510 26999 wdtCmdLine.cpp:282] Returning with code 12 ID_MISMATCH
Short version: you should try to use the connection URL emitted by the receiver side for the sender side to connect to.
But if you want to not use the url, then you need to specify a -transfer_id - for instance:
wdt --destination desthost --transfer_id=none --encryption_type=none
Long version: see https://github.com/facebook/wdt/wiki/Getting-Started-with-the-WDT-command-line
You get an error like
E0729 17:08:27.555832 26484736 ClientSocket.cpp:49] Failed getaddrinfo 192.168.2.2 , 22363 : 8 : nodename nor servname provided, or not known
By default wdt prefers IPV6 and can't resolve an ipv4 address; use --ipv6=false (or --ipv4=true) to fix that. The difference is if you put --ipv6=false and you give a hostname it will try both ipv6 and ipv4 (if the host has both).
Fixed/changed the default in 2015 07 29 build so this shouldn't be a problem anymore
Create a new issue or email: ldemailly at fb dot com
Or chat with us:
Feel free to suggest your favorite collaboration tool - Would a facebook group work for people ?