Errors when creating and seeding a torrent

[ahundt]

Four Questions Below

What version of WebTorrent?
webtorrent --version
1.12.3 (0.100.0)

What operating system and Node.js version?
npm --version
5.6.0

node --version
v10.1.0

Ubuntu 14.04

What browser and version? (if using WebTorrent in the browser)

n/a I'm using cli/desktop

What did you expect to happen?

I ran:

webtorrent create ~/dataset -o ~/dataset.torrent

where ~/dataset is a folder containing 8k files totaling about 220GB of data with all of the files varying from 0KB to 50MB (most common) up to 180MB (the largest) it created the .torrent file and the contents looked reasonable.

I only have 700MB of hard disk space left so I cannot torrent a second copy of the data.

When I tried dropping the .torrent file it appeared to start copying all of the data, which I don't have th e space for. I tried to run ln -s ~/dataset ~/Downloads/dataset because it looked like that was the directory specified in webtorrent desktop but to no avail.

1. How can I make sure it locates the already existing data and does not re-download it?

I then tried to seed the data manually with:

webtorrent seed ~/dataset.torrent

This appeared to walk through the dataset, but when it finally got to seeding it said only a small fraction of the data was already present.

What actually happened?

I expected a torrent to be created and seeded without re-downloading data in a local folder. The data started re-downloading, and when I tried another approach to seeding it didn't appear to find all the local data on disk.

2. How do I specify both the torrent to seed and the location of the data to seed?

There is not really a clear way to specify both a torrent file and data location on either the CLI and the desktop client. Help in these matters would be greatly appreciated!

Considering the errors above, I've become concerned about possible data loss having occurred.

3. How do I make sure my existing files weren't / aren't accidentally blasted away or overwritten by the seeding/downloading process?

4. When I specify a folder to seed and a folder to download, how do I ensure the folder paths match?

In other words, if I say ~/dataset is where I want to download, it may download to ~/dataset/dataset and thus try to download all the data a second time. This ambiguity isn't very clear.

[DiegoRBaquero]

What do you mean by "but when it finally got to seeding it said only a small fraction of the data was already present."?

[ahundt]

@DiegoRBaquero I created the torrent, then immediately tried to seed on the same data without making changes. The original data I used to create the torrent was 220GB, and on the same computer when I was attempting to do the initial seeding, webtorrent said something like only 40GB had been downloaded.

I also did a smaller test and reported at webtorrent/webtorrent-cli#77

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[feross]

So, there's a bit of a misunderstanding of the commands here. webtorrent create /my/path will create a torrent file for the contents of /my/path. That part is correct.

But when you use webtorrent seed /my/path2, you're actually creating a torrent for /my/path2 and seeding the files there. So you don't want to run webtorrent seed somefile.torrent as then you'll be seeding the torrent file itself.

Your webtorrent seed ~/Downloads/test_dataset command is the right one to use.

But you have another problem, which is that you used --private when you created the initial torrent file, but not when you were seeding, so you ended up creating a different torrent file. You need to use consistent flags. I recommend not using --private as that just prevents the use of the DHT to find peers, which is not what you want.

Hope this helps!