Download not starting when using standalone client with private runtime #34

bbijelic · 2017-08-13T18:45:30Z

I would be very thankful for little assistance. Not sure if I'm missing something.

I'm trying to setup simple example you provided, but it fails to start download either by using torrent file or magnet link.

It just starts and hangs at one point, no download has started. I can confirm that magnet url works with e.g. uTorrent application.

JCE has been installed and verified by checking max key sizes.

Environment

OS: Windows 10
Java: Oracle JDK 8u144 + JCE
Windows firewall = disabled (for testing purposes)

Using Bt release from 2017-08-11: 1.3.1, was also not working with 1.3 version.

Example code:

package com.github.bbijelic.torrent;

import java.io.File;

import bt.Bt;
import bt.data.Storage;
import bt.data.file.FileSystemStorage;
import bt.runtime.BtClient;
import bt.torrent.selector.SequentialSelector;

public class TorrentClient {

	public static void main(String[] args) {

		String magnetLink = "magnet:?xt=urn:btih:2b32e64f6cd755a9e54d60e205a9681d6670cfae";
		Storage storage = new FileSystemStorage(new File("C:\\Users\\Snijele\\Desktop\\TEST"));
		BtClient client = Bt.client().storage(storage).magnet(magnetLink)
				.autoLoadModules().build();

		// launch
		client.startAsync(state -> {
			if (state.getPiecesRemaining() == 0) {
				client.stop();
			}
		}, 1000).join();
	}

}

Log output:

20:35:03.443 [main] INFO bt.runtime.BtRuntimeBuilder - Auto-loading module bt.tracker.http.HttpTrackerModule with default configuration
20:35:03.448 [main] INFO bt.runtime.BtRuntimeBuilder - Auto-loading module bt.peerexchange.PeerExchangeModule with default configuration
20:35:03.916 [main] INFO bt.runtime.BtRuntimeBuilder - Auto-loading module bt.tracker.http.HttpTrackerModule with default configuration
20:35:03.916 [main] INFO bt.runtime.BtRuntimeBuilder - Auto-loading module bt.peerexchange.PeerExchangeModule with default configuration
20:35:03.948 [main] DEBUG bt.service.ClasspathApplicationService - Processing manifest file: jar:file:/C:/Users/Snijele/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar!/META-INF/MANIFEST.MF
20:35:03.949 [main] DEBUG bt.service.ClasspathApplicationService - Bundle-SymbolicName: ch.qos.logback.classic
20:35:03.950 [main] DEBUG bt.service.ClasspathApplicationService - Processing manifest file: jar:file:/C:/Users/Snijele/.m2/repository/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar!/META-INF/MANIFEST.MF
20:35:03.950 [main] DEBUG bt.service.ClasspathApplicationService - Bundle-SymbolicName: ch.qos.logback.core
20:35:03.950 [main] DEBUG bt.service.ClasspathApplicationService - Processing manifest file: jar:file:/C:/Users/Snijele/.m2/repository/org/slf4j/slf4j-api/1.7.25/slf4j-api-1.7.25.jar!/META-INF/MANIFEST.MF
20:35:03.951 [main] DEBUG bt.service.ClasspathApplicationService - Bundle-SymbolicName: slf4j.api
20:35:03.951 [main] DEBUG bt.service.ClasspathApplicationService - Processing manifest file: jar:file:/C:/Users/Snijele/.m2/repository/com/github/atomashpolskiy/bt-http-tracker-client/1.3.1/bt-http-tracker-client-1.3.1.jar!/META-INF/MANIFEST.MF
20:35:03.951 [main] DEBUG bt.service.ClasspathApplicationService - Bundle-SymbolicName: null
20:35:03.951 [main] DEBUG bt.service.ClasspathApplicationService - Processing manifest file: jar:file:/C:/Users/Snijele/.m2/repository/org/apache/httpcomponents/httpclient/4.5.2/httpclient-4.5.2.jar!/META-INF/MANIFEST.MF
20:35:03.952 [main] DEBUG bt.service.ClasspathApplicationService - Bundle-SymbolicName: null
20:35:03.952 [main] DEBUG bt.service.ClasspathApplicationService - Processing manifest file: jar:file:/C:/Users/Snijele/.m2/repository/commons-codec/commons-codec/1.9/commons-codec-1.9.jar!/META-INF/MANIFEST.MF
20:35:03.953 [main] DEBUG bt.service.ClasspathApplicationService - Bundle-SymbolicName: org.apache.commons.codec
20:35:03.953 [main] DEBUG bt.service.ClasspathApplicationService - Processing manifest file: jar:file:/C:/Users/Snijele/.m2/repository/com/github/atomashpolskiy/bt-core/1.3.1/bt-core-1.3.1.jar!/META-INF/MANIFEST.MF
20:35:03.954 [main] DEBUG bt.service.ClasspathApplicationService - Bundle-SymbolicName: com.github.atomashpolskiy.bt-core
20:35:03.954 [main] DEBUG bt.service.ClasspathApplicationService - Will read version from manifest file: jar:file:/C:/Users/Snijele/.m2/repository/com/github/atomashpolskiy/bt-core/1.3.1/bt-core-1.3.1.jar!/META-INF/MANIFEST.MF
20:35:03.954 [main] DEBUG bt.service.ClasspathApplicationService - Bt version 1.3
20:35:04.063 [bt.service.executor-thread-1] DEBUG bt.processor.ChainProcessor - Processing next stage: torrent ID (2b32e64f6cd755a9e54d60e205a9681d6670cfae), stage (bt.processor.torrent.CreateSessionStage)
20:35:04.066 [bt.service.executor-thread-1] DEBUG bt.torrent.compiler.MessagingAgentCompiler - Compiling messaging agent type: bt.peerexchange.PeerExchangePeerSourceFactory
20:35:04.067 [bt.service.executor-thread-1] DEBUG bt.torrent.compiler.MessagingAgentCompiler - Compiled producer method: produce
20:35:04.067 [bt.service.executor-thread-1] DEBUG bt.torrent.compiler.MessagingAgentCompiler - Compiled consumer method {consumedType=bt.peerexchange.PeerExchange}: consume
20:35:04.067 [bt.service.executor-thread-1] DEBUG bt.torrent.compiler.MessagingAgentCompiler - Compiled consumer method {consumedType=bt.protocol.extended.ExtendedHandshake}: consume
20:35:04.067 [bt.service.executor-thread-1] DEBUG bt.torrent.compiler.MessagingAgentCompiler - Compiled 3 consumer/producer methods
20:35:04.076 [bt.service.executor-thread-1] DEBUG bt.processor.ChainProcessor - Processing next stage: torrent ID (2b32e64f6cd755a9e54d60e205a9681d6670cfae), stage (bt.processor.magnet.FetchMetadataStage)
20:35:04.076 [bt.service.executor-thread-1] DEBUG bt.torrent.compiler.MessagingAgentCompiler - Compiling messaging agent type: bt.torrent.messaging.MetadataConsumer
20:35:04.077 [bt.service.executor-thread-1] DEBUG bt.torrent.compiler.MessagingAgentCompiler - Compiled producer method: produce
20:35:04.077 [bt.service.executor-thread-1] DEBUG bt.torrent.compiler.MessagingAgentCompiler - Compiled consumer method {consumedType=bt.magnet.UtMetadata}: consume
20:35:04.077 [bt.service.executor-thread-1] DEBUG bt.torrent.compiler.MessagingAgentCompiler - Compiled consumer method {consumedType=bt.protocol.extended.ExtendedHandshake}: consume
20:35:04.077 [bt.service.executor-thread-1] DEBUG bt.torrent.compiler.MessagingAgentCompiler - Compiled 3 consumer/producer methods
20:35:04.077 [bt.service.executor-thread-1] DEBUG bt.torrent.compiler.MessagingAgentCompiler - Compiling messaging agent type: bt.torrent.messaging.BitfieldCollectingConsumer
20:35:04.077 [bt.service.executor-thread-1] DEBUG bt.torrent.compiler.MessagingAgentCompiler - Compiled consumer method {consumedType=bt.protocol.Have}: consume
20:35:04.077 [bt.service.executor-thread-1] DEBUG bt.torrent.compiler.MessagingAgentCompiler - Compiled consumer method {consumedType=bt.protocol.Bitfield}: consume
20:35:04.077 [bt.service.executor-thread-1] DEBUG bt.torrent.compiler.MessagingAgentCompiler - Compiled 2 consumer/producer methods

Am I missing something?

The text was updated successfully, but these errors were encountered:

atomashpolskiy · 2017-08-13T19:42:40Z

Hi @bbijelic ,

You should enable bootstrapping from the public DHT nodes or provide the list of your own. Please see the full usage example in the project's readme. Picture with code is a bit misleading, I need to update it.

bbijelic · 2017-08-14T10:21:46Z

Thanks for suggestion, tried with enabling bootstrapping from public DHT nodes - but no success, problem persists.

// enable multithreaded verification of torrent data
Config config = new Config() {
    @Override
    public int getNumOfHashingThreads() {
        return Runtime.getRuntime().availableProcessors() * 2;
    }
};

// enable bootstrapping from public routers
Module dhtModule = new DHTModule(new DHTConfig() {
    @Override
    public boolean shouldUseRouterBootstrap() {
        return true;
    }
});

String magnetLink = "magnet:?xt=urn:btih:2b32e64f6cd755a9e54d60e205a9681d6670cfae";
Storage storage = new FileSystemStorage(new File("C:\\Users\\Snijele\\Desktop\\TEST").toPath());
BtClient client = Bt.client()
	.config(config)
	.storage(storage)
	.magnet(magnetLink)
	.autoLoadModules()
	.module(dhtModule)
	.build();

// launch
client.startAsync(state -> {
	if (state.getPiecesRemaining() == 0) {
		client.stop();
	}
}, 1000).join();

Tried same for the torrent file case, also no success, although, for torrent file, it shows following in the log, and after that just hangs...:

12:10:35.049 [bt.service.executor-thread-1] DEBUG bt.processor.ChainProcessor - Processing next stage: torrent ID (462101aa44b2e058f6f931bffe8d49531aa40ca8), stage (bt.processor.torrent.ProcessTorrentStage)
12:10:35.088 [bt.service.executor-thread-1] DEBUG bt.tracker.udp.UdpTracker - Executing tracker UDP request of type START: AnnounceRequest{id=-1000147074, torrentId=462101aa44b2e058f6f931bffe8d49531aa40ca8, peerId=[45, 66, 116, 1, 4, 0, 1, 45, -81, 36, 65, 118, -107, 19, -73, -81, 74, 50, -60, 37], downloaded=0, left=0, uploaded=0, eventType=START(2), listeningPort=0, requestString=/announce}
12:10:35.090 [bt.tracker.udp.message-worker] DEBUG bt.tracker.udp.UdpMessageWorker - Creating UDP socket {localAddress=0.0.0.0/0.0.0.0:0}
12:10:35.092 [bt.tracker.udp.message-worker] DEBUG bt.tracker.udp.UdpMessageWorker - Connecting UDP socket {localAddress=0.0.0.0/0.0.0.0:0, remoteAddress=eddie4.nl/87.233.192.220:6969}

I have buit library from source in meantime. Here are my dependencies:

<dependencies>
	<dependency>
		<groupId>com.github.atomashpolskiy</groupId>
		<artifactId>bt-http-tracker-client</artifactId>
		<version>1.4-SNAPSHOT</version>
	</dependency>
	<dependency>
		<groupId>com.github.atomashpolskiy</groupId>
		<artifactId>bt-core</artifactId>
		<version>1.4-SNAPSHOT</version>
	</dependency>
	<dependency>
		<groupId>com.github.atomashpolskiy</groupId>
		<artifactId>bt-dht</artifactId>
		<version>1.4-SNAPSHOT</version>
	</dependency>
</dependencies>

PS. forgot to mention that client Consumer instance is triggered every 1000ms as configured.

atomashpolskiy · 2017-08-14T10:33:39Z

In case with using .torrent file it seems like the client is not discovering any peers. Also, eddie4.nl looks like a private tracker, and if so, then bootstrapping from public DHT will have no effect -- for private torrents only the tracker is queried for peers. Can you confirm if this is a private/public torrent and if the tracker returns any peers?
Full log with trace enabled might be helpful.

In case with using a magnet link, logs show that the metadata has been fetched. This means that:

DHT peer lookup was successful
You've been able to connect to some of the peers and download metadata
You just have to wait until some of the peers unchokes you

atomashpolskiy · 2017-08-14T10:39:10Z

Just tried downloading with CLI:
java -jar bt-launcher.jar -m "magnet:?xt=urn:btih:2b32e64f6cd755a9e54d60e205a9681d6670cfae" -d ~/Downloads/rt
It works:

Downloading World of Warcraft 1.12 Client.rar (5 450 407 230 B)
----------------------------------------------------------------
Elapsed time: 0:02:39       Remaining time: 46:11:09
Peers: 12       Down: 32,0 KB/s

atomashpolskiy · 2017-08-14T10:46:36Z

Updated my initial comment

atomashpolskiy · 2017-08-14T11:06:19Z

DEBUG bt.tracker.udp.UdpTracker - Executing tracker UDP request of type START: AnnounceRequest{..., listeningPort=0, ...}

Oh, this is actually a bug, client should provide the actual listening port to the UDP tracker.. I just pushed to the fix to master.

This might have affected peer discovery in case with .torrent file (effectively preventing other peers from connecting to you, if they discovered you via the tracker). It's unlikely to be the root cause of your problem though.

bbijelic · 2017-08-14T12:08:28Z

FYI, finally managed to run it successfully via magnet link (torrent file case still to be checked) :)

Verified that CLI app is downloading via magnet link (on CentOS VM)
Running CLI on Windows is throwing exception:

Exception in thread "main" java.lang.RuntimeException: Failed to create terminal
        at bt.cli.SessionStatePrinter.<init>(SessionStatePrinter.java:106)
        at bt.cli.SessionStatePrinter$1.<init>(SessionStatePrinter.java:40)
        at bt.cli.SessionStatePrinter.createKeyInputAwarePrinter(SessionStatePrinter.java:40)
        at bt.cli.CliClient.<init>(CliClient.java:131)
        at bt.cli.CliClient.main(CliClient.java:56)
Caused by: java.io.IOException: Cannot run program "/bin/stty": CreateProcess error=2, The system cannot find the file specified
        at java.lang.ProcessBuilder.start(Unknown Source)
        at com.googlecode.lanterna.terminal.ansi.UnixLikeTTYTerminal.exec(UnixLikeTTYTerminal.java:169)
        at com.googlecode.lanterna.terminal.ansi.UnixLikeTTYTerminal.saveTerminalSettings(UnixLikeTTYTerminal.java:117)
        at com.googlecode.lanterna.terminal.ansi.UnixLikeTerminal.aquire(UnixLikeTerminal.java:79)
        at com.googlecode.lanterna.terminal.ansi.UnixLikeTTYTerminal.realAquire(UnixLikeTTYTerminal.java:87)
        at com.googlecode.lanterna.terminal.ansi.UnixLikeTTYTerminal.<init>(UnixLikeTTYTerminal.java:78)
        at com.googlecode.lanterna.terminal.ansi.CygwinTerminal.<init>(CygwinTerminal.java:59)
        at com.googlecode.lanterna.terminal.DefaultTerminalFactory.createCygwinTerminal(DefaultTerminalFactory.java:293)
        at com.googlecode.lanterna.terminal.DefaultTerminalFactory.createWindowsTerminal(DefaultTerminalFactory.java:288)
        at com.googlecode.lanterna.terminal.DefaultTerminalFactory.createTerminal(DefaultTerminalFactory.java:98)
        at bt.cli.SessionStatePrinter.<init>(SessionStatePrinter.java:93)
        ... 4 more
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
        at java.lang.ProcessImpl.create(Native Method)
        at java.lang.ProcessImpl.<init>(Unknown Source)
        at java.lang.ProcessImpl.start(Unknown Source)
        ... 15 more

Since CLI is working, I have checked into source code to see how it is actually setup there, and did the same. First initialized BtRuntime and then passed it into the client builder:

// enable multithreaded verification of torrent data
Config config = new Config() {

	@Override
	public int getNumOfHashingThreads() {
		return Runtime.getRuntime().availableProcessors();
	}
};

// enable bootstrapping from public routers
Module dhtModule = new DHTModule(new DHTConfig() {
	@Override
	public boolean shouldUseRouterBootstrap() {
		return true;
	}

});

// Runtime
BtRuntime runtime = BtRuntime.builder(config).module(dhtModule).autoLoadModules().disableAutomaticShutdown()
		.build();

// Magnet link
String magnetLink = "magnet:?xt=urn:btih:2b32e64f6cd755a9e54d60e205a9681d6670cfae";

// Storage
Storage storage = new FileSystemStorage(new File("C:\\Users\\Snijele\\Desktop\\TEST\\").toPath());

// Client
BtClient client = Bt.client(runtime).storage(storage).magnet(magnetLink).build();

// launch
client.startAsync(state -> {
	if (state.getPiecesRemaining() == 0) {
		client.stop();
	}
}, 1000).join();

and this works like a charm. :)

Summary:

Example in README file does not work
Building client without passing BtRuntime does not start download

Thank you for your support!
Cheers mate!

#34 (plus internal refactoring of client builders)

atomashpolskiy · 2017-08-14T18:58:18Z

I tracked down the cause, turned out that the standalone client was not initializing the runtime properly due to a programming error. I've pushed the fix along with some internal refactoring. Public API haven't changed.

It's good that you've reported the bug, it was really subtle, and I can only wonder how many folks had run into the same problem before :)

Running CLI on Windows is throwing exception

Yeah, it's actually a known issue. GUI framework that I'm using does not support Windows, so the CLI should be run in headless mode there (-H flag)

Considering your latest code: you might want to enable automatic runtime shutdown if the download is one-off task (i.e. you are not starting multiple clients in the same runtime). Otherwise the runtime will keep on wasting cycles (peer lookups, DHT agent, message loop, etc.) and will terminate only on JVM shutdown.

Again, thanks for reporting and for your patience, user feedback is very important to me.

atomashpolskiy changed the title ~~Download not starting by using torrent file or magnet link, help needed~~ Download not starting when using standalone client with private runtime Aug 14, 2017

atomashpolskiy added a commit that referenced this issue Aug 14, 2017

Download not starting when using standalone client with private runtime

6ca586a

#34 (plus internal refactoring of client builders)

atomashpolskiy added the bug label Aug 14, 2017

atomashpolskiy added this to the 1.4 milestone Aug 14, 2017

atomashpolskiy closed this as completed Aug 17, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Download not starting when using standalone client with private runtime #34

Download not starting when using standalone client with private runtime #34

bbijelic commented Aug 13, 2017

atomashpolskiy commented Aug 13, 2017

bbijelic commented Aug 14, 2017 •

edited

Loading

atomashpolskiy commented Aug 14, 2017 •

edited

Loading

atomashpolskiy commented Aug 14, 2017

atomashpolskiy commented Aug 14, 2017

atomashpolskiy commented Aug 14, 2017

bbijelic commented Aug 14, 2017 •

edited

Loading

atomashpolskiy commented Aug 14, 2017

Download not starting when using standalone client with private runtime #34

Download not starting when using standalone client with private runtime #34

Comments

bbijelic commented Aug 13, 2017

atomashpolskiy commented Aug 13, 2017

bbijelic commented Aug 14, 2017 • edited Loading

atomashpolskiy commented Aug 14, 2017 • edited Loading

atomashpolskiy commented Aug 14, 2017

atomashpolskiy commented Aug 14, 2017

atomashpolskiy commented Aug 14, 2017

bbijelic commented Aug 14, 2017 • edited Loading

atomashpolskiy commented Aug 14, 2017

bbijelic commented Aug 14, 2017 •

edited

Loading

atomashpolskiy commented Aug 14, 2017 •

edited

Loading

bbijelic commented Aug 14, 2017 •

edited

Loading