Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse code

Initial import.

  • Loading branch information...
commit 76430c0e79790a08d88cd1f0ee892e40d7a2f164 0 parents
Juliusz Chroboczek authored

Showing 6 changed files with 2,601 additions and 0 deletions. Show diff stats Hide diff stats

  1. +19 0 LICENCE
  2. +9 0 Makefile
  3. +182 0 README
  4. +311 0 dht-example.c
  5. +2,031 0 dht.c
  6. +49 0 dht.h
19 LICENCE
... ... @@ -0,0 +1,19 @@
  1 +Copyright (c) 2009 by Juliusz Chroboczek
  2 +
  3 +Permission is hereby granted, free of charge, to any person obtaining a copy
  4 +of this software and associated documentation files (the "Software"), to deal
  5 +in the Software without restriction, including without limitation the rights
  6 +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
  7 +copies of the Software, and to permit persons to whom the Software is
  8 +furnished to do so, subject to the following conditions:
  9 +
  10 +The above copyright notice and this permission notice shall be included in
  11 +all copies or substantial portions of the Software.
  12 +
  13 +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
  14 +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
  15 +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
  16 +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
  17 +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
  18 +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  19 +THE SOFTWARE.
9 Makefile
... ... @@ -0,0 +1,9 @@
  1 +CFLAGS = -g -Wall -DHAS_STDINT_H
  2 +LDLIBS = -lcrypt
  3 +
  4 +dht-example: dht-example.o dht.o
  5 +
  6 +all: dht-example
  7 +
  8 +clean:
  9 + -rm -f dht-example dht-example.o dht-example.id dht.o *~ core
182 README
... ... @@ -0,0 +1,182 @@
  1 +The files dht.c and dht.h implement the variant of the Kademlia Distributed
  2 +Hash Table (DHT) used in the Bittorrent network (``mainline'' variant).
  3 +
  4 +The file dht-example.c is a stand-alone program that participates in the
  5 +DHT. Another example is a patch against Transmission, which you might or
  6 +might not be able to find somewhere.
  7 +
  8 +The code is designed to work well in both event-driven and threaded code.
  9 +The caller, which is either an event-loop or a dedicated thread, must
  10 +periodically call the function dht_periodic. In addition, it must call
  11 +dht_periodic whenever any data has arrived from the network.
  12 +
  13 +All functions return -1 in case of failure, in which case errno is set, or
  14 +a positive value in case of success.
  15 +
  16 +Initialisation
  17 +**************
  18 +
  19 +* dht_init
  20 +
  21 +This must be called before using the library. You pass it a bound IPv4
  22 +datagram socket, and your node id, a 20-octet array that should be globally
  23 +unique.
  24 +
  25 +Node ids must be well distributed, so you cannot just use your Bittorrent
  26 +id; you should either generate a truly random value (using plenty of
  27 +entropy), or at least take the SHA-1 of something. However, it is a good
  28 +idea to keep the id stable, so you may want to store it in stable storage
  29 +at client shutdown.
  30 +
  31 +* dht_uninit
  32 +
  33 +This may be called at the end of the session. If dofree is true, it frees
  34 +all the memory allocated for the DHT. If dofree is false, this function
  35 +currently does nothing.
  36 +
  37 +Bootstrapping
  38 +*************
  39 +
  40 +The DHT needs to be taught a small number of contacts to begin functioning.
  41 +You can hard-wire a small number of stable nodes in your application, but
  42 +this obviously fails to scale. You may save the list of known good nodes
  43 +at shutdown, and restore it at restart. You may also grab nodes from
  44 +torrent files (the nodes field), and you may exchange contacts with other
  45 +Bittorrent peers using the PORT extension.
  46 +
  47 +* dht_ping
  48 +
  49 +This is the main bootstrapping primitive. You pass it an address at which
  50 +you believe that a DHT node may be living, and a query will be sent. If
  51 +a node replies, and if there is space in the routing table, it will be
  52 +inserted.
  53 +
  54 +* dht_insert_node
  55 +
  56 +This is a softer bootstrapping method, which doesn't actually send
  57 +a query -- it only stores the node in the routing table for later use. It
  58 +is a good idea to use that when e.g. restoring your routing table from
  59 +disk.
  60 +
  61 +Note that dht_insert_node requires that you supply a node id. If the id
  62 +turns out to be wrong, the DHT will eventually recover; still, inserting
  63 +massive amounts of incorrect information into your routing table is
  64 +certainly not a good idea.
  65 +
  66 +An additionaly difficulty with dht_insert_node is that, for various
  67 +reasons, a Kademlia routing table cannot absorb nodes faster than a certain
  68 +rate. Dumping a large number of nodes into a table using dht_insert_node
  69 +will probably cause most of these nodes to be discarded straight away.
  70 +(The tolerable rate is difficult to estimate; it is probably on the order
  71 +of one node every few seconds per node already in the table divided by 8,
  72 +for some suitable value of 8.)
  73 +
  74 +Doing some work
  75 +***************
  76 +
  77 +* dht_periodic
  78 +
  79 +This function should be called by your main loop periodically, and also
  80 +whenever data is available on the socket. The time after which
  81 +dht_periodic should be called if no data is available is returned in the
  82 +parameter tosleep. (You do not need to be particularly accurate; actually,
  83 +it is a good idea to be late by a random value.)
  84 +
  85 +The parameter available indicates whether any data is available on the
  86 +socket. If it is 0, dht_periodic will not try to read data; if it is 1, it
  87 +will.
  88 +
  89 +Dht_periodic also takes a callback, which will be called whenever something
  90 +interesting happens (see below).
  91 +
  92 +* dht_search
  93 +
  94 +This schedules a search for information about the info-hash specified in
  95 +id. If port is not 0, it specifies the TCP port on which the current peer
  96 +is litening; in that case, when the search is complete it will be announced
  97 +to the network. The port is in host order, beware if you got it from
  98 +a struct sockaddr_in.
  99 +
  100 +In either case, data is passed to the callback function as soon as it is
  101 +available, possibly in multiple pieces. The callback function will
  102 +additionally be called when the search is complete.
  103 +
  104 +Up to DHT_MAX_SEARCHES (20) searches can be in progress at a given time;
  105 +any more, and dht_search will return -1. If you specify a new search for
  106 +the same info hash as a search still in progress, the previous search is
  107 +combined with the new one -- you will only receive a completion indication
  108 +once.
  109 +
  110 +Information queries
  111 +*******************
  112 +
  113 +* dht_nodes
  114 +
  115 +This returns the number of known good, dubious and cached nodes in our
  116 +routing table. This can be used to decide whether it's reasonable to start
  117 +a search; a search is likely to be successful as long as we have a few good
  118 +nodes; however, in order to avoid overloading your bootstrap nodes, you may
  119 +want to wait until good is at least 4 and good + doubtful is at least 30 or
  120 +so.
  121 +
  122 +If you want to display a single figure to the user, you should display good
  123 ++ doubtful, which is the total number of nodes in your routing table. Some
  124 +clients try to estimate the total number of nodes, but this doesn't make
  125 +much sense -- since the result is exponential in the number of nodes in the
  126 +routing table, small variations in the latter cause huge jumps in the
  127 +former.
  128 +
  129 +* dht_dump_tables
  130 +* dht_debug
  131 +
  132 +These are debugging aids.
  133 +
  134 +Functions provided by you
  135 +*************************
  136 +
  137 +* The callback function
  138 +
  139 +The callback function is called with 5 arguments. Closure is simply the
  140 +value that you passed to dht_periodic. Event is one of DHT_EVENT_VALUES,
  141 +which indicates that we have new values, or DHT_EVENT_SEARCH_DONE, which
  142 +indicates that a search has completed. In either case, info_hash is set to
  143 +the info-hash of the search.
  144 +
  145 +In the case of DHT_EVENT_VALUES, data is a list of nodes in ``compact''
  146 +format -- 6 bytes per node, 4 for the IP address and 2 for the port. It's
  147 +length in bytes is in data_len.
  148 +
  149 +* dht_hash
  150 +
  151 +This should compute a reasonably strong cryptographic hash of the passed
  152 +values. It should map cleanly to your favourite crypto toolkit's MD5 or
  153 +SHA-1 function.
  154 +
  155 +Final notes
  156 +***********
  157 +
  158 +* NAT
  159 +
  160 +Nothing works well across NATs, but Kademlia is somewhat less impacted than
  161 +many other protocols. The implementation takes care to distinguish between
  162 +unidirectional and bidirectional reachability, and NATed nodes will
  163 +eventually fall out from other nodes' routing tables.
  164 +
  165 +While there is no periodic pinging in this implementation, maintaining
  166 +a full routing table requires slightly more than one packet exchange per
  167 +minute, even in a completely idle network; this should be sufficient to
  168 +make most full cone NATs happy.
  169 +
  170 +* Missing functionality
  171 +
  172 +Some of the code has had very little testing. If it breaks, you get to
  173 +keep both pieces.
  174 +
  175 +There is currently no good way to save and restore your routing table.
  176 +
  177 +IPv6 support is deliberately not included: designing a double-stack
  178 +distributed hash table raises some tricky issues, and doing it naively may
  179 +break connectivity for everyone.
  180 +
  181 + Juliusz Chroboczek
  182 + <jch@pps.jussieu.fr>
311 dht-example.c
... ... @@ -0,0 +1,311 @@
  1 +/* This example code was written by Juliusz Chroboczek.
  2 + You are free to cut'n'paste from it to your heart's content. */
  3 +
  4 +/* For crypt */
  5 +#define _GNU_SOURCE
  6 +
  7 +#include <stdio.h>
  8 +#include <stdlib.h>
  9 +#include <errno.h>
  10 +#include <string.h>
  11 +#include <unistd.h>
  12 +#include <fcntl.h>
  13 +#include <sys/time.h>
  14 +#include <arpa/inet.h>
  15 +#include <sys/types.h>
  16 +#include <sys/socket.h>
  17 +#include <sys/signal.h>
  18 +
  19 +#include "dht.h"
  20 +
  21 +#define MAX_BOOTSTRAP_NODES 20
  22 +static struct sockaddr_in bootstrap_nodes[MAX_BOOTSTRAP_NODES];
  23 +static int num_bootstrap_nodes = 0;
  24 +
  25 +static volatile sig_atomic_t dumping = 0, searching = 0, exiting = 0;
  26 +
  27 +static void
  28 +sigdump(int signo)
  29 +{
  30 + dumping = 1;
  31 +}
  32 +
  33 +static void
  34 +sigtest(int signo)
  35 +{
  36 + searching = 1;
  37 +}
  38 +
  39 +static void
  40 +sigexit(int signo)
  41 +{
  42 + exiting = 1;
  43 +}
  44 +
  45 +static void
  46 +init_signals(void)
  47 +{
  48 + struct sigaction sa;
  49 + sigset_t ss;
  50 +
  51 + sigemptyset(&ss);
  52 + sa.sa_handler = sigdump;
  53 + sa.sa_mask = ss;
  54 + sa.sa_flags = 0;
  55 + sigaction(SIGUSR1, &sa, NULL);
  56 +
  57 + sigemptyset(&ss);
  58 + sa.sa_handler = sigtest;
  59 + sa.sa_mask = ss;
  60 + sa.sa_flags = 0;
  61 + sigaction(SIGUSR2, &sa, NULL);
  62 +
  63 + sigemptyset(&ss);
  64 + sa.sa_handler = sigexit;
  65 + sa.sa_mask = ss;
  66 + sa.sa_flags = 0;
  67 + sigaction(SIGINT, &sa, NULL);
  68 +}
  69 +
  70 +const unsigned char hash[20] = {
  71 + 0x54, 0x57, 0x87, 0x89, 0xdf, 0xc4, 0x23, 0xee, 0xf6, 0x03,
  72 + 0x1f, 0x81, 0x94, 0xa9, 0x3a, 0x16, 0x98, 0x8b, 0x72, 0x7b
  73 +};
  74 +
  75 +/* The call-back function is called by the DHT whenever something
  76 + interesting happens. Right now, it only happens when we get a new value or
  77 + when a search completes, but this may be extended in future versions. */
  78 +static void
  79 +callback(void *closure,
  80 + int event,
  81 + unsigned char *info_hash,
  82 + void *data, size_t data_len)
  83 +{
  84 + if(event == DHT_EVENT_SEARCH_DONE)
  85 + printf("Search done.\n");
  86 + else if(event == DHT_EVENT_VALUES)
  87 + printf("Received %d values.\n", (int)(data_len / 6));
  88 +}
  89 +
  90 +int
  91 +main(int argc, char **argv)
  92 +{
  93 + int i, rc, fd;
  94 + int s, port;
  95 + int have_id = 0;
  96 + unsigned char myid[20];
  97 + time_t tosleep = 0;
  98 +
  99 + /* Ids need to be distributed evenly, so you cannot just use your
  100 + bittorrent id. Either generate it randomly, or take the SHA-1 of
  101 + something. */
  102 + fd = open("dht-example.id", O_RDONLY);
  103 + if(fd >= 0) {
  104 + rc = read(fd, myid, 20);
  105 + if(rc == 20)
  106 + have_id = 1;
  107 + close(fd);
  108 + }
  109 +
  110 + if(!have_id) {
  111 + fd = open("/dev/urandom", O_RDONLY);
  112 + if(fd < 0) {
  113 + perror("open(random)");
  114 + exit(1);
  115 + }
  116 + rc = read(fd, myid, 20);
  117 + if(rc < 0) {
  118 + perror("read(random)");
  119 + exit(1);
  120 + }
  121 + have_id = 1;
  122 + close(fd);
  123 +
  124 + fd = open("dht-example.id", O_WRONLY | O_CREAT | O_TRUNC, 0666);
  125 + if(fd >= 0) {
  126 + rc = write(fd, myid, 20);
  127 + if(rc < 20)
  128 + unlink("dht-example.id");
  129 + close(fd);
  130 + }
  131 + }
  132 +
  133 + if(argc < 2)
  134 + goto usage;
  135 +
  136 + i = 1;
  137 +
  138 + if(argc < i + 1)
  139 + goto usage;
  140 +
  141 + port = atoi(argv[i++]);
  142 + if(port <= 0 || port >= 0x10000)
  143 + goto usage;
  144 +
  145 + while(i < argc) {
  146 + struct sockaddr_in sin;
  147 + if(num_bootstrap_nodes >= MAX_BOOTSTRAP_NODES)
  148 + goto usage;
  149 + memset(&sin, 0, sizeof(sin));
  150 + sin.sin_family = PF_INET;
  151 + rc = inet_pton(AF_INET, argv[i], &sin.sin_addr);
  152 + if(rc == 1) {
  153 + i++; if(i >= argc) goto usage;
  154 + sin.sin_port = htons(atoi(argv[i]));
  155 + if(sin.sin_port == 0) goto usage;
  156 + i++;
  157 + bootstrap_nodes[num_bootstrap_nodes] = sin;
  158 + num_bootstrap_nodes++;
  159 + } else {
  160 + goto usage;
  161 + }
  162 + }
  163 +
  164 + if(i < argc)
  165 + goto usage;
  166 +
  167 + /* If you set dht_debug to a stream, every action taken by the DHT will
  168 + be logged. */
  169 + dht_debug = stdout;
  170 +
  171 + /* We need an IPv4 socket, bound to a stable port. Rumour has it that
  172 + uTorrent works better when it is the same as your Bittorrent port. */
  173 + s = socket(PF_INET, SOCK_DGRAM, 0);
  174 + if(s < 0) {
  175 + perror("socket");
  176 + exit(1);
  177 + }
  178 +
  179 + {
  180 + struct sockaddr_in sin;
  181 + memset(&sin, 0, sizeof(sin));
  182 + sin.sin_family = AF_INET;
  183 + sin.sin_port = htons(port);
  184 + rc = bind(s, (struct sockaddr*)&sin, sizeof(sin));
  185 + if(rc < 0) {
  186 + perror("bind");
  187 + exit(1);
  188 + }
  189 + }
  190 +
  191 + /* Init the dht. This sets the socket into non-blocking mode. */
  192 + rc = dht_init(s, myid);
  193 + if(rc < 0) {
  194 + perror("dht_init");
  195 + exit(1);
  196 + }
  197 +
  198 + init_signals();
  199 +
  200 + /* For bootstrapping, we need an initial list of nodes. This could be
  201 + hard-wired, but can also be obtained from the nodes key of a torrent
  202 + file, or from the PORT bittorrent message.
  203 +
  204 + Dht_ping_node is the brutal way of bootstrapping -- it actually
  205 + sends a message to the peer. If you're going to bootstrap from
  206 + a massive number of nodes (for example because you're restoring from
  207 + a dump) and you already know their ids, it's better to use
  208 + dht_insert_node. If the ids are incorrect, the DHT will recover. */
  209 + for(i = 0; i < num_bootstrap_nodes; i++) {
  210 + dht_ping_node(s, &bootstrap_nodes[i]);
  211 + usleep(random() % 100000);
  212 + }
  213 +
  214 + while(1) {
  215 + struct timeval tv;
  216 + fd_set readfds;
  217 + tv.tv_sec = tosleep;
  218 + tv.tv_usec = random() % 1000000;
  219 +
  220 + FD_ZERO(&readfds);
  221 + FD_SET(s, &readfds);
  222 + rc = select(s + 1, &readfds, NULL, NULL, &tv);
  223 + if(rc < 0) {
  224 + if(errno != EINTR) {
  225 + perror("select");
  226 + sleep(1);
  227 + }
  228 + }
  229 +
  230 + if(exiting)
  231 + break;
  232 +
  233 + rc = dht_periodic(s, rc > 0, &tosleep, callback, NULL);
  234 + if(rc < 0) {
  235 + if(errno == EINTR) {
  236 + continue;
  237 + } else {
  238 + perror("dht_periodic");
  239 + if(rc == EINVAL || rc == EFAULT)
  240 + abort();
  241 + tosleep = 1;
  242 + }
  243 + }
  244 +
  245 + /* This is how you trigger a search for a torrent hash. If port
  246 + (the third argument) is non-zero, it also performs an announce.
  247 + Since peers expire announced data after 30 minutes, it's a good
  248 + idea to reannounce every 14 minutes or so. */
  249 + if(searching) {
  250 + dht_search(s, hash, 0, callback, NULL);
  251 + searching = 0;
  252 + }
  253 +
  254 + /* For debugging, or idle curiosity. */
  255 + if(dumping) {
  256 + dht_dump_tables(stdout);
  257 + dumping = 0;
  258 + }
  259 + }
  260 +
  261 + dht_uninit(s, 0);
  262 + return 0;
  263 +
  264 + usage:
  265 + fprintf(stderr, "Foo!\n");
  266 + exit(1);
  267 +}
  268 +
  269 +/* We need to provide a reasonably strong cryptographic hashing function.
  270 + Here's how we'd do it if we had RSA's MD5 code. */
  271 +#if 0
  272 +void
  273 +dht_hash(void *hash_return, int hash_size,
  274 + const void *v1, int len1,
  275 + const void *v2, int len2,
  276 + const void *v3, int len3)
  277 +{
  278 + static MD5_CTX ctx;
  279 + MD5Init(&ctx);
  280 + MD5Update(&ctx, v1, len1);
  281 + MD5Update(&ctx, v2, len2);
  282 + MD5Update(&ctx, v3, len3);
  283 + MD5Final(&ctx);
  284 + if(hash_size > 16)
  285 + memset((char*)hash_return + 16, 0, hash_size - 16);
  286 + memcpy(hash_return, ctx.digest, hash_size > 16 ? 16 : hash_size);
  287 +}
  288 +#else
  289 +/* But for this example, we might as well use something weaker. */
  290 +void
  291 +dht_hash(void *hash_return, int hash_size,
  292 + const void *v1, int len1,
  293 + const void *v2, int len2,
  294 + const void *v3, int len3)
  295 +{
  296 + const char *c1 = v1, *c2 = v2, *c3 = v3;
  297 + char key[9]; /* crypt is limited to 8 characters */
  298 + int i;
  299 +
  300 + memset(key, 0, 9);
  301 +#define CRYPT_HAPPY(c) ((c % 0x60) + 0x20)
  302 +
  303 + for(i = 0; i < 2 && i < len1; i++)
  304 + key[i] = CRYPT_HAPPY(c1[i]);
  305 + for(i = 0; i < 4 && i < len1; i++)
  306 + key[2 + i] = CRYPT_HAPPY(c2[i]);
  307 + for(i = 0; i < 2 && i < len1; i++)
  308 + key[6 + i] = CRYPT_HAPPY(c3[i]);
  309 + strncpy(hash_return, crypt(key, "jc"), hash_size);
  310 +}
  311 +#endif
2,031 dht.c
... ... @@ -0,0 +1,2031 @@
  1 +/*
  2 +Copyright (c) 2009 by Juliusz Chroboczek
  3 +
  4 +Permission is hereby granted, free of charge, to any person obtaining a copy
  5 +of this software and associated documentation files (the "Software"), to deal
  6 +in the Software without restriction, including without limitation the rights
  7 +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
  8 +copies of the Software, and to permit persons to whom the Software is
  9 +furnished to do so, subject to the following conditions:
  10 +
  11 +The above copyright notice and this permission notice shall be included in
  12 +all copies or substantial portions of the Software.
  13 +
  14 +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
  15 +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
  16 +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
  17 +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
  18 +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
  19 +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  20 +THE SOFTWARE.
  21 +*/
  22 +
  23 +/* Please, please, please.
  24 +
  25 + You are welcome to integrate this code in your favourite Bittorrent
  26 + client. Please remember, however, that it is meant to be usable by
  27 + others, including myself. This means no C++, no relicensing, and no
  28 + gratuitious changes to the coding style. And please send back any
  29 + improvements to the author. */
  30 +
  31 +/* For memmem. */
  32 +#define _GNU_SOURCE
  33 +
  34 +#include <stdio.h>
  35 +#include <stdlib.h>
  36 +#include <errno.h>
  37 +#include <string.h>
  38 +#include <stdarg.h>
  39 +#include <unistd.h>
  40 +#include <fcntl.h>
  41 +#include <sys/time.h>
  42 +#include <arpa/inet.h>
  43 +#include <sys/types.h>
  44 +#include <sys/socket.h>
  45 +
  46 +#include "dht.h"
  47 +
  48 +#ifndef MSG_CONFIRM
  49 +#define MSG_CONFIRM 0
  50 +#endif
  51 +
  52 +/* We set sin_family to 0 to mark unused slots. */
  53 +#if AF_INET == 0
  54 +#error You lose
  55 +#endif
  56 +
  57 +#if defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L
  58 +/* nothing */
  59 +#elif defined(__GNUC__)
  60 +#define inline __inline
  61 +#if (__GNUC__ >= 3)
  62 +#define restrict __restrict
  63 +#else
  64 +#define restrict /**/
  65 +#endif
  66 +#else
  67 +#define inline /**/
  68 +#define restrict /**/
  69 +#endif
  70 +
  71 +struct node {
  72 + unsigned char id[20];
  73 + struct sockaddr_in sin;
  74 + time_t time; /* time of last message received */
  75 + time_t reply_time; /* time of last correct reply received */
  76 + time_t pinged_time; /* time of last request */
  77 + int pinged; /* how many requests we sent since last reply */
  78 + struct node *next;
  79 +};
  80 +
  81 +struct bucket {
  82 + unsigned char first[20];
  83 + int count; /* number of nodes */
  84 + int time; /* time of last reply in this bucket */
  85 + struct node *nodes;
  86 + struct sockaddr_in cached; /* the address of a likely candidate */
  87 + struct bucket *next;
  88 +};
  89 +
  90 +struct search_node {
  91 + unsigned char id[20];
  92 + struct sockaddr_in sin;
  93 + time_t request_time; /* the time of the last unanswered request */
  94 + time_t reply_time; /* the time of the last reply */
  95 + int pinged;
  96 + unsigned char token[40];
  97 + int token_len;
  98 + int replied; /* whether we have received a reply */
  99 + int acked; /* whether they acked our announcement */
  100 +};
  101 +
  102 +struct search {
  103 + unsigned short tid;
  104 + time_t step_time; /* the time of the last search_step */
  105 + unsigned char id[20];
  106 + unsigned short port; /* 0 for pure searches */
  107 + int done;
  108 + struct search_node nodes[8];
  109 + int numnodes;
  110 +};
  111 +
  112 +struct peer {
  113 + time_t time;
  114 + unsigned char ip[4];
  115 + unsigned short port;
  116 +};
  117 +
  118 +/* The maximum number of peers we store for a given hash. */
  119 +#ifndef DHT_MAX_PEERS
  120 +#define DHT_MAX_PEERS 2048
  121 +#endif
  122 +
  123 +struct storage {
  124 + unsigned char id[20];
  125 + int numpeers;
  126 + int maxpeers;
  127 + struct peer *peers;
  128 + struct storage *next;
  129 +};
  130 +
  131 +static int send_ping(int s, struct sockaddr *sa, int salen,
  132 + const unsigned char *tid, int tid_len);
  133 +static int send_pong(int s, struct sockaddr *sa, int salen,
  134 + const unsigned char *tid, int tid_len);
  135 +static int send_find_node(int s, struct sockaddr *sa, int salen,
  136 + const unsigned char *tid, int tid_len,
  137 + const unsigned char *target, int confirm);
  138 +static int send_found_nodes(int s, struct sockaddr *sa, int salen,
  139 + const unsigned char *tid, int tid_len,
  140 + const unsigned char *nodes, int nodes_len,
  141 + const unsigned char *token, int token_len);
  142 +static int send_bucket_nodes(int s, struct sockaddr *sa, int salen,
  143 + const unsigned char *tid, int tid_len,
  144 + struct bucket *b,
  145 + const unsigned char *token, int token_len);
  146 +static int send_get_peers(int s, struct sockaddr *sa, int salen,
  147 + unsigned char *tid, int tid_len,
  148 + unsigned char *infohash, unsigned short port,
  149 + int confirm);
  150 +static int send_announce_peer(int s, struct sockaddr *sa, int salen,
  151 + unsigned char *tid, int tid_len,
  152 + unsigned char *infohas, unsigned short port,
  153 + unsigned char *token, int token_len, int confirm);
  154 +int send_peers_found(int s, struct sockaddr *sa, int salen,
  155 + unsigned char *tid, int tid_len,
  156 + struct peer *peers, int numpeers,
  157 + unsigned char *token, int token_len);
  158 +int send_peer_announced(int s, struct sockaddr *sa, int salen,
  159 + unsigned char *tid, int tid_len);
  160 +
  161 +#define REPLY 0
  162 +#define PING 1
  163 +#define FIND_NODE 2
  164 +#define GET_PEERS 3
  165 +#define ANNOUNCE_PEER 4
  166 +static int parse_message(const unsigned char *buf, int buflen,
  167 + unsigned char *tid_return, int *tid_len,
  168 + unsigned char *id_return,
  169 + unsigned char *info_hash_return,
  170 + unsigned char *target_return,
  171 + unsigned short *port_return,
  172 + unsigned char *token_return, int *token_len,
  173 + unsigned char *nodes_return, int *nodes_len,
  174 + const unsigned char *values_return, int *values_len);
  175 +
  176 +static const unsigned char zeroes[20] = {0};
  177 +static const unsigned char ones[20] = {
  178 + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
  179 + 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
  180 + 0xFF, 0xFF, 0xFF, 0xFF
  181 +};
  182 +static time_t search_time;
  183 +static time_t confirm_nodes_time;
  184 +static time_t rotate_secrets_time;
  185 +
  186 +static unsigned char myid[20];
  187 +static unsigned char secret[8];
  188 +static unsigned char oldsecret[8];
  189 +
  190 +static struct bucket *buckets = NULL;
  191 +static struct storage *storage;
  192 +
  193 +/* The maximum number of concurrent searches. */
  194 +#ifndef DHT_MAX_SEARCHES
  195 +#define DHT_MAX_SEARCHES 20
  196 +#endif
  197 +
  198 +static struct search searches[DHT_MAX_SEARCHES];
  199 +static int numsearches;
  200 +static unsigned short search_id;
  201 +
  202 +/* The maximum number of nodes that we snub. There is probably little
  203 + reason to increase this value. */
  204 +#ifndef DHT_MAX_BLACKLISTED
  205 +#define DHT_MAX_BLACKLISTED 10
  206 +#endif
  207 +static struct sockaddr_in blacklist[DHT_MAX_BLACKLISTED];
  208 +int next_blacklisted;
  209 +
  210 +static struct timeval now;
  211 +static time_t mybucket_grow_time;
  212 +static time_t expire_stuff_time;
  213 +
  214 +FILE *dht_debug = NULL;
  215 +
  216 +#ifdef __GNUC__
  217 + __attribute__ ((format (printf, 1, 2)))
  218 +#endif
  219 +static void
  220 +debugf(const char *format, ...)
  221 +{
  222 + va_list args;
  223 + va_start(args, format);
  224 + if(dht_debug)
  225 + vfprintf(dht_debug, format, args);
  226 + va_end(args);
  227 + fflush(dht_debug);
  228 +}
  229 +
  230 +static void
  231 +debug_printable(const unsigned char *buf, int buflen)
  232 +{
  233 + int i;
  234 + if(dht_debug) {
  235 + for(i = 0; i < buflen; i++)
  236 + putc(buf[i] >= 32 && buf[i] <= 126 ? buf[i] : '.', dht_debug);
  237 + }
  238 +}
  239 +
  240 +static void
  241 +print_hex(FILE *f, const unsigned char *buf, int buflen)
  242 +{
  243 + int i;
  244 + for(i = 0; i < buflen; i++)
  245 + fprintf(f, "%02x", buf[i]);
  246 +}
  247 +
  248 +/* Forget about the ``XOR-metric''. An id is just a path from the
  249 + root of the tree, so bits are numbered from the start. */
  250 +
  251 +static inline int
  252 +id_cmp(const unsigned char *restrict id1, const unsigned char *restrict id2)
  253 +{
  254 + /* Memcmp is guaranteed to perform an unsigned comparison. */
  255 + return memcmp(id1, id2, 20);
  256 +}
  257 +
  258 +/* Find the lowest 1 bit in an id. */
  259 +static int
  260 +lowbit(const unsigned char *id)
  261 +{
  262 + int i, j;
  263 + for(i = 19; i >= 0; i--)
  264 + if(id[i] != 0)
  265 + break;
  266 +
  267 + if(i < 0)
  268 + return -1;
  269 +
  270 + for(j = 7; j >= 0; j--)
  271 + if((id[i] & (0x80 >> j)) != 0)
  272 + break;
  273 +
  274 + return 8 * i + j;
  275 +}
  276 +
  277 +/* Find how many bits two ids have in common. */
  278 +static int
  279 +common_bits(const unsigned char *id1, const unsigned char *id2)
  280 +{
  281 + int i, j;
  282 + unsigned char xor;
  283 + for(i = 0; i < 20; i++) {
  284 + if(id1[i] != id2[i])
  285 + break;
  286 + }
  287 +
  288 + if(i == 20)
  289 + return 160;
  290 +
  291 + xor = id1[i] ^ id2[i];
  292 +
  293 + j = 0;
  294 + while((xor & 0x80) == 0) {
  295 + xor <<= 1;
  296 + j++;
  297 + }
  298 +
  299 + return 8 * i + j;
  300 +}
  301 +
  302 +/* We keep buckets in a sorted linked list. A bucket b ranges from
  303 + b->first inclusive up to b->next->first exclusive. */
  304 +static int
  305 +in_bucket(const unsigned char *id, struct bucket *b)
  306 +{
  307 + return id_cmp(b->first, id) <= 0 &&
  308 + (b->next == NULL || id_cmp(id, b->next->first) < 0);
  309 +}
  310 +
  311 +static struct bucket *
  312 +find_bucket(unsigned const char *id)
  313 +{
  314 + struct bucket *b = buckets;
  315 +
  316 + while(1) {
  317 + if(b->next == NULL)
  318 + return b;
  319 + if(id_cmp(id, b->next->first) < 0)
  320 + return b;
  321 + b = b->next;
  322 + }
  323 +}
  324 +
  325 +static struct bucket *
  326 +previous_bucket(struct bucket *b)
  327 +{
  328 + struct bucket *p = buckets;
  329 +
  330 + if(b == p)
  331 + return NULL;
  332 +
  333 + while(1) {
  334 + if(p->next == NULL)
  335 + return NULL;
  336 + if(p->next == b)
  337 + return p;
  338 + p = p->next;
  339 + }
  340 +}
  341 +
  342 +/* Every bucket contains an unordered list of nodes. */
  343 +static struct node *
  344 +find_node(const unsigned char *id)
  345 +{
  346 + struct bucket *b = find_bucket(id);
  347 + struct node *n;
  348 +
  349 + if(b == NULL)
  350 + return NULL;
  351 + n = b->nodes;
  352 + while(n) {
  353 + if(id_cmp(n->id, id) == 0)
  354 + return n;
  355 + n = n->next;
  356 + }
  357 + return NULL;
  358 +}
  359 +
  360 +/* Return a random node in a bucket. */
  361 +static struct node *
  362 +random_node(struct bucket *b)
  363 +{
  364 + struct node *n;
  365 + int nn;
  366 +
  367 + if(b->count == 0)
  368 + return NULL;
  369 +
  370 + nn = random() % b->count;
  371 + n = b->nodes;
  372 + while(nn > 0 && n) {
  373 + n = n->next;
  374 + nn--;
  375 + }
  376 + return n;
  377 +}
  378 +
  379 +/* Return the middle id of a bucket. */
  380 +static int
  381 +bucket_middle(struct bucket *b, unsigned char *id_return)
  382 +{
  383 + int bit1 = lowbit(b->first);
  384 + int bit2 = b->next ? lowbit(b->next->first) : -1;
  385 + int bit = (bit1 >= bit2 ? bit1 : bit2) + 1;
  386 +
  387 + if(bit >= 160)
  388 + return -1;
  389 +
  390 + memcpy(id_return, b->first, 20);
  391 + id_return[bit / 8] |= (0x80 >> (bit % 8));
  392 + return 1;
  393 +}
  394 +
  395 +/* Return a random id within a bucket. */
  396 +static int
  397 +bucket_random(struct bucket *b, unsigned char *id_return)
  398 +{
  399 + int bit1 = lowbit(b->first);
  400 + int bit2 = b->next ? lowbit(b->next->first) : -1;
  401 + int bit = (bit1 >= bit2 ? bit1 : bit2) + 1;
  402 + int i;
  403 +
  404 + if(bit >= 160) {
  405 + memcpy(id_return, b->first, 20);
  406 + return 1;
  407 + }
  408 +
  409 + memcpy(id_return, b->first, bit / 8);
  410 + id_return[bit / 8] = b->first[bit / 8] & (0xFF00 >> (bit % 8));
  411 + id_return[bit / 8] |= random() & 0xFF >> (bit % 8);
  412 + for(i = bit / 8 + 1; i < 20; i++)
  413 + id_return[i] = random() & 0xFF;
  414 + return 1;
  415 +}
  416 +
  417 +/* Insert a new node into a bucket. */
  418 +static struct node *
  419 +insert_node(struct node *node)
  420 +{
  421 + struct bucket *b = find_bucket(node->id);
  422 +
  423 + node->next = b->nodes;
  424 + b->nodes = node;
  425 + b->count++;
  426 + return node;
  427 +}
  428 +
  429 +/* This is our definition of a known-good node. */
  430 +static int
  431 +node_good(struct node *node)
  432 +{
  433 + return
  434 + node->pinged <= 2 &&
  435 + node->reply_time >= now.tv_sec - 7200 &&
  436 + node->time >= now.tv_sec - 900;
  437 +}
  438 +
  439 +/* Every bucket caches the address of a likely node. Ping it. */
  440 +static int
  441 +send_cached_ping(int s, struct bucket *b)
  442 +{
  443 + int rc;
  444 + /* We set family to 0 when there's no cached node. */
  445 + if(b->cached.sin_family == AF_INET) {
  446 + debugf("Sending ping to cached node.\n");
  447 + rc = send_ping(s, (struct sockaddr*)&b->cached,
  448 + sizeof(struct sockaddr_in),
  449 + (unsigned char*)"pn", 2);
  450 + b->cached.sin_family = 0;
  451 + return rc;
  452 + }
  453 + return 0;
  454 +}
  455 +
  456 +/* Split a bucket into two equal parts. */
  457 +static struct bucket *
  458 +split_bucket(int s, struct bucket *b)
  459 +{
  460 + struct bucket *new;
  461 + struct node *nodes;
  462 + int rc;
  463 + unsigned char new_id[20];
  464 +
  465 + rc = bucket_middle(b, new_id);
  466 + if(rc < 0)
  467 + return NULL;
  468 +
  469 + new = calloc(1, sizeof(struct bucket));
  470 + if(new == NULL)
  471 + return NULL;
  472 +
  473 + send_cached_ping(s, b);
  474 +
  475 + memcpy(new->first, new_id, 20);
  476 + new->time = b->time;
  477 +
  478 + nodes = b->nodes;
  479 + b->nodes = NULL;
  480 + b->count = 0;
  481 + new->next = b->next;
  482 + b->next = new;
  483 + while(nodes) {
  484 + struct node *n;
  485 + n = nodes;
  486 + nodes = nodes->next;
  487 + insert_node(n);
  488 + }
  489 + return b;
  490 +}
  491 +
  492 +/* Called whenever we send a request to a node. */
  493 +static void
  494 +pinged(int s, struct node *n, struct bucket *b)
  495 +{
  496 + n->pinged++;
  497 + n->pinged_time = now.tv_sec;
  498 + if(n->pinged >= 3)
  499 + send_cached_ping(s, b ? b : find_bucket(n->id));
  500 +}
  501 +
  502 +/* We just learnt about a node, not necessarily a new one. Confirm is 1 if
  503 + the node sent a message, 2 if it sent us a reply. */
  504 +static struct node *
  505 +new_node(int s, const unsigned char *id, struct sockaddr_in *sin,
  506 + int confirm)
  507 +{
  508 + struct bucket *b = find_bucket(id);
  509 + struct node *n;
  510 + int mybucket = in_bucket(myid, b);
  511 +
  512 + if(id_cmp(id, myid) == 0)
  513 + return NULL;
  514 +
  515 + if(confirm)
  516 + b->time = now.tv_sec;
  517 +
  518 + n = b->nodes;
  519 + while(n) {
  520 + if(id_cmp(n->id, id) == 0) {
  521 + if(confirm || n->time < now.tv_sec - 15 * 60) {
  522 + /* Known node. Update stuff. */
  523 + n->sin = *sin;
  524 + if(confirm)
  525 + n->time = now.tv_sec;
  526 + if(confirm >= 2) {
  527 + n->reply_time = now.tv_sec;
  528 + n->pinged = 0;
  529 + n->pinged_time = 0;
  530 + }
  531 + }
  532 + return n;
  533 + }
  534 + n = n->next;
  535 + }
  536 +
  537 + /* New node. First, try to get rid of a known-bad node. */
  538 + n = b->nodes;
  539 + while(n) {
  540 + if(n->pinged >= 3) {
  541 + memcpy(n->id, id, 20);
  542 + n->sin = *sin;
  543 + n->time = confirm ? now.tv_sec : 0;
  544 + n->reply_time = confirm >= 2 ? now.tv_sec : 0;
  545 + n->pinged_time = 0;
  546 + n->pinged = 0;
  547 + if(mybucket)
  548 + mybucket_grow_time = now.tv_sec;
  549 + return n;
  550 + }
  551 + n = n->next;
  552 + }
  553 +
  554 + if(b->count >= 8) {
  555 + /* Bucket full. Ping a dubious node */
  556 + int dubious = 0;
  557 + n = b->nodes;
  558 + while(n) {
  559 + /* Pick the first dubious node that we haven't pinged in the
  560 + last 15 seconds. This gives nodes the time to reply, but
  561 + tends to concentrate on the same nodes. */
  562 + if(!node_good(n) && n->pinged_time < now.tv_sec - 15) {
  563 + debugf("Sending ping to dubious node.\n");
  564 + send_ping(s,
  565 + (struct sockaddr*)&n->sin, sizeof(struct sockaddr_in),
  566 + (unsigned char*)"pn", 2);
  567 + n->pinged++;
  568 + n->pinged_time = now.tv_sec;
  569 + dubious = 1;
  570 + break;
  571 + }
  572 + n = n->next;
  573 + }
  574 +
  575 + if(!dubious && mybucket) {
  576 + debugf("Splitting.\n");
  577 + b = split_bucket(s, b);
  578 + mybucket_grow_time = now.tv_sec;
  579 + return new_node(s, id, sin, confirm);
  580 + }
  581 +
  582 + /* No space for this node. Cache it away for later. */
  583 + if(confirm || b->cached.sin_family == 0)
  584 + b->cached = *sin;
  585 +
  586 + return NULL;
  587 + }
  588 +
  589 + /* Create a new node. */
  590 + n = calloc(1, sizeof(struct node));
  591 + if(n == NULL)
  592 + return NULL;
  593 + memcpy(n->id, id, 20);
  594 + n->sin = *sin;
  595 + n->time = confirm ? now.tv_sec : 0;
  596 + n->reply_time = confirm >= 2 ? now.tv_sec : 0;
  597 + n->next = b->nodes;
  598 + b->nodes = n;
  599 + b->count++;
  600 + if(mybucket)
  601 + mybucket_grow_time = now.tv_sec;
  602 + return n;
  603 +}
  604 +
  605 +/* Called periodically to purge known-bad nodes. Note that we're very
  606 + conservative here: broken nodes in the table don't do much harm, we'll
  607 + recover as soon as we find better ones. */
  608 +static int
  609 +expire_buckets(int s)
  610 +{
  611 + struct bucket *b = buckets;
  612 +
  613 + while(b) {
  614 + struct node *n, *p;
  615 + int changed = 0;
  616 +
  617 + while(b->nodes && b->nodes->pinged >= 4) {
  618 + n = b->nodes;
  619 + b->nodes = n->next;
  620 + b->count--;
  621 + changed = 1;
  622 + free(n);
  623 + }
  624 +
  625 + p = b->nodes;
  626 + while(p) {
  627 + while(p->next && p->next->pinged >= 4) {
  628 + n = p->next;
  629 + p->next = n->next;
  630 + b->count--;
  631 + changed = 1;
  632 + free(n);
  633 + }
  634 + p = p->next;
  635 + }
  636 +
  637 + if(changed)
  638 + send_cached_ping(s, b);
  639 +
  640 + b = b->next;
  641 + }
  642 + expire_stuff_time = now.tv_sec + 120 + random() % 240;
  643 + return 1;
  644 +}
  645 +
  646 +/* While a search is in progress, we don't necessarily keep the nodes being
  647 + walked in the main bucket table. A search in progress is identified by
  648 + a unique transaction id, a short (and hence small enough to fit in the
  649 + transaction id of the protocol packets). */
  650 +
  651 +struct search *
  652 +find_search(unsigned short tid)
  653 +{
  654 + int i;
  655 + for(i = 0; i < numsearches; i++) {
  656 + if(searches[i].tid == tid)
  657 + return &searches[i];
  658 + }
  659 + return NULL;
  660 +}
  661 +
  662 +/* A search contains a list of nodes, sorted by decreasing distance to the
  663 + target. We just got a new candidate, insert it at the right spot or
  664 + discard it. */
  665 +
  666 +int
  667 +insert_search_node(unsigned char *id, struct sockaddr_in *sin,
  668 + struct search *sr, int replied,
  669 + unsigned char *token, int token_len)
  670 +{
  671 + int bits = common_bits(id, sr->id);
  672 + struct search_node *n;
  673 + int i, j;
  674 +
  675 + for(i = 0; i < sr->numnodes; i++) {
  676 + if(id_cmp(id, sr->nodes[i].id) == 0) {
  677 + n = &sr->nodes[i];
  678 + goto found;
  679 + }
  680 + if(common_bits(sr->id, sr->nodes[i].id) < bits)
  681 + break;
  682 + }
  683 +
  684 + if(i == 8)
  685 + return 0;
  686 +
  687 + if(sr->numnodes < 8)
  688 + sr->numnodes++;
  689 +
  690 + for(j = sr->numnodes - 1; j > i; j--) {
  691 + sr->nodes[j] = sr->nodes[j - 1];
  692 + }
  693 +
  694 + n = &sr->nodes[i];
  695 +
  696 + memset(n, 0, sizeof(struct search_node));
  697 + memcpy(n->id, id, 20);
  698 +
  699 +found:
  700 + n->sin = *sin;
  701 +
  702 + if(replied) {
  703 + n->replied = 1;
  704 + n->reply_time = now.tv_sec;
  705 + n->request_time = 0;
  706 + n->pinged = 0;
  707 + }
  708 + if(token) {
  709 + if(token_len >= 40) {
  710 + debugf("Eek! Overlong token.\n");
  711 + } else {
  712 + memcpy(n->token, token, token_len);
  713 + n->token_len = token_len;
  714 + }
  715 + }
  716 +
  717 + return 1;
  718 +}
  719 +
  720 +static void
  721 +flush_search_node(struct search_node *n, struct search *sr)
  722 +{
  723 + int i = n - sr->nodes, j;
  724 + for(j = i; j < sr->numnodes - 1; j++)
  725 + sr->nodes[j] = sr->nodes[j + 1];
  726 + sr->numnodes--;
  727 +}
  728 +
  729 +/* This must always return 0 or 1, never -1, not even on failure (see below). */
  730 +static int
  731 +search_send_get_peers(int s, struct search *sr, struct search_node *n)
  732 +{
  733 + struct node *node;
  734 + unsigned char tid[4];
  735 +
  736 + if(n == NULL) {
  737 + int i;
  738 + for(i = 0; i < sr->numnodes; i++) {
  739 + if(sr->nodes[i].pinged < 3 && !sr->nodes[i].replied &&
  740 + sr->nodes[i].request_time < now.tv_sec - 15)
  741 + n = &sr->nodes[i];
  742 + }
  743 + }
  744 +
  745 + if(!n || n->pinged >= 3 || n->replied ||
  746 + n->request_time >= now.tv_sec - 15)
  747 + return 0;
  748 +
  749 + debugf("Sending get_peers.\n");
  750 + tid[0] = 'g'; tid[1] = 'p'; memcpy(tid + 2, &sr->tid, 2);
  751 + send_get_peers(s, (struct sockaddr*)&n->sin,
  752 + sizeof(struct sockaddr_in),
  753 + tid, 4, sr->id, sr->tid,
  754 + n->reply_time >= now.tv_sec - 15);
  755 + n->pinged++;
  756 + n->request_time = now.tv_sec;
  757 + /* If the node happens to be in our main routing table, mark it
  758 + as pinged. */
  759 + node = find_node(n->id);
  760 + if(node) pinged(s, node, NULL);
  761 + return 1;
  762 +}
  763 +
  764 +/* When a search is in progress, we periodically call search_step to send
  765 + further requests. */
  766 +void
  767 +search_step(int s, struct search *sr, dht_callback *callback, void *closure)
  768 +{
  769 + int i, j = 0;
  770 + int all_done = 1;
  771 +
  772 + for(i = 0; i < sr->numnodes; i++) {
  773 + struct search_node *n = &sr->nodes[i];
  774 + if(n->pinged < 3 && !n->replied) {
  775 + all_done = 0;
  776 + break;
  777 + }