Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 266 lines (199 sloc) 13.182 kB
833329c @jlouis Move modelines to the top of the file.
authored
1 ; -*- Mode: Markdown; -*-
2 ; vim: filetype=none tw=76 expandtab
3
0a95d82 @jlouis Markdownify.
authored
4 # ETORRENT
e6c8714 @jlouis More GNU-standard style thingies.
authored
5
6 ETORRENT is a bittorrent client written in Erlang. The focus is on
7 robustness and scalability in number of torrents rather than in pure
8 speed. ETORRENT is mostly meant for unattended operation, where one
9 just specifies what files to download and gets a notification when
10 they are.
11
73846e8 @jlouis Mention in the README.md that we had a flag day.
authored
12 ## Flag days
13
14 Flag days are when you need to do something to your setup
15
29e2654 @jlouis Mention the rebar version needed for this piece of software.
authored
16 * *2010-12-10* You will need a rebar with commit 618b292c3d84 as we
17 got some fixes into rebar.
9156326 @jlouis Make it clear to people that they need to repend the application.
authored
18 * *2010-12-06* We now depend on riak_err. You will need to regrab
19 dependencies.
73846e8 @jlouis Mention in the README.md that we had a flag day.
authored
20 * *2010-12-02* The fast-resume-file format has been updated. You
21 may have to delete your fast_resume_file though the system was
22 configured to do a silent system upgrade.
23
3d40cf9 @jlouis Try pushing to the README file a status image set.
authored
24 ## Build Status
25
26 Currently, we are running build-bots through
a58415b @jlouis Improve the build status image.
authored
27 [http://travis-ci.org](Travis CI). [![Build Status](http://travis-ci.org/jlouis/etorrent.png?branch=master)](http://travis-ci.org//jlouis/etorrent)
3d40cf9 @jlouis Try pushing to the README file a status image set.
authored
28
29
73846e8 @jlouis Mention in the README.md that we had a flag day.
authored
30 ## Why
31
e6c8714 @jlouis More GNU-standard style thingies.
authored
32 ETORRENT was mostly conceived as an experiment in how easy it would be
33 to write a bittorrent client in Erlang. The hypothesis is that the
34 code will be cleaner and smaller than comparative bittorrent clients.
35
520c5d9 @jlouis Update maturity statement.
authored
36 ## Maturity
37
38 The code is at this point somewhat mature. It has been used in several
39 scenarios with a good host of different clients and trackers. The code
40 base is not known to act as a bad p2p citizen, although it may be
41 possible that it is.
42
43 The most important missing link currently is that only a few users
44 have been testing it - so there may still be bugs in the code. The
45 `master` branch has been quite stable for months now however, so it is
46 time to get some more users for testing. Please report any bugs,
47 especially if the client behaves badly.
c2a2f5f @jlouis Add a notice on missing battle scarring.
authored
48
73846e8 @jlouis Mention in the README.md that we had a flag day.
authored
49 ## Currently supported BEPs:
f68a584 @jlouis Reflect changes in TODO.
authored
50
0a95d82 @jlouis Markdownify.
authored
51 * BEP 03 - The BitTorrent Protocol Specification.
52 * BEP 04 - Known Number Allocations.
329e328 @jlouis Update what BEPs we support.
authored
53 * BEP 05 - DHT Protocol
54 * BEP 10 - Extension Protocol
398bba0 @jlouis Make it clear we are supporting the BEP-12 extension now.
authored
55 * BEP 12 - Multitracker Metadata Extension.
329e328 @jlouis Update what BEPs we support.
authored
56 * BEP 15 - UDP Tracker Protocol
0a95d82 @jlouis Markdownify.
authored
57 * BEP 23 - Tracker Returns Compact Peer Lists.
f68a584 @jlouis Reflect changes in TODO.
authored
58
73846e8 @jlouis Mention in the README.md that we had a flag day.
authored
59 ## Required software:
0c5716e @jlouis Fix build requirements by gentle prodding on IRC.
authored
60
61 * rebar - you need a working rebar installation to build etorrent.
523a764 @jlouis Sharpen README prerequisites.
authored
62 The rebar must have commit 618b292c3d84. It is known that rebar
63 version `rebar version: 2 date: 20110310_155239 vcs: git 2e1b4da`
64 works, and later versions probably will too.
0c5716e @jlouis Fix build requirements by gentle prodding on IRC.
authored
65 * Erlang/OTP R13B04 or R14 - the etorrent system is written in
66 Erlang and thus requires a working Erlang distribution to
67 work. It may work with older versions, but has mostly been tested
68 with newer versions.
523a764 @jlouis Sharpen README prerequisites.
authored
69
70 If you have the option, running on R14B02 is preferred. The
71 developers are usually running on fairly recent Erlang/OTP
72 releases, so you have much better chances with these systems.
73 * A UNIX-derivative or Windows as the operating system
74 Support has been tested by mulander and is reported to work with
25630a4 @mulander Documented the steps required to make etorrent run on MS Windows
mulander authored
75 some manual labor. For details see the Windows getting started section.
0c5716e @jlouis Fix build requirements by gentle prodding on IRC.
authored
76
475f237 @jlouis Update the README file to match the current installation.
authored
77 ## GETTING STARTED
78
f8cacf1 @jlouis Mention that we need to get deps as well when building.
authored
79 0. `make deps` - Pull the relevant dependencies into *deps/*
80 1. `make compile` - this compiles the source code
81 2. `make rel` - this creates an embedded release in *rel/etorrent* which
475f237 @jlouis Update the README file to match the current installation.
authored
82 can subsequently be moved to a location at your leisure.
f8cacf1 @jlouis Mention that we need to get deps as well when building.
authored
83 3. edit `${EDITOR} rel/etorrent/etc/app.config` - there are a number of directories
475f237 @jlouis Update the README file to match the current installation.
authored
84 which must be set in order to make the system work.
520c5d9 @jlouis Update maturity statement.
authored
85 4. check `${EDITOR} rel/etorrent/etc/vm.args` - Erlang args to supply
19effe3 @jlouis Correct build instructions.
authored
86 5. be sure to protect the erlang cookie or anybody can connect to
87 your erlang system! See the Erlang user manual in [distributed operation](http://www.erlang.org/doc/reference_manual/distributed.html)
f8cacf1 @jlouis Mention that we need to get deps as well when building.
authored
88 6. run `rel/etorrent/bin/etorrent console`
89 7. drop a .torrent file in the watched dir and see what happens.
90 8. call `etorrent:help()`. from the Erlang CLI to get a list of available
475f237 @jlouis Update the README file to match the current installation.
authored
91 commands.
f8cacf1 @jlouis Mention that we need to get deps as well when building.
authored
92 9. If you enabled the webui, you can try browsing to its location. By default the location is 'http://localhost:8080'.
dae83bd @jlouis Do a revamp of the README to reflect new world.
authored
93
25630a4 @mulander Documented the steps required to make etorrent run on MS Windows
mulander authored
94 ## GETTING STARTED (Windows)
95 0. Obviously get and install erlang from erlang.org. This process was tested with R14B01 release.
96 You may want to add the bin directory to your PATH in order to reduce the length of the commands you will enter later on.
d80d373 @mulander Fixed msysgit URL
mulander authored
97 1. Install [msysgit](http://code.google.com/p/msysgit/). Tested with [1.7.3.1.msysgit.0](http://code.google.com/p/msysgit/downloads/detail?name=Git-1.7.3.1-preview20101002.exe&can=2&q=).
25630a4 @mulander Documented the steps required to make etorrent run on MS Windows
mulander authored
98 2. Install [Win32 OpenSSL](http://www.slproweb.com/products/Win32OpenSSL.html).
99 The installer hang on me midway through the process but the libs were properly copied to C:\Windows\system32.
100 3. Confirm that your crypto works correctly by running `crypto:start().` from an erlang shell (Start->Erlang OTP R14B01->Erlang).
101 The shell should respond with `ok`. If you get an error then your openssl libraries are still missing.
102 4. Open up a git bash shell and cd to a directory you want to work in.
07cf142 @mulander The rebar clone instructions were missing the git command
mulander authored
103 5. Clone the rebar repository `git clone https://github.com/basho/rebar.git`
25630a4 @mulander Documented the steps required to make etorrent run on MS Windows
mulander authored
104 6. From a regular cmd.exe shell `cd rebar`. Now if you added the erlang bin directory to your PATH then you can simply run `bootstrap.bat`.
105 If you didn't add Erlang's bin to your path then issue the following command:
fe02323 @mulander More formatting fixes
mulander authored
106
f53a99d @mulander Fixed unterminated backquote
mulander authored
107 `"C:\Program Files\erl5.8.2\bin\escript.exe" bootstrap`
fe02323 @mulander More formatting fixes
mulander authored
108
25630a4 @mulander Documented the steps required to make etorrent run on MS Windows
mulander authored
109 Adjust the path to your Erlang installation directory. From now on, use this invocation for escript.exe and erl.exe. I'll assume it's on PATH.
110 You should now have a `rebar` file created. If you have Erlangs bin dir on your PATH then you may want to also add the rebar directory to your
111 PATH. This will allow you to use the rebar.bat script which will also reduce the amount of typing you will have to do.
112 7. Clone etorrent and copy the `rebar` file into it (unless you have it on path).
113 8. Now, we need to satisfy the dependencies. This should be done by rebar itself but I couldn't get it to work correctly with git.
114 This point describes how to do it manually.
fe02323 @mulander More formatting fixes
mulander authored
115
25630a4 @mulander Documented the steps required to make etorrent run on MS Windows
mulander authored
116 `mkdir deps` # this is where rebar will look for etorrent dependencies
fe02323 @mulander More formatting fixes
mulander authored
117
25630a4 @mulander Documented the steps required to make etorrent run on MS Windows
mulander authored
118 `escript.exe rebar check-deps` # this will give you a list of the missing dependencies and their git repos.
fe02323 @mulander More formatting fixes
mulander authored
119
25630a4 @mulander Documented the steps required to make etorrent run on MS Windows
mulander authored
120 For each dependency reported by check-deps perform the following command in a git bash shell (be sure to be in the etorrent directory)
fe02323 @mulander More formatting fixes
mulander authored
121
25630a4 @mulander Documented the steps required to make etorrent run on MS Windows
mulander authored
122 `git clone git://path_to_repo/name.git deps/name`
fe02323 @mulander More formatting fixes
mulander authored
123
25630a4 @mulander Documented the steps required to make etorrent run on MS Windows
mulander authored
124 Be sure to run `escript.exe rebar check-deps` after cloning each additional repo as new dependencies might be added by them.
fe02323 @mulander More formatting fixes
mulander authored
125
25630a4 @mulander Documented the steps required to make etorrent run on MS Windows
mulander authored
126 For the time of this writing (2011-02-13) I had to clone the following repositories:
fe02323 @mulander More formatting fixes
mulander authored
127
25630a4 @mulander Documented the steps required to make etorrent run on MS Windows
mulander authored
128 `git clone git://github.com/esl/gproc.git deps/gproc`
fe02323 @mulander More formatting fixes
mulander authored
129
25630a4 @mulander Documented the steps required to make etorrent run on MS Windows
mulander authored
130 `git clone git://github.com/esl/edown.git deps/edown`
fe02323 @mulander More formatting fixes
mulander authored
131
25630a4 @mulander Documented the steps required to make etorrent run on MS Windows
mulander authored
132 `git clone git://github.com/basho/riak_err.git deps/riak_err`
133 9. `escript.exe rebar compile` to compile the application from a cmd.exe prompt.
134 10. `escript.exe rebar generate` this creates an embedded release in *rel/etorrent* which
135 can subsequently be moved to a location at your leisure. This command may take some time so be patient.
136 11. Edit `rel/etorrent/etc/app.config` - there are a number of directories which must be set in order to make the system work.
137 Be sure each directory exists before starting etorrent. You also have to change all paths starting with `/` (ie. `/var/lib...`).
138 When setting paths use forward slashes. For example `{dir, "D:/etorrent/torrents"},`.
139 12. Check `rel/etorrent/etc/vm.args` - Erlang args to supply
140 13. Be sure to protect the erlang cookie or anybody can connect to
141 your erlang system! See the Erlang user manual in [distributed operation](http://www.erlang.org/doc/reference_manual/distributed.html)
142 14. We can't run the etorrent script because it's written in bash. So in order to start ettorent cd from a cmd.exe shell into the `rel\etorrent`
143 directory. And perform the following command
fe02323 @mulander More formatting fixes
mulander authored
144
25630a4 @mulander Documented the steps required to make etorrent run on MS Windows
mulander authored
145 `erts-5.8.2\bin\erl.exe -boot release\1.2.1\etorrent -embedded -config etc\app.config -args_file etc\vm.args`
fe02323 @mulander More formatting fixes
mulander authored
146
25630a4 @mulander Documented the steps required to make etorrent run on MS Windows
mulander authored
147 Be sure to substitute the version number in the release path to the current etorrent release version. Do the same for the erts version.
148 Allow port communication when/if the Windows firewall asks for it.
149 15. drop a .torrent file in the watched dir and see what happens.
150 16. call `etorrent:help()`. from the Erlang CLI to get a list of available commands.
151 17. If you enabled the webui, you can try browsing to its location. By default the location is 'http://localhost:8080'.
152
331e9cb @jlouis Point in the direction of the TEST.md document.
authored
153 ## Testing etorrent
154
ec9cbe2 @jlouis Improve TEST documentation.
authored
155 Read the document [etorrent/TEST.md](/jlouis/etorrent/tree/master/TEST.md)
331e9cb @jlouis Point in the direction of the TEST.md document.
authored
156 for how to run tests of the system.
157
9f0f833 @jlouis Update README.md with some hints on package building.
authored
158 ## Troubleshooting
159
160 If the above commands doesn't work, we want to hear about it. This is
161 a list of known problems:
162
163 * General: Many distributions are insane and pack erlang in split
164 packages, so each part of erlang is in its own package. This
165 *always* leads to build problems due to missing stuff. Be sure
166 you have all relevant packages installed. And when you find which
167 packages are needed, please send a patch to this file for the
168 distribution and version so we can keep it up-to-date.
169
170 * Ubuntu 10.10: Ubuntu has a symlink `/usr/lib/erlang/man ->
171 /usr/share/man`. This is insane and generates problems when
172 building a release (spec errors on missing files if a symlink
173 points to nowhere). The easiest fix is to remove the man symlink
174 from `/usr/lib/erlang`. A way better fix is to install a recent
175 Erlang/OTP yourself and use **Begone(tm)** on the supplied version.
176
a19f42c @jlouis Add Erlang build instructions to troubleshooting.
authored
177 ### Installing Erlang
178
179 I (jlouis@) use the following commands to install Erlang:
180
181 * Install *stow*, `sudo aptitude install stow`
182 * Execute `sudo aptitude build-dep erlang` to pull in all build
183 dependencies we need.
184 * Execute `git clone git://github.com/erlang/otp.git`
185 * Get into the directory and fire away `git checkout dev && ./otp_build autoconf`
510b4d6 @jlouis OTP build instructions update.
authored
186 * Get a useful tag for the otp version you have built `otp_version=$(git describe)`
187 * `./configure --prefix=/usr/local/stow/${otp_version}`
a19f42c @jlouis Add Erlang build instructions to troubleshooting.
authored
188 * `make; make docs`
189 * `make install install-docs`
190
191 And then I enable it in stow:
192
510b4d6 @jlouis OTP build instructions update.
authored
193 cd /usr/local/stow && stow ${otp_version}
194
195 You may have to use `stow -D` on an old
a19f42c @jlouis Add Erlang build instructions to troubleshooting.
authored
196
655b534 @edwardw Official kerl repo has been moved
edwardw authored
197 Another way is to use [Basho kerl](https://github.com/spawngrid/kerl), a simple shell script:
1dd896d @edwardw Add kerl to Erlang build instructions
edwardw authored
198
199 ``` bash
200 $ curl -O https://raw.github.com/spawngrid/kerl/master/kerl; chmod a+x kerl
201 $ ./kerl build R14B04 r14b04
202 $ ./kerl install r14b04 /opt/erlang/r14b04
203 $ . /opt/erlang/r14b04/activate
204 ```
205
655b534 @edwardw Official kerl repo has been moved
edwardw authored
206 In the above example, kerl installs Erlang R14B04 to directory /opt/erlang/r14b04 and makes it default. For more information, please check [kerl readme](https://github.com/spawngrid/kerl/blob/master/README.md).
1dd896d @edwardw Add kerl to Erlang build instructions
edwardw authored
207
1d00e6b @jlouis Tell people where we are on IRC.
authored
208 ## QUESTIONS??
209
210 You can either mail them to `jesper.louis.andersen@gmail.com` or you
211 can come by on IRC #etorrent/freenode and ask.
212
075a185 @jlouis Bump documentation for git and for the README file.
authored
213 # Development
214
215 ## PATCHES
216
217 To submit patches, we have documentation in `documentation/git.md`,
218 giving tips to patch submitters.
219
9535d47 @jlouis Document how to create a development environment.
authored
220 ## Setting up a development environment
221
222 When developing for etorrent, you might end up generating a new
223 environment quite often. So ease the configuration, the build
224 infrastructure support this.
225
226 * Create a file `rel/vars/etorrent-dev_vars.config` based upon the file
227 `rel/vars.config`.
228 * run `make compile etorrent-dev`
229 * run `make console`
230
231 Notice that we `-pa` add `../../apps/etorrent/ebin` so you can l(Mod) files
232 from the shell directly into the running system after having
233 recompiled them.
234
e19e96a @jlouis Bump README.md with hacking hints.
authored
235 ## Documentation
236
b0323f7 @jlouis Hint about the git.md file.
authored
237 Read the HACKING.md file in this directory. For how the git repository
238 is worked, see `documentation/git.md`.
e19e96a @jlouis Bump README.md with hacking hints.
authored
239
0a95d82 @jlouis Markdownify.
authored
240 ## ISSUES
e6c8714 @jlouis More GNU-standard style thingies.
authored
241
075a185 @jlouis Bump documentation for git and for the README file.
authored
242 Either mail them to `jesper.louis.andersen@gmail.com` (We are
243 currently lacking a mailing list) or use the [issue tracker](http://github.com/jlouis/etorrent/issues)
4b3639d @jlouis Update the README with reading information.
authored
244
245 ## Reading material for hacking Etorrent:
246
247 - [Protocol specification - BEP0003](http://www.bittorrent.org/beps/bep_0003.html):
248 This is the original protocol specification, tracked into the BEP
249 process. It is worth reading because it explains the general overview
250 and the precision with which the original protocol was written down.
251
252 - [Bittorrent Enhancement Process - BEP0000](http://www.bittorrent.org/beps/bep_0000.html)
253 The BEP process is an official process for adding extensions on top of
254 the BitTorrent protocol. It allows implementors to mix and match the
255 extensions making sense for their client and it allows people to
256 discuss extensions publicly in a forum. It also provisions for the
257 deprecation of certain features in the long run as they prove to be of
258 less value.
259
260 - [wiki.theory.org](http://wiki.theory.org/Main_Page)
261 An alternative description of the protocol. This description is in
262 general much more detailed than the BEP structure. It is worth a read
263 because it acts somewhat as a historic remark and a side channel. Note
264 that there are some commentary on these pages which can be disputed
265 quite a lot.
Something went wrong with that request. Please try again.