Skip to content
This repository
Newer
Older
100644 196 lines (156 sloc) 7.684 kb
0b8aa412 » BuzzTroll
2010-09-21 merged virga1 and master
1 ==========
2 Lantorrent
3 ==========
4
5 Lantorrent is a file distribution protocol integrated into the Nimbus
6 IaaS toolkit. It works as a means to multi-cast virtual machine images
7 to many backend nodes. The protocol is optimized for propagating
8 virtual machine images (typically large files) from a central repository
9 across a LAN to many virtual machine monitor nodes.
10
11 Lantorrent works best for the following scenarios:
12
13 1) large file transfers (VM images are typically measured in gigabytes)
14 2) local area switched network (typical for data center computer racks)
15 3) file recipients are willing peers.
16 4) many endpoints request the same file at roughly the same time
17
18 -----------------
19 Protocol overview
20 -----------------
21
22 When an endpoint wants a file it submits a request to a central agent.
23 This agent aggregates request for files so that they can be sent out in
24 an efficient single multi-cast session. Each request for a source file
25 is stored until either N request on that file have been made or N'
26 seconds have passed since the last request on that source file has been
27 made. This allows for a user to request a single file in several
28 unrelated session yet still have the file transfered in an efficient
29 multi-cast session.
30
31 Once N requests for a given source file have been made or N' seconds
32 have passed the destination set for the source file is determined. A
33 chain of destination endpoints is formed such that each node receives
34 from and sends to one other node. The first node receives from the
35 repository and send to a peer node, that peer node sends to another,
36 and so on until all receive the file. In this way all links of the
37 switch are utilized to send directly to another endpoint in the switch.
38 This results in the most efficient transfer on a LAN switched network.
39
40 Often times in a IaaS system a single network endpoint (VMM) will want
41 multiple copies of the same file. Each file is booted as a virtual
42 machine and that virtual machine will make distinct changes to that file
eb045717 » priteau
2011-03-04 Fix typo in README
43 as it runs, thus it needs its own copy of the file. However that file
0b8aa412 » BuzzTroll
2010-09-21 merged virga1 and master
44 does not need to be transfered across the network more than once.
45 Lantorrent will send the file to each endpoint once and instruct that
46 endpoint to write it to multiple files if needed.
47
48 -----------------------------
49 Enabling Lantorrent in Nimbus
50 -----------------------------
51
aa6a4177 » priteau
2010-10-30 Improve LANTorrent docs, and sync HTML and README
52 Lan torrent is part of the Nimbus distribution as of Nimbus 2.6.
53 However, due to system administrative overhead it is not enabled by
54 default. To enable Lantorrent in Nimbus there are a few configurations
0b8aa412 » BuzzTroll
2010-09-21 merged virga1 and master
55 changes that must be made.
56
aa6a4177 » priteau
2010-10-30 Improve LANTorrent docs, and sync HTML and README
57 The following software is required on both service and VMM nodes:
a0c4bcd9 » BuzzTroll
2011-01-24 updating the doc the show a dependency on py 2.5. There is a problem …
58 - python 2.5
aa6a4177 » priteau
2010-10-30 Improve LANTorrent docs, and sync HTML and README
59 - python simplejson
eed27d22 » priteau
2010-10-07 Add a note about dependencies
60
aa6a4177 » priteau
2010-10-30 Improve LANTorrent docs, and sync HTML and README
61 Lantorrent is run out of xinetd thus it must also be installed on all VMMs.
62
63 To install LANTorrent you must take the following steps:
64
65 1) edit $NIMBUS_HOME/nimbus-setup.conf
0b8aa412 » BuzzTroll
2010-09-21 merged virga1 and master
66 change lantorrent.enabled: False -> lantorrent.enabled: True
67
aa6a4177 » priteau
2010-10-30 Improve LANTorrent docs, and sync HTML and README
68 2) edit $NIMBUS_HOME/services/etc/nimbus/workspace-service/other/common.conf
0b8aa412 » BuzzTroll
2010-09-21 merged virga1 and master
69 change the value of propagate.extraargs:
70 propagate.extraargs=$NIMBUS_HOME/lantorrent/bin/lt-request
71
72 be sure to expand $NIMBUS_HOME to its full and actual path.
73
aa6a4177 » priteau
2010-10-30 Improve LANTorrent docs, and sync HTML and README
74 3) install lantorrent on VMM
8d187eaa » buzztroll
2011-05-19 repackaging lantorrent to be more pythonic
75
76 - run: python setup-vmm.py install. This will output the contents of
77 the xinetd configuration file that you will need in the next step.
d3691d57 » priteau
2011-07-12 Update the LANTorrent installation doc (same info as the admin refere…
78 [optional] You may want to setup a python virtual environment for
79 the installation.
80 To do this, run: virtualenv /path_to_install_lantorrent
81 Then, use the /path_to_install_lantorrent/bin/python binary
82 instead of your system python.
83 [optional] If you don't have Internet connectivity on the VMM, you
84 will have to manually install the simplejson library from source
85 (in the virtual env if you chose to create one).
0b8aa412 » BuzzTroll
2010-09-21 merged virga1 and master
86
aa6a4177 » priteau
2010-10-30 Improve LANTorrent docs, and sync HTML and README
87 4) install lantorrent into xinetd
8d187eaa » buzztroll
2011-05-19 repackaging lantorrent to be more pythonic
88 - the above step outputs and xinetd file like this:
89
90 ============== START WITH THE NEXT LINE ==================
91 service lantorrent
92 {
93 type = UNLISTED
94 disable = no
95 socket_type = stream
96 protocol = tcp
97 user = bresnaha
98 wait = no
99 port = 2893
100 server = /home/bresnaha/lt1/bin/ltserver
2a50462f » buzztroll
2011-08-09 patch for GH-70
101 env = HOME=/opt/nimbus
8d187eaa » buzztroll
2011-05-19 repackaging lantorrent to be more pythonic
102 }
103 =============== END WITH THE PREVIOUS LINE =================
104
105 note the 'user' value. We strongly recommend that it is not 'root'.
106 This user will be the owner of all received files (the nimbus user)
107
108 Copy the output to a file called /etc/xinetd.d/lantorrent. Once
109 this is done restart xinetd (/etc/init.d/xinetd restart).
0b8aa412 » BuzzTroll
2010-09-21 merged virga1 and master
110
495ba905 » BuzzTroll
2010-10-06 lantorrent patches so it will work with older versions of python
111 5) change the propagation method.
112 - edit the file:
113 $NIMBUS_HOME/services/etc/nimbus/workspace-service/other/authz-callout-ACTIVE.xml
114
115 and change:
116 <property name="repoScheme" value="scp" />
117 to:
118 <property name="repoScheme" value="lantorrent" />
119
aa6a4177 » priteau
2010-10-30 Improve LANTorrent docs, and sync HTML and README
120 6) restart the service: $NIMBUS_HOME/bin/nimbusctl restart
121
122 7) [optional] if the path to Nimbus on the workspace control nodes (VMMs)
0b8aa412 » BuzzTroll
2010-09-21 merged virga1 and master
123 is not /opt/nimbus you will also need to edit a configuration file on
124 all backends.
125
126 In the file:
127 <workspace control path>/control/etc/workspace-control/propagation.conf
128
129 make sure the value of:
130
131 lantorrentexe: /opt/nimbus/bin/ltclient.sh
132
3eb657a3 » priteau
2010-10-10 Fix typo
133 points to the proper location of you ltclient.sh script. This should
0b8aa412 » BuzzTroll
2010-09-21 merged virga1 and master
134 be a simple matter of changing /opt/nimbus to the path where you chose
135 to install workspace control.
136
137 ----------------
138 Protocol details
139 ----------------
140
141 requests
142 --------
143
144 transfer
145 --------
146
147 A transfer from one peer to another is performed for forming a TCP
148 connection. A json header is first sent down the socket describing the
149 transfer. It describes the files where data must be written, and the
150 other endpoints where the data must be sent. As data comes in the
151 server will store it to one or more files and forward it on to another
152 endpoint. All of this information is contained in the header. The
153 header has the following format:
154
155 {
156 host
157 port
158 length
159 requests = [ {filename, id, rename}, ]
160 destinations =
161 [ {
162 host
163 port
164 requests = [ { filename, id, rename } ]
165 block_size
166 }, ]
167 }
168
169 When the server forms a connection with the next endpoint it will form
170 another header by removing the first entry of the destination list and
171 using the values contained in it as root level values in the header
172 described above. If the destination list is empty then the server is the
173 last in the chain and no forwarding is needed.
174
175 After the header is sent the binary payload (the contents of the file)
176 is sent down the socket.
177
178 The 'requests' value of the json header contains destination
179 information. Each filename in the list is a file where data should be
180 written. The value of id is the id of the request for that file. The
181 value of rename is a boolean which determines if the file should be
182 first written to a temporary location and then moved to the final
183 location once all of the data is received. This feature is helpful in
184 monitoring file completion.
185
186 security
187 --------
188
189 Each connection to a lantorrent peer comes with a json header. Directly
190 following that header is a line of text in the following format:
191 EOH : <signature>
192
193 The value of signature is the hmac signature of everything that came
194 before the literal EOH. The password for that signature is found in the
195 lt.ini file. Every endpoint and the master repo must have the same
196 value for that password.
Something went wrong with that request. Please try again.