Permalink
Newer
Older
100644 166 lines (130 sloc) 6.31 KB
1
==========
2
Lantorrent
3
==========
4
5
Lantorrent is a file distribution protocol integrated into the Nimbus
6
IaaS toolkit. It works as a means to multi-cast virtual machine images
7
to many backend nodes. The protocol is optimized for propagating
8
virtual machine images (typically large files) from a central repository
9
across a LAN to many virtual machine monitor nodes.
10
11
Lantorrent works best for the following scenarios:
12
13
1) large file transfers (VM images are typically measured in gigabytes)
14
2) local area switched network (typical for data center computer racks)
15
3) file recipients are willing peers.
16
4) many endpoints request the same file at roughly the same time
17
18
-----------------
19
Protocol overview
20
-----------------
21
22
When an endpoint wants a file it submits a request to a central agent.
23
This agent aggregates request for files so that they can be sent out in
24
an efficient single multi-cast session. Each request for a source file
25
is stored until either N request on that file have been made or N'
26
seconds have passed since the last request on that source file has been
27
made. This allows for a user to request a single file in several
28
unrelated session yet still have the file transfered in an efficient
29
multi-cast session.
30
31
Once N requests for a given source file have been made or N' seconds
32
have passed the destination set for the source file is determined. A
33
chain of destination endpoints is formed such that each node receives
34
from and sends to one other node. The first node receives from the
35
repository and send to a peer node, that peer node sends to another,
36
and so on until all receive the file. In this way all links of the
37
switch are utilized to send directly to another endpoint in the switch.
38
This results in the most efficient transfer on a LAN switched network.
39
40
Often times in a IaaS system a single network endpoint (VMM) will want
41
multiple copies of the same file. Each file is booted as a virtual
42
machine and that virtual machine will make distinct changes to that file
43
as it runs, thus it needs it own copy of the file. However that file
44
does not need to be transfered across the network more than once.
45
Lantorrent will send the file to each endpoint once and instruct that
46
endpoint to write it to multiple files if needed.
47
48
-----------------------------
49
Enabling Lantorrent in Nimbus
50
-----------------------------
51
52
Lan torrent is part of the nimbus distribution as of Nimbus 2.XXX.
53
However, due to system administrative overhead it is not enabled by
54
default. To enable Lantorrent in nimbus there are a few configurations
55
changes that must be made.
56
57
1) edit $NIMBUS_HOME/nimbus-setup.conf
58
change lantorrent.enabled: False -> lantorrent.enabled: True
59
60
2) edit $NIMBUS_HOME/services/etc/nimbus/workspace-service/other/common.conf
61
change the value of propagate.extraargs:
62
propagate.extraargs=$NIMBUS_HOME/lantorrent/bin/lt-request
63
64
be sure to expand $NIMBUS_HOME to its full and actual path.
65
66
3) xinetd
67
Lantorrent is run out of xinetd thus it must be installed on all VMMs.
68
69
4) install lantorrent on VMM
70
- recursively copy $NIMBUS_HOME/lantorrent to /opt/nimbus/lantorrent.
71
- run ./vmm-install.sh on each node
72
either run it as your workspace control user or specify the workspace
73
control user as the first and only argument to the script.
74
75
4.1) install lantorrent into xinetd
76
- the vmm-install,sh script creates the file lantorrent.inet. This
77
file is ready to be copied into /etc/xinetd.d/. Once this is done
78
restart xinetd (/etc/init.d/xinetd restart).
79
80
5) change the propagation method.
81
- edit the file:
82
$NIMBUS_HOME/services/etc/nimbus/workspace-service/other/authz-callout-ACTIVE.xml
83
84
and change:
85
<property name="repoScheme" value="scp" />
86
to:
87
<property name="repoScheme" value="lantorrent" />
88
89
6) [optional] if the path to nimbus on the workspace control nodes (VMMs)
90
is not /opt/nimbus you will also need to edit a configuration file on
91
all backends.
92
93
In the file:
94
<workspace control path>/control/etc/workspace-control/propagation.conf
95
96
make sure the value of:
97
98
lantorrentexe: /opt/nimbus/bin/ltclient.sh
99
Oct 10, 2010
100
points to the proper location of you ltclient.sh script. This should
101
be a simple matter of changing /opt/nimbus to the path where you chose
102
to install workspace control.
103
104
----------------
105
Protocol details
106
----------------
107
108
requests
109
--------
110
111
transfer
112
--------
113
114
A transfer from one peer to another is performed for forming a TCP
115
connection. A json header is first sent down the socket describing the
116
transfer. It describes the files where data must be written, and the
117
other endpoints where the data must be sent. As data comes in the
118
server will store it to one or more files and forward it on to another
119
endpoint. All of this information is contained in the header. The
120
header has the following format:
121
122
{
123
host
124
port
125
length
126
requests = [ {filename, id, rename}, ]
127
destinations =
128
[ {
129
host
130
port
131
requests = [ { filename, id, rename } ]
132
block_size
133
}, ]
134
}
135
136
When the server forms a connection with the next endpoint it will form
137
another header by removing the first entry of the destination list and
138
using the values contained in it as root level values in the header
139
described above. If the destination list is empty then the server is the
140
last in the chain and no forwarding is needed.
141
142
After the header is sent the binary payload (the contents of the file)
143
is sent down the socket.
144
145
The 'requests' value of the json header contains destination
146
information. Each filename in the list is a file where data should be
147
written. The value of id is the id of the request for that file. The
148
value of rename is a boolean which determines if the file should be
149
first written to a temporary location and then moved to the final
150
location once all of the data is received. This feature is helpful in
151
monitoring file completion.
152
153
security
154
--------
155
156
Each connection to a lantorrent peer comes with a json header. Directly
157
following that header is a line of text in the following format:
158
EOH : <signature>
159
160
The value of signature is the hmac signature of everything that came
161
before the literal EOH. The password for that signature is found in the
162
lt.ini file. Every endpoint and the master repo must have the same
163
value for that password.
164
165