-
Notifications
You must be signed in to change notification settings - Fork 0
/
TODO
246 lines (208 loc) · 9.9 KB
/
TODO
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
Projects are due 12 noon may 21
13% of grade
Sorage : XML DB / RDBMS
Language: Python / Java
Framework: Restlets / Django
1. Google docs Login
2. View List of Local Files
3. View Contents of Local Files
4. View List of Google Docs Files
5. View Contents of Google Docs Files
6. View List of Remote Files
7. View Contents of Remote Files
(7.5 Support both html / xml representations for the above views)
8. Search Local Files
9. Search Remote Files
MISSING
10. Backup Local Files
11. Restore Files
12. Metadata Replication
MISSING Merge stuff
A. For full credit, devices may exchange information received by other devices.
13. Data Replication
14. Discovery
A. For full credit, implement a solution that allows devices to be dynamically added to the system.
P1 Reqs
1. Login -
(DONE) A) The user will visit a main login page and your web
service will use one of the Google Account Authentication API methods to log in.
(DONE) B) Settings - Once logged in, the user may visit a settings page that will allow him/her
to set and/or change the root directory that will be exposed by the application.
In other words, if the web service receives a request to list all of a user's files, it will
provide a listing of all files and directories found under the root directory.
2. File View
(DONE) A) The main interface of your service will be a file view that lists all
files available in the root directory, and all of the files available in the
user's Google Documents account. For each file, the interface will list (at
minimum) the name of the file, the size of the file, and the last time the
file was modified.
(DONE) B) You will also provide the user with an option to open a file and
(DONE) C) an option to upload a local file to the user's Google Documents account.
(DONE) 3. Search - You will also provide the user with an option to search for files
meeting certain criteria. The user may search by file name (contains or
matches) or by modified date. Your service will provide the file view listing
only files matching the given criteria.
(DONE) 1.
/login
Allows user to authenticate using Google username and password.
(DONE) 2.
/settings
200 OK
<Settings>
<Root directory="/path/to/directory"/>
</Settings>
(DONE) 3.
/settings?rootdir={directory}
200 OK Sets root directory to directory given.
(DONE) 4.
{device_name}/xml/filelist
200 OK
<FileList>
<File>
<Path>/path/to/file</Path>
<Name>filename.ext</Name>
<Size>file size</Size>
<LastModified>last mod date</LastModified>
</File>
</FileList>
List all files on the computer identified by device_name.
(DONE) 5.
{device_name}/xml/filelist?contains={query}
200 OK
<FileList>
<File>
<Path>/path/to/file</Path>
<Name>filename.ext</Name>
<Size>file size</Size>
<LastModified>last mod date</LastModified>
</File>
</FileList>
List all files on the computer identified by device_name that have a file name that contains the query term.
(DONE) 6.
{device_name}/xml/filelist?matches={query}
200 OK
<FileList>
<File>
<Path>/path/to/file</Path>
<Name>filename.ext</Name>
<Size>file size</Size>
<LastModified>last mod date</LastModified>
</File>
</FileList>
List all files on the computer identified by device_name that have a file name that match the query term exactly.
(DONE) 7.
{device_name}/xml/filelist?modifiedsince={date}
200 OK
<FileList>
<File>
<Path>/path/to/file</Path>
<Name>filename.ext</Name>
<Size>file size</Size>
<LastModified>last mod date</LastModified>
</File>
</FileList>
List all files on the computer identified by device_name that have been modified since the date provided.
(DONE) 8.
gdocs/xml/filelist
200 OK
<FileList>
<File>
<Path>/path/to/file</Path>
<Name>filename.ext</Name>
<Size>file size</Size>
<LastModified>last mod date</LastModified>
</File>
</FileList>
List all files in the current user's Google Documents account.
(DONE) 9.
xml/gdocsupload/path/to/file/and/filename.ext
200 OK
Upload the file specified to the current user's Google Documents account.
10. (DONE)
xml/file/path/to/file/and/filename.ext
200 OK
File Contents Return contents of specified file.
HTML INTERFACE
(DONE) 11. For each /xml URL above, you will also support html/{URL} where html takes the
place of xml. The content returned in response to the HTML request will be the
XML content transformed into the appropriate HTML format for viewing in a
browser.
P2 Reqs
(DONE) 1. Login - The user must use his/her Google credentials to log in to your application
(DONE) 2. View List of Local Files - See Project
(DONE) 3. View Contents of Local Files - See Project 1.
(DONE) 4. View List of Google Docs Files - See Project 1.
(DONE) 5. View Contents of Google Docs Files - See Project 1.
6. (DONE) View List of Remote Files - The user may select any of his/her devices and
see a list of files available on that device. Note, you will serve cached
information that is potentially stale.
7. View Contents of Remote Files - If the remote device is online, allow the
user to view files stored on that device. Optionally, serve cached versions of
the requested files.
(DONE) 8. Search Local Files - See Project 1.
9. Search Remote Files - Allow the user to search the cached list of remote
files. Support the searches specified for Project 1: contains, matches,
modifiedsince.
10. Backup Local Files - Allow the user to request that local files be backed
up on other devices. The user will specify the desired number of replicas.
Your service will, in the background, choose N other devices and send a copy of
the file to those devices. Restore Files - Allow the user to restore a file
that has been deleted locally, but that was backed up before it was deleted.
If the user asks to restore a file, your server will contact one of the devices
on which the file has been backed up, and will download the file and put it
back in its original location.
Requirements
(DONE) 12. Metadata Replication - Devices will use a push-based mechanism to exchange
Problems?
13. FileList information . Periodically, each online device will send its own
information to all of the other devices in the system.
For a maximum grade of B, devices will send only information generated by the
local update thread. For full credit, devices may exchange information
received by other devices. In other words, you will support the following
scenario: device A sends its local information to device B while device C is
offline. Device A goes offline and, later, device C comes online. Device B
sends its local information to device C, and also sends the information it has
cached from device A. To fully support this scenario, you will need to ensure
that newer data is not overwritten by older data, for example by using Lamport
timestamps for versioning.
14. Data Replication - A user may specify the number of replicas he/she wants
to create for a particular file. If the user has requested more replicas than
there are online devices, your service will create as many replicas as possible
at the time of the request, and create new replicas as other devices come
online.
(DONE) 15. Discovery - You must implement a mechanism for devices to discover one another.
Conceptually, the system is made up of several devices (i.e., all of the
devices in a user's collection or a set of devices supporting multiple users
but under a single administrative domain). The devices in the system maintain
a neighbor table with the IP addresses of the other devices in the system. For
a maximum grade of B, statically configure the set of devices, for example by
using a config file that is manually generated by an administrator. For full
credit, implement a solution that allows devices to be dynamically added to the
system. One option is to implement a name server. Each device is statically
configured with the IP address of the name server. Devices register with the
name server when they come online and may ask the name server for a list of
devices in the system. For extra credit, support secure communication among
devices by using your name server as a certificate authority (CA). Each device
in the system is statically configured with a key it can use to communicate
with the CA via SSL. Each device has a public/private key pair. When it
registeres with the CA, it sends it public key, which can be retrieved by other
devices in the system. To communicate securely with another device, a device
retrieves a public key from the CA.
Other Notes
You will need to decide how and where you will store replicas. For example,
you might allow the user to configure the "Backup" directory where other
devices may store replicas.
Extra:
Write tests (unit?) to exercise each project requirement
Comment methods
reorganize utils / methods, split views.py up
DEFERRED
3. (EH, maybe not such a big priority, we can probably assume a single user ... )
Need to multiplex files on the user so that multiple (different users) can log in
and see their own set of files, currently its global, all open
DONE
Two Problems
(DONE) 1. Worker doesnt watch fs for changes
(DONE) 2. nested dirs arent supported
Write log() which prepends messages w/ timestamp and logging device name
Enable admin view for filesync