update documentation

docs/examples/hosts

@@ -0,0 +1,11 @@
+127.0.0.1	localhost
+127.0.1.1	MobileBastard
+127.0.0.1 s3.amazonaws.com
+127.0.0.1 s3.localhost
+
+# The following lines are desirable for IPv6 capable hosts
+::1     ip6-localhost ip6-loopback
+fe00::0 ip6-localnet
+ff00::0 ip6-mcastprefix
+ff02::1 ip6-allnodes
+ff02::2 ip6-allrouters

docs/examples/s3cfg

@@ -0,0 +1,44 @@
+[default]
+access_key = 
+bucket_location = US
+cloudfront_host = cloudfront.amazonaws.com
+default_mime_type = binary/octet-stream
+delete_removed = False
+dry_run = False
+enable_multipart = True
+encoding = UTF-8
+encrypt = False
+follow_symlinks = False
+force = False
+get_continue = False
+gpg_command = /usr/bin/gpg
+gpg_decrypt = %(gpg_command)s -d --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
+gpg_encrypt = %(gpg_command)s -c --verbose --no-use-agent --batch --yes --passphrase-fd %(passphrase_fd)s -o %(output_file)s %(input_file)s
+gpg_passphrase = 
+guess_mime_type = True
+host_base = localhost:8080
+host_bucket = %(bucket)s.localhost:8080
+human_readable_sizes = False
+invalidate_on_cf = False
+list_md5 = False
+log_target_prefix = 
+mime_type = 
+multipart_chunk_size_mb = 15
+preserve_attrs = True
+progress_meter = True
+proxy_host = 
+proxy_port = 0
+recursive = False
+recv_chunk = 4096
+reduced_redundancy = False
+secret_key = 
+send_chunk = 4096
+simpledb_host = sdb.amazonaws.com
+skip_existing = False
+socket_timeout = 300
+urlencoding_mode = normal
+use_https = False
+verbosity = WARNING
+website_endpoint = http://%(bucket)s.localhost:8080/
+website_error = 
+website_index = index.html

readme.md

@@ -11,6 +11,7 @@ Archistar is a  multi-cloud prototype written during the *tada* Archistar projec
 - uses [archistar-smc](https://github.com/archistar/archistar-smc) for secret sharing, currently test-cases utilize Sharmir's Secret Sharing
 - Multiple backend storage options (memory-only, file-backed, S3-backed)
 - TLS authentication and encryption between communication partners -- key validation is not implemented yet
+- client-side S3 interface
 
 ## Design Goals
 
@@ -20,65 +21,61 @@ Archistar is a  multi-cloud prototype written during the *tada* Archistar projec
 
 ## Running archistar
 
-This example uses the ftp demostrator which consists of four locally started storage servers. Each of the storage servers uses a local filesystem backend for data storage (which can be found under `/tmp/test-ftp-filesystem/1..4`).
+This example uses the ftp demostrator which consists of four locally started storage servers. Each of the storage servers uses a local filesystem backend for data storage (which can be found under `/var/spool/archistar/test-s3/1..4`).
 
-First start the fake-FTP server frontend:
+First start the fake-S3 server frontend:
 
 ```bash
-$ mvn compile
-   ...
-$ mvn exec:java -Dexec.mainClass="at.ac.ait.archistar.bin.ArchistarS3"
-
+$ mvn exec:java -Dexec.mainClass="at.ac.ait.archistar.frontend.ArchistarS3"
 [INFO] Scanning for projects...
-[INFO]                                                                         
+[INFO] Searching repository for plugin with prefix: 'exec'.
 [INFO] ------------------------------------------------------------------------
-[INFO] Building archistar 1.0-SNAPSHOT
+[INFO] Building archistar
+[INFO]    task-segment: [exec:java]
 [INFO] ------------------------------------------------------------------------
-[INFO] 
-[INFO] >>> exec-maven-plugin:1.2.1:java (default-cli) @ archistar >>>
-[INFO] 
-[INFO] <<< exec-maven-plugin:1.2.1:java (default-cli) @ archistar <<<
-[INFO] 
-[INFO] --- exec-maven-plugin:1.2.1:java (default-cli) @ archistar ---
-[at.ac.ait.archistar.bin.ArchistarFTP.main()] INFO at.ac.ait.archistar.bin.ArchistarFTP - Starting archistar storage engine
-[Thread-6] INFO at.ac.ait.archistar.bft.ozymandias.server.OzymandiasServer - successful transactions: 0.0
-[Thread-6] INFO at.ac.ait.archistar.bft.ozymandias.server.OzymandiasServer - server: 0 transaction length: 0ms
-[at.ac.ait.archistar.bin.ArchistarFTP.main()] WARN at.ac.ait.archistar.metadata.SimpleMetadataService - creating and syncing a new database
-[at.ac.ait.archistar.bin.ArchistarFTP.main()] INFO at.ac.ait.archistar.bin.ArchistarFTP - Starting FTP server on port 30022
-[Thread-9] INFO org.mockftpserver.stub.StubFtpServer - Starting the server on port 30022
+[INFO] Preparing exec:java
+[INFO] No goals needed for project - skipping
+[INFO] [exec:java {execution: default-cli}]
+[at.ac.ait.archistar.frontend.ArchistarS3.main()] INFO at.ac.ait.archistar.frontend.ArchistarS3 - Starting archistar storage engine
+[Thread-7] INFO at.ac.ait.archistar.bft.BftEngine - successful transactions: 0.0
+[Thread-7] INFO at.ac.ait.archistar.bft.BftEngine - server: 0 transaction length: 0ms
 ```
 
-Now you can use a standard ftp program to access the server. Please note that there's currenlty no user authentication so you can pass anything you want as username and password:
+To use the s3cmd program to access the server you'll need to adopt your /etc/hosts configuration file. The s3cmd uses hard-coded amazon server urls so we redirect these to our local server by adding the following lines to `/etc/hosts` (an example configuration file can be found at docs/examples/hosts):
 
-```bash
-$ ftp localhost 30022
-Connected to localhost (127.0.0.1).
-220 Service ready for new user. (MockFtpServer 2.4; see http://mockftpserver.sourceforge.net)
-Name (localhost:andy): 
-331 User name okay, need password.
-Password:
-501 Syntax error in parameters or arguments.
-Login failed.
-Remote system type is "WINDOWS".
-ftp> ls
-227 Entering Passive Mode (127,0,0,1,163,241).
-202
-ftp> put /etc/passwd tmp
-local: /etc/passwd remote: tmp
-227 Entering Passive Mode (127,0,0,1,218,64).
-150 File status okay; about to open data connection.
-226 Closing data connection. Requested file action successful.
-2508 bytes sent in 9.5e-05 secs (26400.00 Kbytes/sec)
-ftp> get tmp some-file.txt
-local: some-file.txt remote: tmp
-227 Entering Passive Mode (127,0,0,1,169,160).
-150 File status okay; about to open data connection.
-226 Closing data connection. Requested file action successful.
-2508 bytes received in 0.103 secs (24.27 Kbytes/sec)
-ftp> ls
-227 Entering Passive Mode (127,0,0,1,204,163).
-202 tmp
-ftp> 
+```
+127.0.0.1 s3.amazonaws.com
+127.0.0.1 s3.localhost
+```
+
+s3cmd needs a configuration file, an example file can be found at docs/examples/s3cfg.
+
+Now you can use s3cmd to access the server (note that currently fake_bucket is hard coded, also I already had two files in my bucket):
+
+``` bash
+$ s3cmd -c docs/examples/s3cfg ls s3://fake_bucket
+2006-02-03 16:41       140   s3://fake_bucket/iptables-rule.txt
+2006-02-03 16:41      1151   s3://fake_bucket/link-list.txt
+
+$ echo "testdata" > testfile
+$ md5sum testfile
+73d643ec3f4beb9020eef0beed440ad0  testfile
+
+$ s3cmd -c docs/examples/s3cfg put testfile s3://fake_bucket/testfile
+testfile -> s3://fake_bucket/testfile  [1 of 1]
+ 9 of 9   100% in    0s    48.76 B/s  done
+
+$ s3cmd -c docs/examples/s3cfg ls s3://fake_bucket
+2006-02-03 16:41       140   s3://fake_bucket/iptables-rule.txt
+2006-02-03 16:41      1151   s3://fake_bucket/link-list.txt
+2006-02-03 16:41         9   s3://fake_bucket/testfile
+
+$ s3cmd -c docs/examples/s3cfg get s3://fake_bucket/testfile testfile.2
+s3://fake_bucket/testfile -> testfile.2  [1 of 1]
+ 9 of 9   100% in    0s   416.49 B/s  done
+
+$ md5sum testfile.2 
+73d643ec3f4beb9020eef0beed440ad0  testfile.2
 ```
 
 ## Developing archistar