Skip to content

Commit

Permalink
updated readme
Browse files Browse the repository at this point in the history
  • Loading branch information
maxmcd committed Feb 11, 2015
1 parent 772eca0 commit 12d94e8
Showing 1 changed file with 13 additions and 33 deletions.
46 changes: 13 additions & 33 deletions readme.md
Original file line number Diff line number Diff line change
@@ -1,47 +1,27 @@
# GoBox

## Setup
- Must have `GOBOX_AWS_ACCESS_KEY_ID` and `GOBOX_AWS_SECRET_ACCESS_KEY` set for aws client.
GoBox is a Dropbox clone written in Go.

The design is a client/server architecture using HTTP endpoints to communicate file change events between clients. A global journal of all file changes is kept on the server in a Postgres database.

Local file changes are hashed and sent to the server, which discerns whether or not it has a file under that hash already. If not, the client uploads the file to the server, where the file is hashed to check for integrity, and if valid uploaded to an Amazon S3 instance. All other clients are alerted that a change has been made through a UDP socket, and then the other clients request the necessary changes through an HTTP endpoint. Clients then get the necessary changes directly from the Amazon S3 instance through an S3 signed URL.

## Notes
- Must have `GOBOX_AWS_ACCESS_KEY_ID` and `GOBOX_AWS_SECRET_ACCESS_KEY` set for aws client.
- os.FileMode struct has all the information me need to handle files. symlink, permission, directory, etc....
- https://blogs.dropbox.com/tech/2014/07/streaming-file-synchronization/
- https://www.youtube.com/watch?v=PE4gwstWhmc

## Api

#### Server
Endpoints:

##### /meta:
Recieves a piece of client metadata. Logs the request, and returns if the file needs to be uploaded to s3.
```go
resp, err := http.Post("/meta", "application/json", jsonMetadataBytes)
// resp.Body #=> true
```

##### /upload:
Takes a file, computes a base256 hash value, uploads the file. Returns any errors or sucess.

##### /download:
Takes a file hash. (Confirms this user has access to that hash?) Passes the file from s3.

##### /clients:
Lists clients that this user has to check.

##### /client/1?lastCheck=dateTime:
Returns metadata for client by id after specified time.

>**Notes:**
>Does this work? Is this enough information for the filesystem to synch?
>How do we store user data on the server? Are there any security issues? What if a new user wants to download all files?
>The manifests are merely a log of all files changes. Can we combine the user manifests to create a server side record of the files that exist? Then show this to the user? Show this to new user clients?
#### Server Endpoints:

>How does the client parse the metadata and correctly download the right files?
##### POST: /file-actions/

>Right now we're using lines of json in a file. How should we be storing it instead? Database? Would a database allow us to more easily create "computed" manifests?
##### POST: /upload/

>Can we configure the meta endpoint to accept more than 1 piece of meta information?
##### POST: /download/

##### POST: /clients/

## Resources
- https://blogs.dropbox.com/tech/2014/07/streaming-file-synchronization/

0 comments on commit 12d94e8

Please sign in to comment.