-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
7dde87e
commit 1757ae0
Showing
3 changed files
with
40 additions
and
3 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,38 @@ | ||
## S3 Browser [![Build Status](https://travis-ci.org/andrewgross/s3browser.svg?branch=master)](https://travis-ci.org/andrewgross/s3browser) [![Coverage Status](https://coveralls.io/repos/github/andrewgross/s3browser/badge.svg?branch=master)](https://coveralls.io/github/andrewgross/s3browser?branch=master) | ||
|
||
|
||
S3Browser is a tool to help you browse your S3 Buckets like a local filesystem. It features `cd`, `ls`, and `pwd` for now, with some added bonuses around showing rollups for directory sizes and the most recently modified file. For now it is read only, though if you want any features feel free to suggest them. | ||
|
||
|
||
### Installation | ||
|
||
``` | ||
pip install s3browser | ||
``` | ||
|
||
### Usage | ||
|
||
You can pass access keys to `s3browser` directly, or just let it pick them up from your environment. It uses `boto` under the hood so you can use an existing configurations for that. | ||
|
||
``` | ||
usage: s3browser [-h] [--access-key-id ACCESS_KEY_ID] | ||
[--secret-access-key SECRET_ACCESS_KEY] | ||
Run S3Browser | ||
optional arguments: | ||
-h, --help show this help message and exit | ||
--access-key-id ACCESS_KEY_ID | ||
AWS_ACCESS_KEY_ID used by Boto | ||
--secret-access-key SECRET_ACCESS_KEY | ||
AWS_SECRET_ACCESS_KEY used by Boto | ||
``` | ||
|
||
Once you are in the CLI, it will automatically load a list of all of your available S3 buckets. You can use the `help` command to get detailed information for each of the commands. | ||
|
||
|
||
### Gotchas | ||
|
||
`s3browser` is written in Python, so it is not the most efficient. For really large buckets be prepared to wait a while for it to complete. S3 requires us to page through all the files to retrieve them, and currently that is done serially in chunks of 1000. Additionally, the internal representation of each S3 Key is ~800 Bytes, once you use `refresh` on a bucket with millions of keys, expect some memory pressure. | ||
|
||
I have successfully browsed ~15mm keys on my dev machine with 16GB of RAM, of which python used ~12GB. If key retrieval speed or memory usage are big issues for you, feel free to open a ticket and we can spend the time to find better ways to implement the internal structures so they are more compact! |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,4 +2,4 @@ | |
from __future__ import unicode_literals | ||
|
||
__title__ = 's3browser' | ||
__version__ = '0.0.11' | ||
__version__ = '0.0.1' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters