-
Notifications
You must be signed in to change notification settings - Fork 55
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
d55a1cf
commit bc5a8cf
Showing
3 changed files
with
130 additions
and
9 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,9 +1,67 @@ | ||
# S3FS | ||
|
||
S3FS is a [PyFilesystem interface](https://docs.pyfilesystem.org/) to | ||
S3FS is a [PyFilesystem](https://www.pyfilesystem.org/) interface to | ||
Amazon S3 cloud storage. | ||
|
||
As a PyFilesystem concrete class, S3FS allows you to work with S3 in the | ||
same as any other supported filesystem. | ||
As a PyFilesystem concrete class, [S3FS](http://fs-s3fs.readthedocs.io/en/latest/) allows you to work with S3 in the | ||
same way as any other supported filesystem. | ||
|
||
[Documentation](http://fs-s3fs.readthedocs.io/en/latest/) | ||
|
||
## Opening a S3FS | ||
|
||
Open an S3FS by explicitly using the constructor: | ||
|
||
```python | ||
from s3_s3fs import s3FS | ||
s3fs = S3FS('mybucket') | ||
``` | ||
|
||
Or with a FS URL: | ||
|
||
```python | ||
from fs import open_fs | ||
s3fs = open_fs('s3://mybucket') | ||
``` | ||
|
||
## Downloading Files | ||
|
||
To *download* files from an S3 bucket, open a file on the S3 | ||
filesystem for reading, then write the data to a file on the local | ||
filesystem. Here's an example that copies a file `example.mov` from | ||
S3 to your HD: | ||
|
||
```python | ||
from fs.tools import copy_file_data | ||
with s3fs.open('example.mov', 'rb') as remote_file: | ||
with open('example.mov', 'wb') as local_file: | ||
copy_file_data(remote_file, local_file) | ||
``` | ||
|
||
Although it is preferable to use the higher-level functionality in the | ||
`fs.copy` module. Here's an example: | ||
|
||
```python | ||
from fs.copy import copy_file | ||
copy_file(s3fs, 'example.mov', './', 'example.mov') | ||
``` | ||
|
||
## Uploading Files | ||
|
||
You can *upload* files in the same way. Simply copy a file from a | ||
source filesystem to the S3 filesystem. | ||
See [Moving and Copying](https://docs.pyfilesystem.org/en/latest/guide.html#moving-and-copying) | ||
for more information. | ||
|
||
## S3 URLs | ||
|
||
You can get a public URL to a file on a S3 bucket as follows: | ||
|
||
```python | ||
movie_url = s3fs.geturl('example.mov') | ||
``` | ||
|
||
## Documentation | ||
|
||
- [PyFilesystem Wiki](https://www.pyfilesystem.org) | ||
- [S3FS Reference](http://fs-s3fs.readthedocs.io/en/latest/) | ||
- [PyFilesystem Reference](https://docs.pyfilesystem.org/en/latest/reference/base.html) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,10 +1,73 @@ | ||
S3FS | ||
==== | ||
|
||
S3FS is a `PyFilesystem interface <https://docs.pyfilesystem.org/>`__ to | ||
S3FS is a `PyFilesystem <https://www.pyfilesystem.org/>`__ interface to | ||
Amazon S3 cloud storage. | ||
|
||
As a PyFilesystem concrete class, S3FS allows you to work with S3 in the | ||
same as any other supported filesystem. | ||
As a PyFilesystem concrete class, | ||
`S3FS <http://fs-s3fs.readthedocs.io/en/latest/>`__ allows you to work | ||
with S3 in the same way as any other supported filesystem. | ||
|
||
`Documentation <http://fs-s3fs.readthedocs.io/en/latest/>`__ | ||
Opening a S3FS | ||
-------------- | ||
|
||
Open an S3FS by explicitly using the constructor: | ||
|
||
.. code:: python | ||
from s3_s3fs import s3FS | ||
s3fs = S3FS('mybucket') | ||
Or with a FS URL: | ||
|
||
.. code:: python | ||
from fs import open_fs | ||
s3fs = open_fs('s3://mybucket') | ||
Downloading Files | ||
----------------- | ||
|
||
To *download* files from an S3 bucket, open a file on the S3 filesystem | ||
for reading, then write the data to a file on the local filesystem. | ||
Here's an example that copies a file ``example.mov`` from S3 to your HD: | ||
|
||
.. code:: python | ||
from fs.tools import copy_file_data | ||
with s3fs.open('example.mov', 'rb') as remote_file: | ||
with open('example.mov', 'wb') as local_file: | ||
copy_file_data(remote_file, local_file) | ||
Although it is preferable to use the higher-level functionality in the | ||
``fs.copy`` module. Here's an example: | ||
|
||
.. code:: python | ||
from fs.copy import copy_file | ||
copy_file(s3fs, 'example.mov', './', 'example.mov') | ||
Uploading Files | ||
--------------- | ||
|
||
You can *upload* files in the same way. Simply copy a file from a source | ||
filesystem to the S3 filesystem. See `Moving and | ||
Copying <https://docs.pyfilesystem.org/en/latest/guide.html#moving-and-copying>`__ | ||
for more information. | ||
|
||
S3 URLs | ||
------- | ||
|
||
You can get a public URL to a file on a S3 bucket as follows: | ||
|
||
.. code:: python | ||
movie_url = s3fs.geturl('example.mov') | ||
Documentation | ||
------------- | ||
|
||
- `PyFilesystem Wiki <https://www.pyfilesystem.org>`__ | ||
- `S3FS Reference <http://fs-s3fs.readthedocs.io/en/latest/>`__ | ||
- `PyFilesystem | ||
Reference <https://docs.pyfilesystem.org/en/latest/reference/base.html>`__ |