Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
zytek committed Nov 5, 2019
1 parent e60cd2f commit b06a810
Showing 1 changed file with 7 additions and 3 deletions.
10 changes: 7 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
<img src="https://cdn.onlinewebfonts.com/svg/img_535582.png" width="128" height="128">

# s3perka - copy S3 objects between different AWS partitions (China, GovCloud)

Use this tool if you need to copy S3 objects to/from AWS China or GovCloud, which
are **different AWS partitions** and hence you **cannot** use `aws s3 sync`

It supports massive parallelism (dozens or 100+ simultaneus copies) as this has proven speeding up copying especially to AWS China.
* supports massive parallelism (dozens or 100+ simultaneus copies) as this has proven speeding up copying especially to AWS China
* only copies files from source, that do not exist or whose size on destination differs
* does not delete files on destination bucket

# Installation

Expand All @@ -13,7 +17,7 @@ cd ~/git
git clone https://github.com/zytek/s3perka
cd s3perka
```
2. Using go 1.13+ run
2. Build, using go 1.13+ for module support

```go build```

Expand All @@ -39,6 +43,6 @@ parallel=50

# Caveats

* `s3perka` downloads files on disk and then uploads them. Currently there is no "on-the-fly" copy mode
* `s3perka` downloads files on disk and then uploads them. Currently there is no "on-the-fly" (in-memory) copy mode
* make sure you have enough disk space for largest file to fit
* make sure youd HDD can handle such parallelism level

0 comments on commit b06a810

Please sign in to comment.