Skip to content

Commit

Permalink
Added comment about GCS chunk size
Browse files Browse the repository at this point in the history
Signed-off-by: d-kuro <kurosawa7620@gmail.com>
  • Loading branch information
d-kuro committed Mar 19, 2023
1 parent e0fe38b commit 816f1e4
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion pkg/bucket/gcs.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,12 @@ func (b *gcsBucket) Put(ctx context.Context, key string, data io.Reader, objectS
bucket := b.client.Bucket(b.name)

w := bucket.Object(key).NewWriter(ctx)
w.ChunkSize = int(decidePartSize(objectSize))

// Chunk size is set to 16 MiB by default.
// There is a trade-off between upload speed and memory space for the chunk size.
// The default value is respected here.
// https://cloud.google.com/storage/docs/resumable-uploads#go
// w.ChunkSize = int(decidePartSize(objectSize))

if _, err := io.Copy(w, data); err != nil {
return err
Expand Down

0 comments on commit 816f1e4

Please sign in to comment.