Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Google Cloud Storage Offloader #2067

Closed
sijie opened this issue Jul 2, 2018 · 1 comment
Closed

Google Cloud Storage Offloader #2067

sijie opened this issue Jul 2, 2018 · 1 comment
Assignees
@jiazhai
Labels
area/tieredstorage type/feature The PR added a new feature or issue requested a new feature
Milestone
2.2.0

Comments

@sijie
Copy link
Member

sijie commented Jul 2, 2018

Feature

Implements a GCS offloader to offload pulsar segments to GCS.
@sijie sijie added type/feature The PR added a new feature or issue requested a new feature area/tieredstorage labels Jul 2, 2018
@sijie sijie added this to the 2.2.0-incubating milestone Jul 2, 2018
@zhaijack zhaijack mentioned this issue Jul 2, 2018
Merged
sijie pushed a commit that referenced this issue Jul 2, 2018
@zhaijack @sijie
GCS offload support(1): rename s3offload related classes to be reuse-…
7a10d93 
…able (#2064)

This is the first part to support `Google Cloud Storage` offload.
change:
rename Offload related classes to make them also reuse-able for GCS.

Master Issue: #2067
@sijie sijie changed the title GCS Offloader Google Cloud Storage Offloader Jul 2, 2018
@zhaijack zhaijack mentioned this issue Jul 2, 2018
Merged
This was referenced Jul 13, 2018
Merged
Merged
sijie pushed a commit that referenced this issue Jul 20, 2018
@zhaijack @sijie
GCS offload support(2): replace s3client api with jclouds related…
3c8d13c 
… api (#2065)

This is the second part to support `Google Cloud Storage` offload. 
It aims to replace "s3 client" api with "jclouds" api, and make sure unit test and integration test passed.
There will be a following change to add `Google Cloud Storage` support and related test.

change:
replace `s3client` api with `jclouds` related api in `S3ManagedLedgerOffloader`

Master Issue: #2067
sijie pushed a commit that referenced this issue Jul 24, 2018
@zhaijack @sijie
GCS offload support(3): add configs to support GCS driver (#2151)
de05a9b 
This is the third part to support Google Cloud Storage offload.
It aims to support GCS related config in `ManagedLedgerOffloader`.  It is based on PR #2065.  Please only review the commits after [e5b1f7a](e5b1f7a).

Currently it passed the real test in GCS, by test case [ManagedLedgerOffloaderTest#testGcsRealOffload](eda5097#diff-6387a2ab4cf9c9243135b5a34c8522efR584), since lack of GCS mock docker image, will try to use read GCS to do the integration test in later PR.

Master Issue: #2067
sijie pushed a commit that referenced this issue Aug 8, 2018
@zhaijack @sijie
GCS offload support(4): add documentations for GCS (#2152)
5e1ee37 
This is the 4th part to support Google Cloud Storage offload.
It aims to add documentations for GCS. And it is based on PR #2151 

Master Issue: #2067
@sijie
Copy link
Member Author

sijie commented Sep 12, 2018

all the related tasks have been merged.

@sijie sijie closed this as completed Sep 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jiazhai jiazhai

Labels
area/tieredstorage type/feature The PR added a new feature or issue requested a new feature
Projects
None yet
Milestone
2.2.0
Development

No branches or pull requests
2 participants
@sijie @jiazhai