add config values for GCS deep storage#5875
Conversation
|
|
||
| ### Configuration | ||
|
|
||
| |Property|Possible Values|Description|Default| |
There was a problem hiding this comment.
I think we already have a document for this. Please check http://druid.io/docs/latest/development/extensions-core/hdfs.html#google-cloud-storage.
The removed configurations are actually valid when directly using GCS without Hadoop.
There was a problem hiding this comment.
We have two extensions that enable using GCS in deep storage: ../extensions-contrib/google.md and ../extensions-core/hdfs.html. When using the google extension, the druid.storage.type needs to be google, which isn't documented anywhere as far as I know. I was hoping to make it more clear how to use GCS with either this extension or the hdfs extension.
There was a problem hiding this comment.
Thanks, I got your point.
druid.storage.type should be specified as you said. But, druid.storage.storageDirectory is used only in HDFS or local deep storage types. GCS deep storage type needs to set druid.google.bucket and druid.google.prefix instead.
There was a problem hiding this comment.
Ah okay. Thanks, Jihoon!
|
It needs the following environment variable for authentication as well. It bugs me a bit at the beginning. We might need to clearly document it as well. |
|
@stevenchen3 thanks. Are you interested in making a PR for that? |
5 participants
No description provided.