Add proxy support to the dataset container #2512 #2513
Conversation
|
|
charts/graphscope/values.yaml
Outdated
| @@ -70,6 +70,9 @@ engines: | |||
| enabled: False | |||
| image: | |||
| name: dataset | |||
| # proxy: | |||
There was a problem hiding this comment.
Could you add a comment here for what is proxy for, and what's the target scenario, it makes people or developers like me could understand this more easily
| nargs="?", | ||
| const="{}", | ||
| default="{}", | ||
| help="Proxy for worker dataset." |
There was a problem hiding this comment.
And please make this help more meaningful. That would be very helpful.
| @@ -144,6 +145,8 @@ def __init__( | |||
| self._preemptive = preemptive | |||
| self._vineyard_shared_mem = vineyard_shared_mem | |||
|
|
|||
| self._dataset_proxy = dataset_proxy | |||
There was a problem hiding this comment.
I prefer to let the dataset_proxy follow the logic of volumes, they are both json strings. i.e.
- loads the str to json in here, not in the
EngineCluster(dataset_proxy=json.loads(dataset_proxy), ...); - b64encode it in the chart then decode here.
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #2513 +/- ##
=======================================
Coverage 72.62% 72.62%
=======================================
Files 88 88
Lines 9816 9816
=======================================
Hits 7129 7129
Misses 2687 2687 Continue to review full report in Codecov by Sentry.
|
2. add help info for argument --dataset_proxy 3. change logic of dataset_proxy, input with base64 and decode in EngineCluster's init function
|
|
LGTM, we could merge this after the CI check. |
|
Merged. Thanks for your contribution @YANGBoSunning ! |
|
Thanks for your review, good luck! |
|
I'm sorry about that i missed one line code in last commit and i have add it in branch YANGBoSunning:pr-2512. Please fix it. @siyuan0322 |
Got it 😂 , thank you Pushed #2525 |
This PR adds an option of configuring the proxy service address for the dataset container should be added to the configuration file of helm
The dataset container of graphscope provides the function of obtaining sample dataset online, but there are many inconveniences in accessing the external network to obtain data in many Intranet environments.
Fixes #2512