feat(data-catalog): Adding Hue #227
Conversation
|
Hi @tumido. Thanks for your PR. I'm waiting for a opendatahub-io member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
tumido
force-pushed
the
datacatalog-hue
branch
from
ffdf0f0
to
2686104
Compare
tumido
changed the title
tumido
force-pushed
the
datacatalog-hue
branch
from
5278e9c
to
832a685
Compare
|
/retest |
|
/retest test failures not related to the code change |
tumido
force-pushed
the
datacatalog-hue
branch
from
832a685
to
7e6008d
Compare
|
/retest |
tumido
force-pushed
the
datacatalog-hue
branch
from
7e6008d
to
a7c70f7
Compare
|
/retest |
tumido
force-pushed
the
datacatalog-hue
branch
from
a7c70f7
to
3b84872
Compare
tumido
force-pushed
the
datacatalog-hue
branch
2 times, most recently
from
bfb99e2
to
3834da7
Compare
| - ReadWriteOnce | ||
| resources: | ||
| requests: | ||
| storage: "1Gi" |
There was a problem hiding this comment.
I have a very little understanding about what the DB is actually used for - do you know if 1Gi is a reasonable size?
There was a problem hiding this comment.
It's used to store tables and metadata, we have been running 1 Gi in prod for some time now and it hasn't complained.
There was a problem hiding this comment.
Same answer as at the Thrift server PR. I've used values based on the internal data hub team expertise. 🙂
Hue stores mainly it's "user" data and preferences, (it's an old school Django app), query history, sample data cache for individual tables, data autocomplete and hive metadata cache, etc... In our experience this value is enough, though I don't really know how much lower we can go.
| @@ -0,0 +1,47 @@ | |||
| # Cloudera Hue | |||
There was a problem hiding this comment.
It seems like this does not work without thrifserver, right? If so, can you mention it in the readme and maybe even as a comment in KFDef?
There was a problem hiding this comment.
It can work without thriftserver, but you'd need to connect it to something else (yarn, Hive directly, etc) via modifying the hue.ini. Or, you can use it as an S3 explorer. Which is a standalone functionality. In our scenario and set up it requires Thrift server to be available. I'll explain it in the readme. 👍
There was a problem hiding this comment.
I've added a paragraph in there. WDYT? 🙂
There was a problem hiding this comment.
This clarification looks good to me.
tumido
force-pushed
the
datacatalog-hue
branch
2 times, most recently
from
22af6e7
to
d3720e7
Compare
tumido
force-pushed
the
datacatalog-hue
branch
from
d3720e7
to
93dad29
Compare
tumido
force-pushed
the
datacatalog-hue
branch
from
93dad29
to
adf80aa
Compare
|
I only tested the S3 browser and it worked perfectly to view & upload files. |
tumido
force-pushed
the
datacatalog-hue
branch
from
adf80aa
to
b0b57c6
Compare
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: crobby, LaVLaS, tumido The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
Update anaconda-ce-validator cronjob API level
…ndatahub-io#227) #### Motivation It may be useful for some built-in runtime adapters to have the model server's inferencing endpoint information in addition to the existing "management" port number that's passed (in case it is different). #### Modifications - Set a new `RUNTIME_DATA_ENDPOINT` env var on the built-in adapter container - Parse the `ADAPTER_PORT` env var value from the ServingRuntime `grpcEndpoint` field instead of hardcoding to 8085 Signed-off-by: Nick Hill <nickhill@us.ibm.com>
Adding Hue component Data catalog.
Part of: https://github.com/opendatahub-io/odh-manifests/issues/222, https://github.com/opendatahub-io/odh-manifests/issues/105, DATAHUB-2294
Based on reference implementation in AICoE#27 for Internal DH.
It will need further cleanup and extraction of parts into overlays later on (storage class, externalize the database). This is expected to happen in consecutive PRs.