Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add HDP3 image with KMS #177

Merged
merged 1 commit into from
Sep 12, 2023
Merged

Add HDP3 image with KMS #177

merged 1 commit into from
Sep 12, 2023

Conversation

nineinchnick
Copy link
Member

@nineinchnick nineinchnick commented Sep 8, 2023

I'm testing this manually in trinodb/trino#18931

Needed for trinodb/trino#1652

Based mainly on testing/cdh5.15-hive-kerberized-kms, with some Hive 3 patches applied, mainly permission related.

@cla-bot cla-bot bot added the cla-signed label Sep 8, 2023
@nineinchnick
Copy link
Member Author

Tested by building it locally:

make testing/hdp3.1-hive-kerberized-kms
docker tag testing/hdp3.1-hive-kerberized-kms:latest ghcr.io/trinodb/testing/hdp3.1-hive-kerberized-kms:82

and running tests from suite 6 in Trino:

./testing/bin/ptl test run \
  --jdk-tmp-download-path=$HOME/tmp/ptl-jdk \
  --trino-jdk-version=temurin21-rc \
  --config=config-default \
  --environment=singlenode-kerberos-kms-hdfs-no-impersonation \
  -- -g storage_formats

End result:

tests               | ===============================================
tests               | tempto-tests
tests               | Total tests run: 106, Failures: 0, Skips: 0
tests               | ===============================================

Copy link
Contributor

@wendigo wendigo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it works, it ships 🚀

@hashhar
Copy link
Member

hashhar commented Sep 12, 2023

let's add timeouts in ci.yml. Seems hive3.1-hive has tendency to get stuck (startup times out sometimes and keeps timing out) - it's been running for almost 3 hours now.

@wendigo
Copy link
Contributor

wendigo commented Sep 12, 2023

@nineinchnick can you add timeouts? in a separate PR please

@nineinchnick
Copy link
Member Author

@hashhar here you go #180

@wendigo
Copy link
Contributor

wendigo commented Sep 12, 2023

Fetching hadoop from archive.apache.org is painfully slow :(

@hashhar
Copy link
Member

hashhar commented Sep 12, 2023

merging this PR since it seems ready to go.

@hashhar hashhar merged commit 70d01cf into trinodb:master Sep 12, 2023
15 of 17 checks passed
@nineinchnick nineinchnick deleted the hdp3-kms branch September 12, 2023 13:36
@nineinchnick nineinchnick mentioned this pull request Sep 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

3 participants