[RANGER-2395] Add Presto plugin #36
Conversation
bolkedebruin
changed the title
|
Lgtm |
bolkedebruin
force-pushed
the
presto_plugin
branch
3 times, most recently
from
9c9c220
to
1ba70d1
Compare
bolkedebruin
force-pushed
the
presto_plugin
branch
2 times, most recently
from
4c69f07
to
fded9a1
Compare
|
Ready to commit? |
|
Close.we are gathering production stats at the moment |
This implements a plugin for Presto, a distributed SQL engine.
bolkedebruin
force-pushed
the
presto_plugin
branch
from
fded9a1
to
10d10fe
Compare
|
Is this ready? |
|
Yep pretty much So. I’m attaching it to JIRA this week. |
|
Gentle ping. |
|
Patch is here: https://issues.apache.org/jira/browse/RANGER-2395 Waiting for merge |
|
This was merged in master. |
|
I tried the https://cwiki.apache.org/confluence/display/RANGER/Presto+Plugin steps but getting below error. Ranger is working fine with Hive queries but with Presto always getting access denied even after setting up service with allow policies in the Ranger WebUI. Presto server startup log has this stacktrace but the presto server stays up. I have valid https... value for ranger.plugin.presto.policy.rest.url in ranger-presto-security.xml. Would like to know what jdbc url format you use in the Ranger UI Service connection too. 2019-06-02T00:06:15.483Z INFO main Bootstrap PROPERTY DEFAULT RUNTIME DESCRIPTION |
|
2019-06-02T00:06:15.654Z ERROR main org.apache.ranger.authorization.hadoop.config.RangerConfiguration addResourceIfReadable(ranger-presto-audit.xml): couldn't find resource file location Are key. The plugin can't read your configs. Where did you place them? |
|
they are auto generated, from memory they went in presto install direc/etc/ |
|
Verify don't guess please. To be sure you can also place them in "presto/plugins/ranger dir/ranger-plugin-impl/" (check those dirs I'm not behind a computer at the moment) |
|
The xmls went to PRESTOINSTALL/etc/ Even after copying (and chmod 777) to PRESTOINSTALL/plugins/ranger/ranger-presto-plugin-impl/ still getting same errors |
|
After changing presto's launcher.py to have the -cp line include ranger folder got further to this error:
1 error
1 error |
|
Show the contents of your presto ranger plugin folder recursively please, ls -l style. You don’t need to change launcher.py the plugin was working but couldn’t find it’s config files |
|
presto-server-313]$ ls -l plugin/ -R plugin/accumulo: plugin/atop: plugin/blackhole: plugin/cassandra: plugin/example-http: plugin/geospatial: plugin/hive-hadoop2: plugin/jmx: plugin/kafka: plugin/kudu: plugin/localfile: plugin/memory: plugin/ml: plugin/mongodb: plugin/mysql: plugin/password-authenticators: plugin/phoenix: plugin/postgresql: plugin/presto-elasticsearch: plugin/presto-thrift: plugin/ranger: plugin/raptor: plugin/redis: plugin/redshift: plugin/resource-group-managers: plugin/session-property-managers: plugin/sqlserver: plugin/teradata-functions: plugin/tpcds: plugin/tpch: ls -l plugin/ranger/ranger-presto-plugin-impl/ |
|
try putting your config files in plugin/ranger/ranger-presto-plugin-impl/conf/ |
|
bingo! its all working now :) One thing i noticed was that any ranger policy changes do not take affect until 30 seconds after policy change. I also had to copy the config files to /etc/hadoop/conf/ otherwise faced 'The value of property hadoop.security.credential.provider.path must not be null' error |
|
@bolkedebruin running SET SESSION query_max_run_time = '10m' is always denied. com.facebook.presto.spi.security.AccessDeniedException: Access Denied: Cannot set system session property query_max_run_time Could there be issue in https://github.com/apache/ranger/blob/master/plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java#L131 ? I already have ADMIN policy in ranger |
|
Could be. It might require that we cannot check against empty resources. Will have a look later |
|
@bolkedebruin if access-control.name=ranger not access-control.name=file how are you locking each principal to one user? as security.config-file can't be provided |
|
some fixes: https://github.com/apache/ranger/blob/master/plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java#L130 change checkCanSetSystemSessionProperty to have a dummy IF condition that always results in false so that session properties are never denied https://github.com/apache/ranger/blob/master/plugin-presto/src/main/java/org/apache/ranger/authorization/presto/authorizer/RangerSystemAccessControl.java#L116 change checkCanSetUser to have an IF condition (principal does not equal userName) then accessDenied. This is critical so that users can't impersonate the privileges of another user public void checkCanSetUser(final Optional principal, final String userName) { prestodb/presto#13394 remove deny in presto code |
|
@bolkedebruin have you found a way to allow CREATE VIEW but not CREATE TABLE? |
|
Is this thread still active? |
For now, i using presto version 3.16. |
|
@brucemen711 presto has changed its Security API around ~325 which nakes the plugin incompatible. Update is in the works, but I am holding ff until Row level security and column level security have landed in the presto release so I don't have to create too many patches. |
|
Thanks @bolkedebruin, im looking for it. |
|
@bolkedebruin for getting plugin to work again (not add row/col level) is it just a matter of updating version in pom.xml or code changes too? I'm guessing trinodb/trino#1624 in release 320 and trinodb/trino#171 in release 318 broke it as users mention 317 works. |
|
Code changes as the API has changed. |
|
Hi @bolkedebruin , Thanks. |
|
@brucemen711 working on/testing the patch, 331 has not been released yet which my patch is dependent on. |
|
Awesome! |
|
https://issues.apache.org/jira/browse/RANGER-2754 has a patch attached to test with Presto 331 |
|
Hi @bolkedebruin , any progress on this? |
|
@brucemen711 start chasing the committers on the mailinglist please. I have not received feedback yet |
@tooptoop4 Please, could you tell me, what configurations are you talking about? i'm facing exactly the same error |
This implements a plugin for Presto, a distributed SQL
engine.
(this was submitted to the review board as an older version, let me know what you like better)