PHOENIX-4688 Kerberize python phoenixdb #307

pu239ppy · 2018-06-28T19:04:24Z

Lets rip out httplib and replace with requests and use requests kerberos

Notes

This PR mirrors requests kerberos until such time that the maintainers of reuests-kerberos can merge allow to specify a mech_oid to authGSSClientInit requests/requests-kerberos#115
This is trivial comparing to the integration test required
Will not work out of the box, since requires virtualenv on anaconda
Excluded the IT from list, needs to be invoked manually
mvn -Dit.test=org.apache.phoenix.end2end.SecureQueryServerPhoenixDBIT verify
Supports both virtualenv and conda install now via this hack
conda create -y -p $PY_ENV_PATH || virtualenv $PY_ENV_PATH

Standalone install

This is currently a manual procedure with a few steps due to the current version not being in a public pipy and also the fact that we had to fork requests-kerberos

Create a python virtual environment by calling conda create or virtual env
. activate the virtual environment
pip install phoenix/python/requests-kerberos
pip install phoenix/python/phoenixdv-module
You need kerberos credentials run kinit
Profit

Example script

import phoenixdb
import phoenixdb.cursor
import sys


if __name__ == '__main__':
    pqs_port = sys.argv[1]
    database_url = 'http://localhost:' + str(pqs_port) + '/'

    print "CREATING PQS CONNECTION"
    conn = phoenixdb.connect(database_url, autocommit=True, auth="SPNEGO")
    cursor = conn.cursor()

    cursor.execute("CREATE TABLE users (id INTEGER PRIMARY KEY, username VARCHAR)")
    cursor.execute("UPSERT INTO users VALUES (?, ?)", (1, 'admin'))
    cursor.execute("UPSERT INTO users VALUES (?, ?)", (2, 'user'))
    cursor.execute("SELECT * FROM users")
    print cursor.fetchall()

…while upserting

…e links for views and not for indexes

…ies in CQSI Signed-off-by: aertoria <castives@gmail.com>

…ableRef, Map<ImmutableBytesPtr,RowMutationState>> mutations

…Nakhoda)

…s (Pedro Boado)

…ableRef, Map<ImmutableBytesPtr,RowMutationState>> mutations (addendum)

…mTablesToSystemNamespaceIT

…ng Map<TableRef, Map<ImmutableBytesPtr,RowMutationState>> mutations (addendum)" This reverts commit 4e0c0a3.

…rk in SkipScanFilter

…ableRef, Map<ImmutableBytesPtr,RowMutationState>> mutations (addendum)

…a we are using in hadoop ecosystem

…statement Signed-off-by: aertoria <castives@gmail.com>

…a salted table

…rcase Schema Names)

…y setting attempt on read-only configuration(Rajeshbabu)

…y setting attempt on read-only configuration-addendum(Rajeshbabu)

…eed it

…lause using 4.10 phoenix client on 4.13 phoenix server

…ifferent client fails

…ifferent client fails (addendum)

…getExplainPlan() and pull optimize() out of getExplainPlan()

…oshihiro Suzuki) When using the thin-client in Spark, we encounter problems in that Spark is placing its own version of avatica on the classpath as well. We can relocate most of Avatica (all but the protobuf generated messages as their classnames are required to be 'org.apache.calcite.avatica.proto' presently) and hadoop-common to avoid future problems.

joshelser

Pulling this down and trying it out today. Some general thoughts:

Need instructions on building (now that it's more complicated)
How do we make sure requests-kerberos is built in a way that phoenixdb will grab it? (e.g. 0.13.0.dev0-phoenixdb)

joshelser · 2018-07-09T16:59:07Z

phoenix-queryserver/src/it/bin/test_phoenixdb.sh

+export KRB5_TRACE=/dev/stdout
+
+#echo "RUNNING KINIT"
+kinit -kt $KEYTAB_LOC $PRINC


Can we kinit to a custom location? e.g. the -c option. Then, later, we just set the variable KRB5CCNAME in the shell ENV.

This would help prevent us from bashing the user's ticket (if they already have one).

I tried something similar

File KRB5CCNAME = File.createTempFile("krb5ccname", null); kinitEnv.put("KRB5CCNAME", KRB5CCNAME.getAbsolutePath());

~~This stalled, although looking at the code now it probably should have been a directory, which is why kinit stalled~~

I just tried on the command line and MAC OS (Heimdal) kinit does not require a directory, I edited out my real user name and realm

$ export KRB5CCNAME=cc $ kinit -c cc me me@REALM.COM's password: $ klist Credentials cache: FILE:cc Principal: me@REALM.COM Issued Expires Principal Jul 10 18:08:52 2018 Jul 11 18:08:47 2018 krbtgt/REALM.COM@REALM.COM

So it looks like my code is correct and I still do not know why kinit stalled

~~I can try this again...~~

However activate will modify the environment to ensure that python is used from the virtual environment where activate was run. At this point we can

try to figure out what the environment changes are and capture them

pass the when executing python
or just continue running in the same shell, which is why I stopped attempts to make ny further reductions to the shell script

I just tried on the command line and MAC OS (Heimdal) kinit does not require a directory

Yeah, convention is to use ${tmpdir}/krb5cc_$(current-user uid).

pass the when executing python or just continue running in the same shell, which is why I stopped attempts to make ny further reductions to the shell script

Oh right, I forgot they would bash the environment. Let's just let this be for now. Will be easier to come back to it later.

joshelser · 2018-07-09T17:00:29Z

phoenix-queryserver/src/it/bin/test_phoenixdb.sh

+function cleanup {
+    set +e
+    set +u
+    kdestroy


If we use a custom directory for the kinit, then this just becomes removing that custom directory.

Not to be overly pedantic, but you would want to still pass krb5ccname and just call kdestroy to make sure proper cleanup is done. Especially in cases where memory caches are used vs file based caches. If i get java called kinit to work, I am fairly sure I can get kdestroy to work as well

Ok, cool. I didn't think kdestroy was doing more than just cleaning up those token :)

joshelser · 2018-07-09T17:01:58Z

phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/SecureQueryServerPhoenixDBIT.java

+        cmdList.add(Integer.toString(PQS_PORT));
+        cmdList.add(Paths.get(currentDirectory, "src", "it", "bin", "test_phoenixdb.py").toString());
+
+        // kinit in some random credcache


Delete this stuff?

~~I'll give this one more shot in a bit~~
deleted all references to kinit

joshelser · 2018-07-09T17:03:02Z

phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/SecureQueryServerPhoenixDBIT.java

+            LOG.info("MINIKDC PORT " + kdcPort);
+            // Render a Heimdal compatible krb5.conf
+            // Currently kinit will only try tcp if the KDC is defined as
+            // kdc = tcp/hostname:port


Nice! I learned something here :)

Yes mini KDC will render a krb5.conf file, however it is useless on MAC OS as Heimdal seemingly decided to want to specify protocols as opposed to trying them all. This has been fixed but Apple has not packaged it yet I guess

joshelser · 2018-07-09T17:03:33Z

phoenix-queryserver/src/it/java/org/apache/phoenix/end2end/SecureQueryServerPhoenixDBIT.java

+        return dest;
+    }
+
+    String kinit(String principal, File keytab, File krb5ConfFile) throws IOException{


This is unused now, right?

joshelser · 2018-07-09T17:05:02Z

python/phoenixdb-module/phoenixdb/__init__.py

-# The ASF licenses this file to You under the Apache License, Version 2.0
-# (the "License"); you may not use this file except in compliance with
-# the License.  You may obtain a copy of the License at
+# Copyright 2015 Lukas Lalinsky


Any reason for the re-add of this? We don't need this after the IP Clearance process, I think. NOTICE file should be sufficient.

I am not sure how that went back in, it is possible that I may have copied init.py from the time I was doing this work on my own before I found out that this has been moved to phoenix. I will change the header

joshelser · 2018-07-09T17:07:05Z

python/requests-kerberos/LICENSE

@@ -0,0 +1,15 @@
+ISC License


Just calling out that this is allowed: ISC is a Category-A license per https://www.apache.org/legal/resolved.html

joshelser · 2018-07-10T16:54:26Z

phoenix-queryserver/pom.xml

@@ -47,6 +47,11 @@
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-failsafe-plugin</artifactId>
+        <configuration>
+          <excludes>
+            <exclude>**/SecureQueryServerPhoenixDBIT.java</exclude>


You not intending for this test to be executed during the normal build process?

There are a few prerequisites

Either anaconda or virtual env must to be installed

System must provide either MIT or Heimdal kerberos utilities and libraries

joshelser · 2018-07-10T17:21:56Z

How do we make sure requests-kerberos is built in a way that phoenixdb will grab it? (e.g. 0.13.0.dev0-phoenixdb)

Ah, I see this is updated in python/requests-kerberos/requests_kerberos/init.py. We should update python/phoenixdb/requirements.txt to make sure we pull that 0.13.0.dev0-phoenixdb, right?

joshelser · 2018-07-10T19:17:18Z

Tried to run the test, but it failed with:

2018-07-10 13:43:55,599 ERROR [main] end2end.SecureQueryServerPhoenixDBIT(359): + . deactivate ''
2018-07-10 13:43:55,599 ERROR [main] end2end.SecureQueryServerPhoenixDBIT(359): /Users/jelser/projects/phoenix.git/phoenix-queryserver/./src/it/bin/test_phoenixdb.sh: line 12: deactivate: No such file or directory

Might have been me hacking on things... digging more.

pu239ppy · 2018-07-11T15:34:26Z

@joshelser If you are still unable to get .deactivate to work, remove it from the shell script for now and we can revisit it. Again I am being overly pedantic here, but a the shell exits, we really do not need to clean up any environment.

joshelser · 2018-07-12T17:02:56Z

If you are still unable to get .deactivate to work, remove it from the shell script for now and we can revisit it. Again I am being overly pedantic here, but a the shell exits, we really do not need to clean up any environment.

Ok. I'm looking at this again today. Thanks for the feedback so far.

The test/code looks good to me. I think we just need to update documentation to explain what we're doing (maybe a README.md in python/?)

joshelser · 2018-07-12T21:23:20Z

Ugh, I'm getting frustrated:

I have MIT kerberos on my Mac, so I unblocked myself first by just forcing the minikdc config file to be made instead of the if-branch you added, Lev.

The next thing, I get a failure trying to launch python from the virtualenv:

/Users/jelser/projects/phoenix.git/phoenix-queryserver/./src/it/bin/test_phoenixdb.sh: line 59: 66933 Illegal instruction: 4  python $PYTHON_SCRIPT $PQS_PORT

This was reproducible doing it by hand, so I thought maybe it was related to me using python-2.7.14 (old).

So, I switched over to Python-3.6.4, reinstalled everything, and I got this.

Traceback (most recent call last):
  File "/private/var/folders/4q/q02ykc2j5l1fg8nbs_sczskh0000gp/T/tmp.iUMwkyIZ/lib/python3.6/site-packages/requests_kerberos/kerberos_.py", line 2, in <module>
    import kerberos
ImportError: dlopen(/private/var/folders/4q/q02ykc2j5l1fg8nbs_sczskh0000gp/T/tmp.iUMwkyIZ/lib/python3.6/site-packages/kerberos.cpython-36m-darwin.so, 2): Symbol not found: _mempcpy
  Referenced from: /private/var/folders/4q/q02ykc2j5l1fg8nbs_sczskh0000gp/T/tmp.iUMwkyIZ/lib/python3.6/site-packages/kerberos.cpython-36m-darwin.so
  Expected in: flat namespace
 in /private/var/folders/4q/q02ykc2j5l1fg8nbs_sczskh0000gp/T/tmp.iUMwkyIZ/lib/python3.6/site-packages/kerberos.cpython-36m-darwin.so

I ran into another GH issue saying that pykerberos==1.1.14 fixed it for them, but I'm not seeing a difference locally.

How do you feel about requiring Docker, @pu239ppy? ;)

pu239ppy · 2018-07-13T12:52:21Z

@joshelser Not sure how you got MIT on Mac OS X, is it in some ports package? I can try this later on ubuntu perhaps, if you want a test with MIT. I suppose integrating docker into the mix would make things interesting. I'll try it over the weekend if I get the time.

On the other front I think the only outstanding issues we have now is

correct module version in requirements (though I am not sure it matters since we install "by hand" anyway)
README.md in phoenix/python to explain how this works

joshelser · 2018-07-13T16:10:40Z

Not sure how you got MIT on Mac OS X, is it in some ports package

Yeah, homebrew has a krb5 package which I use for running stuff locally (e.g. I put it on the PATH before the Heimdal, osx-provided variant)

I suppose integrating docker into the mix would make things interesting. I'll try it over the weekend if I get the time.

I'm playing with different versions of python now, but am just worried about the feasibility of this actually working on the general person's machine given how much I'm struggling :\

joshelser · 2018-07-13T16:18:41Z

Ah, I think my issue might have been cruft sitting in python/requests-kerberos/, the build dir and the .egg-info dir.

Getting a straightforward HTTP/401 error now. That I know how to deal with :)

joshelser · 2018-07-13T17:23:06Z

Ok, where I'm at now:

Python 2.7.15 (installed via pyenv)
Using virtualenv to circumvent the .sh script
Modified the junit test to just leave it running
Modified the junit test to just use the minikdc's kdc.conf
Pulled back the pykerberos dependency to 1.1.14 to get past an "illegal instruction error" that I get with pykerberos-1.2.1 (or whatever pip found)

This gets the phoenixdb client to actually submit the initial POST and get the WWW-Authenticate: Negotiate header back. However, my client seems to be unable to generate its challenge data from our mini kdc:

DEBUG:phoenixdb.avatica.client:POST http://localhost:60358/ '\n@org.apache.calcite.avatica.proto.Requests$CloseConnectionRequest\x12&\n$f71fb5c5-a814-4766-9691-8aeddfc0eea4' {'content-type': 'application/x-google-protobuf'}
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:60358
send: 'POST / HTTP/1.1\r\nHost: localhost:60358\r\nConnection: keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: python-requests/2.19.1\r\ncontent-type: application/x-google-protobuf\r\nContent-Length: 106\r\n\r\n\n@org.apache.calcite.avatica.proto.Requests$CloseConnectionRequest\x12&\n$f71fb5c5-a814-4766-9691-8aeddfc0eea4'
reply: 'HTTP/1.1 401 Unauthorized\r\n'
header: Date: Fri, 13 Jul 2018 17:06:02 GMT
header: WWW-Authenticate: Negotiate
header: Cache-Control: must-revalidate,no-cache,no-store
header: Content-Type: text/html; charset=ISO-8859-1
header: Content-Length: 281
header: Server: Jetty(9.2.19.v20160908)
DEBUG:urllib3.connectionpool:http://localhost:60358 "POST / HTTP/1.1" 401 281
DEBUG:requests_kerberos.kerberos_:handle_401(): Handling: 401
ERROR:requests_kerberos.kerberos_:generate_request_header(): authGSSClientStep() failed:
Traceback (most recent call last):
  File "/Users/jelser/projects/phoenix.git/python/requests-kerberos/requests_kerberos/kerberos_.py", line 235, in generate_request_header
    negotiate_resp_value)
GSSError: (('Unspecified GSS failure.  Minor code may provide more information', 851968), ('Message stream modified', 100001))
ERROR:requests_kerberos.kerberos_:(('Unspecified GSS failure.  Minor code may provide more information', 851968), ('Message stream modified', 100001))
Traceback (most recent call last):
  File "/Users/jelser/projects/phoenix.git/python/requests-kerberos/requests_kerberos/kerberos_.py", line 235, in generate_request_header
    negotiate_resp_value)
GSSError: (('Unspecified GSS failure.  Minor code may provide more information', 851968), ('Message stream modified', 100001))

I can't seem to unwrap what's wrong with the request to the KDC which is preventing that from happening. Need to find more debug...

joshelser · 2018-07-13T17:26:49Z

Turning back on KRB5_TRACE...

DEBUG:phoenixdb.avatica.client:POST http://localhost:60358/ '\n?org.apache.calcite.avatica.proto.Requests$OpenConnectionRequest\x12&\n$386e3317-e23e-4a0e-9fc6-2efaa546ffc4' {'content-type': 'application/x-google-protobuf'}
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:60358
send: 'POST / HTTP/1.1\r\nHost: localhost:60358\r\nConnection: keep-alive\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: python-requests/2.19.1\r\ncontent-type: application/x-google-protobuf\r\nContent-Length: 105\r\n\r\n\n?org.apache.calcite.avatica.proto.Requests$OpenConnectionRequest\x12&\n$386e3317-e23e-4a0e-9fc6-2efaa546ffc4'
reply: 'HTTP/1.1 401 Unauthorized\r\n'
header: Date: Fri, 13 Jul 2018 17:23:46 GMT
header: WWW-Authenticate: Negotiate
header: Cache-Control: must-revalidate,no-cache,no-store
header: Content-Type: text/html; charset=ISO-8859-1
header: Content-Length: 281
header: Server: Jetty(9.2.19.v20160908)
DEBUG:urllib3.connectionpool:http://localhost:60358 "POST / HTTP/1.1" 401 281
DEBUG:requests_kerberos.kerberos_:handle_401(): Handling: 401
[28575] 1531502626.856661: ccselect module realm chose cache FILE:/tmp/krb5cc_502 with client principal user1@EXAMPLE.COM for server principal HTTP/localhost@EXAMPLE.COM
[28575] 1531502626.856662: Getting credentials user1@EXAMPLE.COM -> HTTP/localhost@ using ccache FILE:/tmp/krb5cc_502
[28575] 1531502626.856663: Retrieving user1@EXAMPLE.COM -> HTTP/localhost@ from FILE:/tmp/krb5cc_502 with result: -1765328243/Matching credential not found (filename: /tmp/krb5cc_502)
[28575] 1531502626.856664: Retrying user1@EXAMPLE.COM -> HTTP/localhost@EXAMPLE.COM with result: -1765328243/Matching credential not found (filename: /tmp/krb5cc_502)
[28575] 1531502626.856665: Server has referral realm; starting with HTTP/localhost@EXAMPLE.COM
[28575] 1531502626.856666: Retrieving user1@EXAMPLE.COM -> krbtgt/EXAMPLE.COM@EXAMPLE.COM from FILE:/tmp/krb5cc_502 with result: 0/Success
[28575] 1531502626.856667: Starting with TGT for client realm: user1@EXAMPLE.COM -> krbtgt/EXAMPLE.COM@EXAMPLE.COM
[28575] 1531502626.856668: Requesting tickets for HTTP/localhost@EXAMPLE.COM, referrals on
[28575] 1531502626.856669: Generated subkey for TGS request: aes128-cts/86C4
[28575] 1531502626.856670: etypes requested in TGS request: aes256-cts, aes128-cts, aes256-sha2, aes128-sha2, des3-cbc-sha1, rc4-hmac, camellia128-cts, camellia256-cts
[28575] 1531502626.856672: Encoding request body and padata into FAST request
[28575] 1531502626.856673: Sending request (807 bytes) to EXAMPLE.COM
[28575] 1531502626.856674: Resolving hostname localhost
[28575] 1531502626.856675: Initiating TCP connection to stream ::1:60299
[28575] 1531502626.856676: Terminating TCP connection to stream ::1:60299
[28575] 1531502626.856677: Initiating TCP connection to stream 127.0.0.1:60299
[28575] 1531502626.856678: Sending TCP request to stream 127.0.0.1:60299
[28575] 1531502626.856679: Received answer (119 bytes) from stream 127.0.0.1:60299
[28575] 1531502626.856680: Terminating TCP connection to stream 127.0.0.1:60299
[28575] 1531502626.856681: Sending DNS URI query for _kerberos.EXAMPLE.COM.
[28575] 1531502626.856682: No URI records found
[28575] 1531502626.856683: Sending DNS SRV query for _kerberos-master._udp.EXAMPLE.COM.
[28575] 1531502626.856684: Sending DNS SRV query for _kerberos-master._tcp.EXAMPLE.COM.
[28575] 1531502626.856685: No SRV records found
[28575] 1531502626.856686: Response was not from master KDC
[28575] 1531502626.856687: TGS request result: -1765328343/Message stream modified
[28575] 1531502626.856688: Requesting tickets for HTTP/localhost@EXAMPLE.COM, referrals off
[28575] 1531502626.856689: Generated subkey for TGS request: aes128-cts/F96F
[28575] 1531502626.856690: etypes requested in TGS request: aes256-cts, aes128-cts, aes256-sha2, aes128-sha2, des3-cbc-sha1, rc4-hmac, camellia128-cts, camellia256-cts
[28575] 1531502626.856692: Encoding request body and padata into FAST request
[28575] 1531502626.856693: Sending request (807 bytes) to EXAMPLE.COM
[28575] 1531502626.856694: Resolving hostname localhost
[28575] 1531502626.856695: Initiating TCP connection to stream ::1:60299
[28575] 1531502626.856696: Terminating TCP connection to stream ::1:60299
[28575] 1531502626.856697: Initiating TCP connection to stream 127.0.0.1:60299
[28575] 1531502626.856698: Sending TCP request to stream 127.0.0.1:60299
[28575] 1531502626.856699: Received answer (119 bytes) from stream 127.0.0.1:60299
[28575] 1531502626.856700: Terminating TCP connection to stream 127.0.0.1:60299
[28575] 1531502626.856701: Sending DNS URI query for _kerberos.EXAMPLE.COM.
[28575] 1531502626.856702: No URI records found
[28575] 1531502626.856703: Sending DNS SRV query for _kerberos-master._udp.EXAMPLE.COM.
[28575] 1531502626.856704: Sending DNS SRV query for _kerberos-master._tcp.EXAMPLE.COM.
[28575] 1531502626.856705: No SRV records found
[28575] 1531502626.856706: Response was not from master KDC
[28575] 1531502626.856707: TGS request result: -1765328343/Message stream modified
ERROR:requests_kerberos.kerberos_:generate_request_header(): authGSSClientStep() failed:
Traceback (most recent call last):
  File "/Users/jelser/projects/phoenix.git/python/requests-kerberos/requests_kerberos/kerberos_.py", line 235, in generate_request_header
    negotiate_resp_value)
GSSError: (('Unspecified GSS failure.  Minor code may provide more information', 851968), ('Message stream modified', 100001))
ERROR:requests_kerberos.kerberos_:(('Unspecified GSS failure.  Minor code may provide more information', 851968), ('Message stream modified', 100001))
Traceback (most recent call last):
  File "/Users/jelser/projects/phoenix.git/python/requests-kerberos/requests_kerberos/kerberos_.py", line 235, in generate_request_header
    negotiate_resp_value)
GSSError: (('Unspecified GSS failure.  Minor code may provide more information', 851968), ('Message stream modified', 100001))

So, definitely the KDC throwing a fit and telling us to go away: [28575] 1531502626.856707: TGS request result: -1765328343/Message stream modified

joshelser · 2018-07-13T17:36:15Z

If there's a bright-side, it's something in the python code, not the test harness/setup itself. Spinning up the JDBC thin client against this setup works:

$ PHOENIX_OPTS="$PHOENIX_OPTS -Dsun.security.krb5.debug=true -Djava.security.krb5.conf=/Users/jelser/projects/phoenix.git/phoenix-queryserver/target/test-data/8bc1abb8-79fa-4beb-aa56-fe3ae4edff64/kdc/1531499757782/krb5.conf " /usr/local/lib/phoenix-4.14.0-HBase-1.4/bin/sqlline-thin.py http://localhost:60358 -a SPNEGO

joshelser · 2018-07-13T19:31:14Z

On the note about docker, trying to think about this, I feel like a docker environment that can spin up all of the necessary stuff to run both of the python tests as well as this new one will be best.

Essentially, in the docker container, we have PQS up with all of the necessary environment stuff which will make all of our current tests (and any future test) that much easier to automate.

I'm also happy to try to help with that. I know you've spent a bunch of time on this already @pu239ppy

…ontainer

joshelser · 2018-08-27T21:57:23Z

Thanks for the update, @pu239ppy. I think you got bit by 5.x moving over to master. Any chance you could throw a rebase on here? Holler if it's ready for me to look at, too. I see your new jar for testing.

zhouwei0914 · 2018-09-17T06:51:23Z

Hi pu239ppy, I have some questions, can you help me.
Questions are as follows：
1、In the fourth step, pip install phoenix/python/phoenixdv-module does not mean to install phoenixdb? I run: pip install phoenixdb
2、In the last setp, Profit: What is the operation to perform?
3、I have executed kinit in the virtual environment, but the code in the Example script is reported incorrectly. The detailed error message is as follows:
HTTP ERROR: 401
Problem accessing /. Reason:
Unauthorized

Am I wrong, please guide me, thank you very much.

pu239ppy · 2018-09-17T14:30:21Z

@zhouwei0914 This PR is no longer active, please see #344
However here is a quick recap of the issue: It has been relatively easy to rewrite python-phoenixdb to use requests, however we found out that during authentication requests-kerberos sends a kerberos OID rather then a SPNEGO OID for mechanism. There were two workarounds

Patch requests-kerberos (was never merged requests-kerberos#115 )
A new SPNEGO handler for Avatica that would override Jetty's default handler (CALCITE-1922)

It turned out that 2 Produced the desired effect that I abandoned the path of attempting to path requests-kerberos. However in the current PR #344 on top of Phoenix 5 it appears that this strategy no longer works. See additional details in later comments in PHOENIX-4688

vincentpoon and others added 30 commits November 14, 2017 10:47

PHOENIX-4373 Local index variable length key can have trailing nulls …

b2d5b4d

…while upserting

PHOENIX-4379 Upgrade code to create CHILD links should only create th…

2053905

…e links for views and not for indexes

PHOENIX-4381 Calculate the estimatedSize of MutationState incrementally

ef3bce1

PHOENIX-4361: Remove redundant argument in separateAndValidatePropert…

ddd3ef2

…ies in CQSI Signed-off-by: aertoria <castives@gmail.com>

PHOENIX-4386 Calculate the estimatedSize of MutationState using Map<T…

4944081

…ableRef, Map<ImmutableBytesPtr,RowMutationState>> mutations

PHOENIX-4384 Phoenix server jar doesn't include icu4j jars (Shehzaad …

03fc331

…Nakhoda)

PHOENIX-4387 DefaultColumnValueIT failing in non-US build environment…

ca1e17b

…s (Pedro Boado)

PHOENIX-4360 Prevent System.Catalog from splitting.

c216b66

PHOENIX-4386 Calculate the estimatedSize of MutationState using Map<T…

4e0c0a3

…ableRef, Map<ImmutableBytesPtr,RowMutationState>> mutations (addendum)

PHOENIX-4389 Flapping tests SystemTablePermissionsIT and MigrateSyste…

d46d4e5

…mTablesToSystemNamespaceIT

Revert "PHOENIX-4386 Calculate the estimatedSize of MutationState usi…

355ee52

…ng Map<TableRef, Map<ImmutableBytesPtr,RowMutationState>> mutations (addendum)" This reverts commit 4e0c0a3.

PHOENIX-672 Add GRANT and REVOKE commands using HBase AccessController

88038a2

PHOENIX-4288 Indexes not used when ordering by primary key

6807dac

PHOENIX-4322 DESC primary key column with variable length does not wo…

d77c237

…rk in SkipScanFilter

PHOENIX-3050 Handle DESC columns in child/parent join optimization

071fbce

PHOENIX-4386 Calculate the estimatedSize of MutationState using Map<T…

25359a9

…ableRef, Map<ImmutableBytesPtr,RowMutationState>> mutations (addendum)

PHOENIX-4439 QueryServer pid file name doesn't comply the usual schem…

ee728a4

…a we are using in hadoop ecosystem

PHOENIX-3837 Feature enabling to set property on an index with Alter …

1c3387d

…statement Signed-off-by: aertoria <castives@gmail.com>

PHOENIX-4415 Ignore CURRENT_SCN property if set in Pig Storer

d6e61af

PHOENIX-4397 Incorrect query results when with stats are disabled on …

72bc819

…a salted table

PHOENIX-4424 Allow users to create "DEFAULT" and "HBASE" Schema (Uppe…

c075a17

…rcase Schema Names)

PHOENIX-4446 Sequence table region opening failing because of propert…

334eb15

…y setting attempt on read-only configuration(Rajeshbabu)

PHOENIX-4446 Sequence table region opening failing because of propert…

1a19d1e

…y setting attempt on read-only configuration-addendum(Rajeshbabu)

PHOENIX-4456 queryserver script doesn't perform as expected.

90c7241

PHOENIX-4449 Bundle a copy of Argparse-1.4.0 for installations that n…

5cb02da

…eed it

PHOENIX-4460 High GC / RS shutdown when we use select query with IN c…

9d8be0e

…lause using 4.10 phoenix client on 4.13 phoenix server

PHOENIX-4468 Looking up a parent index table of a child view from a d…

c935f57

…ifferent client fails

PHOENIX-4468 Looking up a parent index table of a child view from a d…

9355a4d

…ifferent client fails (addendum)

PHOENIX-4437 Make QueryPlan.getEstimatedBytesToScan() independent of …

412329a

…getExplainPlan() and pull optimize() out of getExplainPlan()

joshelser requested changes Jul 10, 2018

View reviewed changes

joshelser reviewed Jul 10, 2018

View reviewed changes

added back the apache liscence statement

1101790

removed references to in java kinit code since it is not used

91ddda6

asfgit force-pushed the master branch from 6b363b3 to 8a874cc Compare July 25, 2018 22:35

Lev Bronshtein added 7 commits August 23, 2018 17:11

more output

11fbda5

no conda and deactivate always fails remove it since we will run in c…

120ea2a

…ontainer

use requests-gssapi

554f46a

use avatica 1.11 CALCITE-1922

cd45f8d

we no longer require this

38fef6a

kitchensink

2761188

need testing utils

c91b948

update requirements

25b2b5b

pu239ppy closed this Sep 7, 2018

pu239ppy mentioned this pull request Sep 7, 2018

PHOENIX-4688 Kerberize python phoenixdb -- Phoenix 5 #344

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PHOENIX-4688 Kerberize python phoenixdb #307

PHOENIX-4688 Kerberize python phoenixdb #307

pu239ppy commented Jun 28, 2018 •

edited

joshelser left a comment

joshelser Jul 9, 2018

pu239ppy Jul 10, 2018 •

edited

joshelser Jul 12, 2018

joshelser Jul 9, 2018

pu239ppy Jul 10, 2018 •

edited

joshelser Jul 12, 2018

joshelser Jul 9, 2018

pu239ppy Jul 10, 2018 •

edited

joshelser Jul 9, 2018

pu239ppy Jul 11, 2018

joshelser Jul 9, 2018

joshelser Jul 9, 2018

pu239ppy Jul 11, 2018

joshelser Jul 9, 2018

joshelser Jul 10, 2018

pu239ppy Jul 10, 2018

joshelser commented Jul 10, 2018

joshelser commented Jul 10, 2018

pu239ppy commented Jul 11, 2018

joshelser commented Jul 12, 2018

joshelser commented Jul 12, 2018

pu239ppy commented Jul 13, 2018 •

edited

joshelser commented Jul 13, 2018

joshelser commented Jul 13, 2018

joshelser commented Jul 13, 2018

joshelser commented Jul 13, 2018

joshelser commented Jul 13, 2018 •

edited

joshelser commented Jul 13, 2018

joshelser commented Aug 27, 2018

zhouwei0914 commented Sep 17, 2018

pu239ppy commented Sep 17, 2018

PHOENIX-4688 Kerberize python phoenixdb #307

PHOENIX-4688 Kerberize python phoenixdb #307

Conversation

pu239ppy commented Jun 28, 2018 • edited

Notes

Standalone install

Example script

joshelser left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pu239ppy Jul 10, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pu239ppy Jul 10, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pu239ppy Jul 10, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joshelser commented Jul 10, 2018

joshelser commented Jul 10, 2018

pu239ppy commented Jul 11, 2018

joshelser commented Jul 12, 2018

joshelser commented Jul 12, 2018

pu239ppy commented Jul 13, 2018 • edited

joshelser commented Jul 13, 2018

joshelser commented Jul 13, 2018

joshelser commented Jul 13, 2018

joshelser commented Jul 13, 2018

joshelser commented Jul 13, 2018 • edited

joshelser commented Jul 13, 2018

joshelser commented Aug 27, 2018

zhouwei0914 commented Sep 17, 2018

pu239ppy commented Sep 17, 2018

pu239ppy commented Jun 28, 2018 •

edited

pu239ppy Jul 10, 2018 •

edited

pu239ppy Jul 10, 2018 •

edited

pu239ppy Jul 10, 2018 •

edited

pu239ppy commented Jul 13, 2018 •

edited

joshelser commented Jul 13, 2018 •

edited