Switched Accumulo Dependency to 1.7.1 and ported AccumuloStore Class to work with accumulo 1.7.1 #97

vaibhavthapliyal · 2017-02-17T08:45:04Z

No description provided.

work with accumulo 1.7.1

lewismc · 2017-02-17T17:24:12Z

@vaibhavthapliyal thank you for this, can you please format the code in 2 space indents? Thank you.

work with 1.7.1

# Conflicts: # gora-accumulo/src/main/java/org/apache/gora/accumulo/store/AccumuloStore.java

vaibhavthapliyal · 2017-02-18T08:13:51Z

@lewismc Can you check it now, if it's formatted correctly?

lewismc · 2017-02-21T23:10:51Z

Yes this looks much better. I'll try and review shortly.

vaibhavthapliyal · 2017-02-22T03:57:36Z

Thanks. Will be looking forward to hearing from you.

lewismc · 2017-02-22T04:32:56Z

Hi @vaibhavthapliyal when I run the test suite, I get the following

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.gora.accumulo.store.AccumuloStoreTest
2017-02-21 20:15:03,855 INFO  [main] store.DataStoreTestBase (DataStoreTestBase.java:setUpClass(67)) - setting up class
2017-02-21 20:15:03,857 INFO  [main] gora.GoraTestDriver (GoraAccumuloTestDriver.java:setUpClass(50)) - Starting Accumulo MiniAccumuloCluster...
2017-02-21 20:15:07,283 INFO  [main] impl.MiniAccumuloClusterImpl (MiniAccumuloClusterImpl.java:start(609)) - Starting MAC against instance goraTest and zookeeper(s) localhost:56321.
2017-02-21 20:15:09,847 INFO  [main] store.DataStoreTestBase (DataStoreTestBase.java:setUp(90)) - setting up test
2017-02-21 20:15:10,671 WARN  [main] client.ClientConfiguration (ClientConfiguration.java:loadFromSearchPath(227)) - Found no client.conf in default paths. Using default client configuration values.
2017-02-21 20:15:10,754 INFO  [main] zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
2017-02-21 20:15:10,755 INFO  [main] zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:host.name=lmc-056430
2017-02-21 20:15:10,755 INFO  [main] zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.version=1.8.0_91
2017-02-21 20:15:10,756 INFO  [main] zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.vendor=Oracle Corporation
2017-02-21 20:15:10,756 INFO  [main] zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.home=/Library/Java/JavaVirtualMachines/jdk1.8.0_91.jdk/Contents/Home/jre
2017-02-21 20:15:10,757 INFO  [main] zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.class.path=/usr/local/gora/gora-accumulo/target/test-classes:/usr/local/gora/gora-accumulo/target/classes
...
/Users/lmcgibbn/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-hs/2.5.2/hadoop-mapreduce-client-hs-2.5.2.jar:
2017-02-21 20:15:10,759 INFO  [main] zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.library.path=/Users/lmcgibbn/Library/Java/Extensions:/Library/Java/Extensions:/Network/Library/Java/Extensions:/System/Library/Java/Extensions:/usr/lib/java:.
2017-02-21 20:15:10,760 INFO  [main] zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.io.tmpdir=/var/folders/mw/f_p4r21n5ld16hnyd7j8r8w4fv8362/T/
2017-02-21 20:15:10,760 INFO  [main] zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:java.compiler=<NA>
2017-02-21 20:15:10,760 INFO  [main] zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:os.name=Mac OS X
2017-02-21 20:15:10,761 INFO  [main] zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:os.arch=x86_64
2017-02-21 20:15:10,763 INFO  [main] zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:os.version=10.11.6
2017-02-21 20:15:10,764 INFO  [main] zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:user.name=lmcgibbn
2017-02-21 20:15:10,764 INFO  [main] zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:user.home=/Users/lmcgibbn
2017-02-21 20:15:10,764 INFO  [main] zookeeper.ZooKeeper (Environment.java:logEnv(100)) - Client environment:user.dir=/usr/local/gora/gora-accumulo
2017-02-21 20:15:10,765 INFO  [main] zookeeper.ZooKeeper (ZooKeeper.java:<init>(438)) - Initiating client connection, connectString=localhost:56321 sessionTimeout=30000 watcher=org.apache.accumulo.fate.zookeeper.ZooSession$ZooWatcher@4009e306
2017-02-21 20:15:10,798 INFO  [main-SendThread(localhost:56321)] zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(966)) - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:56321. Will not attempt to authenticate using SASL (unknown error)
2017-02-21 20:15:10,821 WARN  [main-SendThread(localhost:56321)] zookeeper.ClientCnxn (ClientCnxn.java:run(1089)) - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
2017-02-21 20:15:10,930 INFO  [main-SendThread(localhost:56321)] zookeeper.ClientCnxn (ClientCnxn.java:logStartConnect(966)) - Opening socket connection to server localhost/127.0.0.1:56321. Will not attempt to authenticate using SASL (unknown error)
2017-02-21 20:15:10,933 INFO  [main-SendThread(localhost:56321)] zookeeper.ClientCnxn (ClientCnxn.java:primeConnection(849)) - Socket connection established to localhost/127.0.0.1:56321, initiating session
2017-02-21 20:15:10,944 INFO  [main-SendThread(localhost:56321)] zookeeper.ClientCnxn (ClientCnxn.java:onConnected(1207)) - Session establishment complete on server localhost/127.0.0.1:56321, sessionid = 0x15a64079dd40004, negotiated timeout = 30000
2017-02-21 20:17:13,269 WARN  [Thrift Connection Pool Checker] impl.ThriftTransportPool (ThriftTransportPool.java:checkForStuckIO(206)) - Thread "main" stuck on IO to LMC-056430:54933 (0) for at least 120171 ms

Tests then hang. I found some existing issues similar to this, I'm seeing if I can debug and get further with this.
I did notice that we don't use a client.conf, this may be something we need to implement in the Driver setup or via client.conf. Not sure yet.

lewismc · 2017-02-22T04:33:33Z

I also created https://issues.apache.org/jira/browse/GORA-504 to track the issue

vaibhavthapliyal · 2017-02-22T06:54:48Z

Hi,
Do the tests proceed further in your case after this line?
2017-02-21 20:17:13,269 WARN [Thrift Connection Pool Checker] impl.ThriftTransportPool (ThriftTransportPool.java:checkForStuckIO(206)) - Thread "main" stuck on IO to LMC-056430:54933 (0) for at least 120171 ms

I get a build sucess even after the Connection Refused. I'll check for some fixes and revert back.

lewismc · 2017-02-22T07:07:13Z

Do the tests proceed further in your case after this line?

No they stall... can someone else check for tests continuing and passing with this patch?

@renato2099 can you please check?

vaibhavthapliyal · 2017-02-22T09:03:17Z

@lewismc Hi, I think the issue is with your /etc/hosts file.

Make sure you have these entries in your host file.
127.0.0.1 localhost
127.0.0.1 your_hostname
Can you please check if this fixes the issue?
A reference to the solution of this can be found here:
https://community.cloudera.com/t5/Storage-Random-Access-HDFS/Problems-running-a-write-query-on-an-accumulo-cluster/td-p/21786/page/2

renato2099 · 2017-02-22T12:50:26Z

@vaibhavthapliyal @lewismc I applied the PR and gora-accumulo passed all tests successfully 👍

renato2099 · 2017-02-22T12:53:56Z

@lewismc regarding the 2space vs 4space, the only issue with this is that now the whole AccumuloStore looks like if it was modified, didn't we had a style check on maven at some point? maybe we should have a JIRA to just make the whole code standard and keep the changes of this PR only addressing the ones that it's supposed to

lewismc · 2017-02-22T15:18:14Z

Regarding the formatting... yeah I agree. That's my fault, I thought AccumuloStore was already formatted 2 space indents. We should probably use the maven formatting plugin with out code template in a subsequent issue.

On Wed, Feb 22, 2017 at 4:53 AM Renato Marroquin ***@***.***> wrote: @lewismc <https://github.com/lewismc> regarding the 2space vs 4space, the only issue with this is that now the whole AccumuloStore looks like if it was modified, didn't we had a style check on maven at some point? maybe we should have a JIRA to just make the whole code standard and keep the changes of this PR only addressing the ones that it's supposed to — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#97 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABHJl1EHr9v5PUkq3mjn0w2XA8nIqeg8ks5rfC_kgaJpZM4MEAkC> .

-- *Lewis* Dr. Lewis J. McGibbney Ph.D, B.Sc Director, MCMA Associates Phone: +1(626)498-3090 Skype: lewis.john.mcgibbney Email: lewis.mcgibbney@gmail.com

lewismc · 2017-02-22T16:19:50Z

I adjusted my /etc/hosts file and all is good, thank you @vaibhavthapliyal

vaibhavthapliyal · 2017-02-22T17:21:11Z

Cheers!

renato2099 · 2017-02-22T18:35:13Z

thanks @vaibhavthapliyal !

Switched Accumulo Dependency to 1.7.1 and ported AccumuloStore Class to

ce945da

work with accumulo 1.7.1

vaibhavthapliyal added 2 commits February 18, 2017 13:26

Switched Accumulo Dependency to 1.7.1 and ported AccumuloStore Class to

1d38afc

work with 1.7.1

Merge branch 'master' of https://github.com/vaibhavthapliyal/gora

4bbf52e

# Conflicts: # gora-accumulo/src/main/java/org/apache/gora/accumulo/store/AccumuloStore.java

asfgit merged commit 4bbf52e into apache:master Feb 22, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Switched Accumulo Dependency to 1.7.1 and ported AccumuloStore Class to work with accumulo 1.7.1 #97

Switched Accumulo Dependency to 1.7.1 and ported AccumuloStore Class to work with accumulo 1.7.1 #97

vaibhavthapliyal commented Feb 17, 2017

lewismc commented Feb 17, 2017

vaibhavthapliyal commented Feb 18, 2017

lewismc commented Feb 21, 2017

vaibhavthapliyal commented Feb 22, 2017

lewismc commented Feb 22, 2017

lewismc commented Feb 22, 2017

vaibhavthapliyal commented Feb 22, 2017

lewismc commented Feb 22, 2017 •

edited

Loading

vaibhavthapliyal commented Feb 22, 2017 •

edited

Loading

renato2099 commented Feb 22, 2017

renato2099 commented Feb 22, 2017

lewismc commented Feb 22, 2017 via email

lewismc commented Feb 22, 2017

vaibhavthapliyal commented Feb 22, 2017

renato2099 commented Feb 22, 2017

Switched Accumulo Dependency to 1.7.1 and ported AccumuloStore Class to work with accumulo 1.7.1 #97

Switched Accumulo Dependency to 1.7.1 and ported AccumuloStore Class to work with accumulo 1.7.1 #97

Conversation

vaibhavthapliyal commented Feb 17, 2017

lewismc commented Feb 17, 2017

vaibhavthapliyal commented Feb 18, 2017

lewismc commented Feb 21, 2017

vaibhavthapliyal commented Feb 22, 2017

lewismc commented Feb 22, 2017

lewismc commented Feb 22, 2017

vaibhavthapliyal commented Feb 22, 2017

lewismc commented Feb 22, 2017 • edited Loading

vaibhavthapliyal commented Feb 22, 2017 • edited Loading

renato2099 commented Feb 22, 2017

renato2099 commented Feb 22, 2017

lewismc commented Feb 22, 2017 via email

lewismc commented Feb 22, 2017

vaibhavthapliyal commented Feb 22, 2017

renato2099 commented Feb 22, 2017

lewismc commented Feb 22, 2017 •

edited

Loading

vaibhavthapliyal commented Feb 22, 2017 •

edited

Loading