Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fixed bug that caused travisCI build timeout #959

Merged
merged 1 commit into from
Oct 25, 2017

Conversation

keith-turner
Copy link
Contributor

In the travisCI docs I found that its possible to use docker to run travis builds locally (very cool!). I did this and got the FluoAdminIT test to hang. I found the problem was that a weird accumulo classpath context was being set. This caused tablets to fail to load sometimes (it was timing issue, seemed more likely in docker env). When the tablet failed to load the test would hang forever trying to delete the table in cleanup.

Below is information I collected form running travis build env locally in docker.

Jstack of test process

"main" #1 prio=5 os_prio=0 tid=0x00007fc52c009800 nid=0x1799 runnable [0x00007fc5348da000]
   java.lang.Thread.State: RUNNABLE
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
	- locked <0x00000000d7ebde18> (a java.io.BufferedInputStream)
	at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:127)
	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
	at org.apache.thrift.transport.TFramedTransport.readFrame(TFramedTransport.java:129)
	at org.apache.thrift.transport.TFramedTransport.read(TFramedTransport.java:101)
	at org.apache.thrift.transport.TTransport.readAll(TTransport.java:84)
	at org.apache.accumulo.core.client.impl.ThriftTransportPool$CachedTTransport.readAll(ThriftTransportPool.java:273)
	at org.apache.thrift.protocol.TCompactProtocol.readByte(TCompactProtocol.java:601)
	at org.apache.thrift.protocol.TCompactProtocol.readMessageBegin(TCompactProtocol.java:470)
	at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
	at org.apache.accumulo.core.master.thrift.FateService$Client.recv_waitForFateOperation(FateService.java:174)
	at org.apache.accumulo.core.master.thrift.FateService$Client.waitForFateOperation(FateService.java:159)
	at org.apache.accumulo.core.client.impl.TableOperationsImpl.waitForFateOperation(TableOperationsImpl.java:266)
	at org.apache.accumulo.core.client.impl.TableOperationsImpl.doFateOperation(TableOperationsImpl.java:308)
	at org.apache.accumulo.core.client.impl.TableOperationsImpl.doFateOperation(TableOperationsImpl.java:294)
	at org.apache.accumulo.core.client.impl.TableOperationsImpl.doTableFateOperation(TableOperationsImpl.java:1592)
	at org.apache.accumulo.core.client.impl.TableOperationsImpl.delete(TableOperationsImpl.java:679)
	at org.apache.fluo.integration.ITBaseImpl.tearDownFluo(ITBaseImpl.java:110)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:367)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:274)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:161)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)
2017-10-25 19:14:53,166 [constraints.ConstraintChecker] ERROR: Failed to load constraints 2 org.apache.commons.vfs2.FileSystemException: Could not find file with URI 
org.apache.commons.vfs2.FileSystemException: Could not find file with URI "impl-test0" because it is a relative path, and no base URI was provided.
        at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:719)
        at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:649)
        at org.apache.commons.vfs2.impl.DefaultFileSystemManager.resolveFile(DefaultFileSystemManager.java:605)
        at org.apache.accumulo.start.classloader.vfs.AccumuloVFSClassLoader.resolve(AccumuloVFSClassLoader.java:140)
ader.<init>(AccumuloReloadingVFSClassLoader.java:143)
        at org.apache.accumulo.start.classloader.vfs.AccumuloReloadingVFSClassLoader.<init>(AccumuloReloadingVFSClassLoader.java:161)
        at org.apache.accumulo.start.classloader.vfs.ContextManager$Context.getClassLoader(ContextManager.java:46)
        at org.apache.accumulo.start.classloader.vfs.ContextManager.getClassLoader(ContextManager.java:174)
        at org.apache.accumulo.tserver.constraints.ConstraintChecker.<init>(ConstraintChecker.java:55)
        at org.apache.accumulo.tserver.Tablet$1.reloadConstraints(Tablet.java:1330)
        at org.apache.accumulo.tserver.Tablet$1.propertiesChanged(Tablet.java:1335)
        at org.apache.accumulo.tserver.Tablet.<init>(Tablet.java:1369)
        at org.apache.accumulo.tserver.Tablet.<init>(Tablet.java:1239)
        at org.apache.accumulo.tserver.Tablet.<init>(Tablet.java:1095)
        at org.apache.accumulo.tserver.Tablet.<init>(Tablet.java:1071)
        at org.apache.accumulo.tserver.TabletServer$AssignmentHandler.run(TabletServer.java:3015)
        at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
        at org.apache.accumulo.tserver.ActiveAssignmentRunnable.run(ActiveAssignmentRunnable.java:61)
        at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.accumulo.trace.instrument.TraceRunnable.run(TraceRunnable.java:47)
        at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
        at java.lang.Thread.run(Thread.java:748)
[zk: localhost:38381(CONNECTED) 12] get /accumulo/ea698c66-5ff4-4852-a574-eb00cc3f2bbb/config/general.vfs.context.classpath.fluo-impl-test0
impl-test0
cZxid = 0x1dd
ctime = Wed Oct 25 19:14:53 UTC 2017
mZxid = 0x1dd
mtime = Wed Oct 25 19:14:53 UTC 2017
pZxid = 0x1dd
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 10
numChildren = 0

@kennethmcfarland
Copy link
Contributor

kennethmcfarland commented Oct 25, 2017 via email

This was referenced Oct 25, 2017
@keith-turner keith-turner merged commit d123d42 into apache:master Oct 25, 2017
@keith-turner keith-turner deleted the fluo-adminIT-timeout branch August 8, 2019 21:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants