java.lang.IllegalStateException raised spuriously. #335

bigdelys · 2016-09-09T00:43:15Z

We are seeing java.lang.IllegalStateException every once in a while, sometimes it stays for a few seconds, and after multiple refreshes:
java.lang.IllegalStateException: thread 'back-39' in group 'back' interrupted at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:182) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136) at com.s3auth.hosts.DefaultBucket.bucket(DefaultBucket.java:112) at com.s3auth.hosts.DefaultHost.fetch_aroundBody2(DefaultHost.java:178) at com.s3auth.hosts.DefaultHost$AjcClosure3.run(DefaultHost.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapMethod(MethodLogger.java:164) at com.s3auth.hosts.DefaultHost.fetch_aroundBody4(DefaultHost.java:168) at com.s3auth.hosts.DefaultHost$AjcClosure5.run(DefaultHost.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:134) at com.s3auth.hosts.DefaultHost.fetch_aroundBody6(DefaultHost.java:168) at com.s3auth.hosts.DefaultHost$AjcClosure7.run(DefaultHost.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodInterrupter.wrap(MethodInterrupter.java:115) at com.s3auth.hosts.DefaultHost.fetch(DefaultHost.java:168) at com.s3auth.hosts.SmartHost.fetch_aroundBody6(SmartHost.java:119) at com.s3auth.hosts.SmartHost$AjcClosure7.run(SmartHost.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapMethod(MethodLogger.java:164) at com.s3auth.hosts.SmartHost.fetch_aroundBody8(SmartHost.java:104) at com.s3auth.hosts.SmartHost$AjcClosure9.run(SmartHost.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:134) at com.s3auth.hosts.SmartHost.fetch_aroundBody10(SmartHost.java:104) at com.s3auth.hosts.SmartHost$AjcClosure11.run(SmartHost.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodInterrupter.wrap(MethodInterrupter.java:115) at com.s3auth.hosts.SmartHost.fetch(SmartHost.java:104) at com.s3auth.hosts.FastHost.fetch_aroundBody0(FastHost.java:90) at com.s3auth.hosts.FastHost$AjcClosure1.run(FastHost.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodInterrupter.wrap(MethodInterrupter.java:115) at com.s3auth.hosts.FastHost.fetch(FastHost.java:90) at com.s3auth.hosts.RejectingHost.fetch(RejectingHost.java:108) at com.s3auth.relay.SecuredHost.secured(SecuredHost.java:199) at com.s3auth.relay.SecuredHost.fetch_aroundBody0(SecuredHost.java:103) at com.s3auth.relay.SecuredHost$AjcClosure1.run(SecuredHost.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212) at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1) at com.jcabi.aspects.aj.MethodLogger.wrapMethod(MethodLogger.java:164) at com.s3auth.relay.SecuredHost.fetch_aroundBody2(SecuredHost.java:102) at com.s3auth.relay.SecuredHost$AjcClosure3.run(SecuredHost.java:1) at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149) at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:134) at com.s3auth.relay.SecuredHost.fetch(SecuredHost.java:102) at com.s3auth.relay.HttpThread.resource(HttpThread.java:247) at com.s3auth.relay.HttpThread.dispatch(HttpThread.java:169) at com.s3auth.relay.HttpFacade$HttpThreadRunnable.run(HttpFacade.java:264) at com.jcabi.log.VerboseRunnable.run(VerboseRunnable.java:198) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at com.jcabi.log.VerboseThreads$Wrap.run(VerboseThreads.java:220) at java.lang.Thread.run(Thread.java:701)

The text was updated successfully, but these errors were encountered:

davvd · 2016-09-12T11:18:41Z

@yegor256 please do something about it, see par.21

yegor256 · 2016-09-16T00:43:57Z

@bigdelys I'll fix it now

yegor256 · 2016-09-16T00:49:08Z

@rultor deploy

rultor · 2016-09-16T00:49:37Z

@rultor deploy

@yegor256 OK, I'll try to deploy now. You can check the progress here

rultor · 2016-09-16T00:56:41Z

@rultor deploy

@bigdelys @yegor256 Oops, I failed. You can see the full log here (spent 7min)

[INFO] s3auth ............................................. SUCCESS [ 40.072 s]
[INFO] s3auth-hosts ....................................... FAILURE [ 51.066 s]
[INFO] s3auth-relay ....................................... SKIPPED
[INFO] s3auth-rest ........................................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:37 min
[INFO] Finished at: 2016-09-16T00:55:22+00:00
[INFO] Final Memory: 48M/468M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project s3auth-hosts: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/r/repo/s3auth-hosts/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.17:test (default-test) on project s3auth-hosts: There are test failures.

Please refer to /home/r/repo/s3auth-hosts/target/surefire-reports for the individual test results.
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
    at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
    at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
    at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
    at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoFailureException: There are test failures.

Please refer to /home/r/repo/s3auth-hosts/target/surefire-reports for the individual test results.
    at org.apache.maven.plugin.surefire.SurefireHelper.reportExecution(SurefireHelper.java:82)
    at org.apache.maven.plugin.surefire.SurefirePlugin.handleSummary(SurefirePlugin.java:195)
    at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:861)
    at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:729)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
    ... 20 more
[ERROR] 
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :s3auth-hosts
container 0546af85c5202758e1b62bee56938e6caad0e2baebd0937f113e3937f6edb959 is dead
Fri Sep 16 00:56:37 UTC 2016

yegor256 · 2016-09-16T01:01:05Z

@rultor try to deploy

rultor · 2016-09-16T01:02:06Z

@rultor try to deploy

@yegor256 OK, I'll try to deploy now. You can check the progress here

rultor · 2016-09-16T01:22:59Z

@rultor try to deploy

@yegor256 Done! FYI, the full log is here (took me 21min)

yegor256 · 2016-09-16T01:25:35Z

@bigdelys should be fine now, please check

bigdelys · 2016-09-16T04:31:18Z

Thank you very much. It seems to be fixed.

Nima

On Thu, Sep 15, 2016 at 6:25 PM, Yegor Bugayenko notifications@github.com
wrote:

@bigdelys https://github.com/bigdelys should be fine now, please check

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#335 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEAw6OV579rvw0smtATV4FHC0Dz5Lr1_ks5qqfAPgaJpZM4J4l7t
.

zaharidichev · 2017-03-02T08:22:37Z

Well.... It does not seem that resolved after all.

java.lang.IllegalStateException: thread 'back-58' in group 'back' interrupted
    at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:182)
    at com.jcabi.aspects.aj.MethodLogger.ajc$inlineAccessMethod$com_jcabi_aspects_aj_MethodLogger$com_jcabi_aspects_aj_MethodLogger$wrap(MethodLogger.java:1)
    at com.jcabi.aspects.aj.MethodLogger.wrapClass(MethodLogger.java:136)
    at com.s3auth.hosts.DefaultBucket.name(DefaultBucket.java:94)
    at com.s3auth.hosts.DefaultHost.fetch_aroundBody2(DefaultHost.java:228)
    at com.s3auth.hosts.DefaultHost$AjcClosure3.run(DefaultHost.java:1)
    at org.aspectj.runtime.reflect.JoinPointImpl.proceed(JoinPointImpl.java:149)
    at com.jcabi.aspects.aj.MethodLogger.wrap(MethodLogger.java:212)

Can we request some urgency here as we relied on that particular service and that brings us quite a bit of problems at this very moment.

lukesaunders · 2017-03-08T14:15:21Z

I have this same issue. Using the hosted s3auth.com service, sometimes requests for files work fine, and other times I get a 500. The stacktrace given is https://gist.github.com/lukesaunders/3b28da26a8e0036c01d7e9c0abb4fc63

Strangely, of the two domains I've set up, one works fine and the other has the problem. The only difference seems to be concurrency - one domain loads a few files concurrently (around 4), and the other can load many (around 60). Is there a concurrency limit?

lukesaunders · 2017-03-08T14:16:17Z

@yegor256 could you reopen this issue or shall I create a new one?

yegor256 · 2017-05-11T05:39:21Z

We have #348 to continue with this problem

yegor256 pushed a commit that referenced this issue Sep 16, 2016

#335 Timeable removed

5cc7372

yegor256 pushed a commit that referenced this issue Sep 16, 2016

#335 test fixed

3b6ae12

yegor256 closed this as completed Nov 22, 2016

yegor256 added the invalid label Nov 22, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

java.lang.IllegalStateException raised spuriously. #335

java.lang.IllegalStateException raised spuriously. #335

bigdelys commented Sep 9, 2016

davvd commented Sep 12, 2016

yegor256 commented Sep 16, 2016

yegor256 commented Sep 16, 2016

rultor commented Sep 16, 2016

rultor commented Sep 16, 2016

yegor256 commented Sep 16, 2016

rultor commented Sep 16, 2016

rultor commented Sep 16, 2016

yegor256 commented Sep 16, 2016

bigdelys commented Sep 16, 2016

zaharidichev commented Mar 2, 2017

lukesaunders commented Mar 8, 2017

lukesaunders commented Mar 8, 2017

yegor256 commented May 11, 2017

java.lang.IllegalStateException raised spuriously. #335

java.lang.IllegalStateException raised spuriously. #335

Comments

bigdelys commented Sep 9, 2016

davvd commented Sep 12, 2016

yegor256 commented Sep 16, 2016

yegor256 commented Sep 16, 2016

rultor commented Sep 16, 2016

rultor commented Sep 16, 2016

yegor256 commented Sep 16, 2016

rultor commented Sep 16, 2016

rultor commented Sep 16, 2016

yegor256 commented Sep 16, 2016

bigdelys commented Sep 16, 2016

zaharidichev commented Mar 2, 2017

lukesaunders commented Mar 8, 2017

lukesaunders commented Mar 8, 2017

yegor256 commented May 11, 2017