netty error after plugin runs

[transamericamoon]

Hi we are using the latest build 1.4.0 with sonar 7.2.1.14109.

We are invoking the sonar plugin (3.3.0.603) from maven using: mvn sonar:sonar

The plugin is working fine and sending notifications to bitbucket, but we are getting the following error in the jenkins log after maven finishes.

Exception in thread "AsyncHttpClient-2-1" java.lang.NoClassDefFoundError: io/netty/util/concurrent/DefaultPromise$1 at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:428) at io.netty.util.concurrent.DefaultPromise.setSuccess(DefaultPromise.java:95) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:170) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ClassNotFoundException: io.netty.util.concurrent.DefaultPromise$1 at org.sonar.classloader.ParentFirstStrategy.loadClass(ParentFirstStrategy.java:39) at org.sonar.classloader.ClassRealm.loadClass(ClassRealm.java:87) at org.sonar.classloader.ClassRealm.loadClass(ClassRealm.java:76) ... 5 more

Any ideas what it could be?

[t-8ch]

@transamericamoon
Could you provide logs of the steps happening directly before?
(And if possible more of the stacktrace)

Could you also try an execution with the CLI scanner?

[transamericamoon]

This is what we see in jenkins console log:

[INFO] 6186/6186 components tracked
[INFO] ANALYSIS SUCCESSFUL
[INFO] Executing post-job Stash/Bitbucket notification
[INFO] SonarQube issues reported to Stash by user "stashUser" have been reset
[INFO] New SonarQube issues have been reported to Stash.
[INFO] SonarQube analysis overview has been reported to Stash.
[INFO] Task total time: 30.190 s
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
<List of projects Names removed>
[INFO] app ................................ SUCCESS [01:50 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:06 min
[INFO] Finished at: 2018-08-13T17:28:31-04:00
[INFO] Final Memory: 952M/2266M
[INFO] ------------------------------------------------------------------------
Exception in thread "AsyncHttpClient-2-1" java.lang.NoClassDefFoundError: io/netty/util/concurrent/DefaultPromise$1
	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:428)
	at io.netty.util.concurrent.DefaultPromise.setSuccess(DefaultPromise.java:95)
	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:170)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: io.netty.util.concurrent.DefaultPromise$1
	at org.sonar.classloader.ParentFirstStrategy.loadClass(ParentFirstStrategy.java:39)
	at org.sonar.classloader.ClassRealm.loadClass(ClassRealm.java:87)
	at org.sonar.classloader.ClassRealm.loadClass(ClassRealm.java:76)
	... 5 more
Notified Stash for commit with id cff7b12b43bbe201030f0d7ea34ac74f0afca7d1
Notified Stash for commit with id 1ca4852beb147795c635fdd52c9cc8777fba86b3
Finished: SUCCESS

I'm trying to get the jenkins server log from our devops. Is there some log i should look at in the sonar server?

[t-8ch]

Are the logs really this garbled?
Are you using plain Maven or the Jenkins Maven integration?

[transamericamoon]

sorry i formatted it wrong (fixed now). we are using a freestyle project with the "Invoke top-level maven target"

[t-8ch]

Can you try it without the built-in maven support?
Execute as script it's called afaik.

[transamericamoon]

When we run it directly on the jenkins build node from the commandline, i get the same error, when running locally the error does not occur.

[t-8ch]

Where does "Notified Stash for commit id" from?
Does the reporting to stash also happen when you execute it directly on the build node/locally?
Which HTTP library does the stash notification use?

[transamericamoon]

that is from the notify stash plugin. When running directly from the commandline we only invoke sonar:sonar.

Here is the tailed output from the commandline run:

[WARNING] Invalid character encountered in file /tmp/app/src/main/webapp/assets/css/fonts/692680/HelveticaNeueLigh.eot at line 1 for encoding UTF-8. Please fix file content or configure the encoding to be used using property 'sonar.sourceEncoding'.
[INFO] 'src/main/webapp/assets/md-bootstrap/bootstrap/_print.scss' generating issue exclusions
[INFO] 'src/main/webapp/assets/styles/ta-icons.css' generating issue exclusions
[INFO] 'src/main/webapp/assets/templates/companymatch_widget.html' generating issue exclusions
[INFO] 6348/6348 components tracked
[INFO] ANALYSIS SUCCESSFUL
[INFO] Executing post-job Stash/Bitbucket notification
[INFO] SonarQube issues reported to Stash by user "stashUser" have been reset
[INFO] New SonarQube issues have been reported to Stash.
[INFO] SonarQube analysis overview has been reported to Stash.
[INFO] Task total time: 1:45.586 s
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
<APPS_REMOVED>
[INFO] app ................................ SUCCESS [02:33 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:52 min
[INFO] Finished at: 2018-08-14T12:52:18-04:00
[INFO] Final Memory: 948M/4878M
[INFO] ------------------------------------------------------------------------
Exception in thread "AsyncHttpClient-2-1" java.lang.NoClassDefFoundError: io/netty/util/concurrent/DefaultPromise$1
	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:428)
	at io.netty.util.concurrent.DefaultPromise.setSuccess(DefaultPromise.java:95)
	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:170)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: io.netty.util.concurrent.DefaultPromise$1
	at org.sonar.classloader.ParentFirstStrategy.loadClass(ParentFirstStrategy.java:39)
	at org.sonar.classloader.ClassRealm.loadClass(ClassRealm.java:87)
	at org.sonar.classloader.ClassRealm.loadClass(ClassRealm.java:76)
	... 5 more

[transamericamoon]

Quick question, just so i understand this better, does sonarqube server send the request directly to bitbucket or does the client running sonar-runner send the http request?

[t-8ch]

It is executed locally.
(I will continue working on this tomorrow)

[transamericamoon]

Thanks. I do see some of our modules do use older versions of netty (netty-3.10.6.final.jar which do not comain "DefaultPromise"), but not sure how those would get on the plugin classpath, so this is unlikely.

My other thought is maybe the process using AsyncHttpClient is ending before the connection is closed. This seems more probable as sometimes we only get one exception, and sometimes i see more (10 or more).

AsyncHttpClient/async-http-client#1412

Just an FYI, this is a multi-module project.

[t-8ch]

You could:

• Try it with a minimal project
• Enable debugging

[transamericamoon]

After doing some testing, single projects seem to work fine, i think with more projects (around 70) in the reactor it's more likely to happen.

[transamericamoon]

Same error someone else is having using asyncHttpClient:
eed3si9n/gigahorse#20

[transamericamoon]

After looking at the source, it looks like you implement AutoClosable on StashClient, but you don't use try-with-resources when you instantiate it (ie. close is never called).

sonar-stash/src/main/java/org/sonar/plugins/stash/StashIssueReportingPostJob.java

Line 52 in 7ac1bd7

try (StashClient stashClient = new StashClient(stashURL,