Skip to content
This repository has been archived by the owner on Jan 7, 2021. It is now read-only.

netty error after plugin runs #191

Open
transamericamoon opened this issue Aug 10, 2018 · 22 comments
Open

netty error after plugin runs #191

transamericamoon opened this issue Aug 10, 2018 · 22 comments

Comments

@transamericamoon
Copy link

Hi we are using the latest build 1.4.0 with sonar 7.2.1.14109.

We are invoking the sonar plugin (3.3.0.603) from maven using: mvn sonar:sonar

The plugin is working fine and sending notifications to bitbucket, but we are getting the following error in the jenkins log after maven finishes.

Exception in thread "AsyncHttpClient-2-1" java.lang.NoClassDefFoundError: io/netty/util/concurrent/DefaultPromise$1 at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:428) at io.netty.util.concurrent.DefaultPromise.setSuccess(DefaultPromise.java:95) at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:170) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ClassNotFoundException: io.netty.util.concurrent.DefaultPromise$1 at org.sonar.classloader.ParentFirstStrategy.loadClass(ParentFirstStrategy.java:39) at org.sonar.classloader.ClassRealm.loadClass(ClassRealm.java:87) at org.sonar.classloader.ClassRealm.loadClass(ClassRealm.java:76) ... 5 more

Any ideas what it could be?
@t-8ch
Copy link
Contributor

t-8ch commented Aug 13, 2018

@transamericamoon
Could you provide logs of the steps happening directly before?
(And if possible more of the stacktrace)

Could you also try an execution with the CLI scanner?

@transamericamoon
Copy link
Author

transamericamoon commented Aug 14, 2018 •
edited

This is what we see in jenkins console log:

[INFO] 6186/6186 components tracked
[INFO] ANALYSIS SUCCESSFUL
[INFO] Executing post-job Stash/Bitbucket notification
[INFO] SonarQube issues reported to Stash by user "stashUser" have been reset
[INFO] New SonarQube issues have been reported to Stash.
[INFO] SonarQube analysis overview has been reported to Stash.
[INFO] Task total time: 30.190 s
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
<List of projects Names removed>
[INFO] app ................................ SUCCESS [01:50 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:06 min
[INFO] Finished at: 2018-08-13T17:28:31-04:00
[INFO] Final Memory: 952M/2266M
[INFO] ------------------------------------------------------------------------
Exception in thread "AsyncHttpClient-2-1" java.lang.NoClassDefFoundError: io/netty/util/concurrent/DefaultPromise$1
	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:428)
	at io.netty.util.concurrent.DefaultPromise.setSuccess(DefaultPromise.java:95)
	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:170)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: io.netty.util.concurrent.DefaultPromise$1
	at org.sonar.classloader.ParentFirstStrategy.loadClass(ParentFirstStrategy.java:39)
	at org.sonar.classloader.ClassRealm.loadClass(ClassRealm.java:87)
	at org.sonar.classloader.ClassRealm.loadClass(ClassRealm.java:76)
	... 5 more
Notified Stash for commit with id cff7b12b43bbe201030f0d7ea34ac74f0afca7d1
Notified Stash for commit with id 1ca4852beb147795c635fdd52c9cc8777fba86b3
Finished: SUCCESS

I'm trying to get the jenkins server log from our devops. Is there some log i should look at in the sonar server?

@t-8ch
Copy link
Contributor

t-8ch commented Aug 14, 2018

Are the logs really this garbled?
Are you using plain Maven or the Jenkins Maven integration?

@transamericamoon
Copy link
Author

sorry i formatted it wrong (fixed now). we are using a freestyle project with the "Invoke top-level maven target"

@t-8ch
Copy link
Contributor

t-8ch commented Aug 14, 2018

Can you try it without the built-in maven support?
Execute as script it's called afaik.

@transamericamoon
Copy link
Author

When we run it directly on the jenkins build node from the commandline, i get the same error, when running locally the error does not occur.

@t-8ch
Copy link
Contributor

t-8ch commented Aug 14, 2018

Where does "Notified Stash for commit id" from?
Does the reporting to stash also happen when you execute it directly on the build node/locally?
Which HTTP library does the stash notification use?

@transamericamoon
Copy link
Author

that is from the notify stash plugin. When running directly from the commandline we only invoke sonar:sonar.

Here is the tailed output from the commandline run:

[WARNING] Invalid character encountered in file /tmp/app/src/main/webapp/assets/css/fonts/692680/HelveticaNeueLigh.eot at line 1 for encoding UTF-8. Please fix file content or configure the encoding to be used using property 'sonar.sourceEncoding'.
[INFO] 'src/main/webapp/assets/md-bootstrap/bootstrap/_print.scss' generating issue exclusions
[INFO] 'src/main/webapp/assets/styles/ta-icons.css' generating issue exclusions
[INFO] 'src/main/webapp/assets/templates/companymatch_widget.html' generating issue exclusions
[INFO] 6348/6348 components tracked
[INFO] ANALYSIS SUCCESSFUL
[INFO] Executing post-job Stash/Bitbucket notification
[INFO] SonarQube issues reported to Stash by user "stashUser" have been reset
[INFO] New SonarQube issues have been reported to Stash.
[INFO] SonarQube analysis overview has been reported to Stash.
[INFO] Task total time: 1:45.586 s
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
<APPS_REMOVED>
[INFO] app ................................ SUCCESS [02:33 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 02:52 min
[INFO] Finished at: 2018-08-14T12:52:18-04:00
[INFO] Final Memory: 948M/4878M
[INFO] ------------------------------------------------------------------------
Exception in thread "AsyncHttpClient-2-1" java.lang.NoClassDefFoundError: io/netty/util/concurrent/DefaultPromise$1
	at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:428)
	at io.netty.util.concurrent.DefaultPromise.setSuccess(DefaultPromise.java:95)
	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:170)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: io.netty.util.concurrent.DefaultPromise$1
	at org.sonar.classloader.ParentFirstStrategy.loadClass(ParentFirstStrategy.java:39)
	at org.sonar.classloader.ClassRealm.loadClass(ClassRealm.java:87)
	at org.sonar.classloader.ClassRealm.loadClass(ClassRealm.java:76)
	... 5 more

@transamericamoon
Copy link
Author

Quick question, just so i understand this better, does sonarqube server send the request directly to bitbucket or does the client running sonar-runner send the http request?

@t-8ch
Copy link
Contributor

t-8ch commented Aug 14, 2018

It is executed locally.
(I will continue working on this tomorrow)

@transamericamoon
Copy link
Author

transamericamoon commented Aug 14, 2018 •
edited

Thanks. I do see some of our modules do use older versions of netty (netty-3.10.6.final.jar which do not comain "DefaultPromise"), but not sure how those would get on the plugin classpath, so this is unlikely.

My other thought is maybe the process using AsyncHttpClient is ending before the connection is closed. This seems more probable as sometimes we only get one exception, and sometimes i see more (10 or more).

AsyncHttpClient/async-http-client#1412

Just an FYI, this is a multi-module project.

@t-8ch
Copy link
Contributor

t-8ch commented Aug 14, 2018

You could:

  • Try it with a minimal project
  • Enable debugging

@transamericamoon
Copy link
Author

After doing some testing, single projects seem to work fine, i think with more projects (around 70) in the reactor it's more likely to happen.

@transamericamoon
Copy link
Author

transamericamoon commented Aug 14, 2018 •
edited

Same error someone else is having using asyncHttpClient:
eed3si9n/gigahorse#20

@transamericamoon
Copy link
Author

After looking at the source, it looks like you implement AutoClosable on StashClient, but you don't use try-with-resources when you instantiate it (ie. close is never called).

sonar-stash/src/main/java/org/sonar/plugins/stash/StashIssueReportingPostJob.java

Line 52 in 7ac1bd7

try (StashClient stashClient = new StashClient(stashURL,

@t-8ch
Copy link
Contributor

t-8ch commented Aug 14, 2018

Hm, as far as I see this is a try-with-resources block.

@transamericamoon
Copy link
Author

My bad. I'm at a loss as to what is causing it.

@t-8ch
Copy link
Contributor

t-8ch commented Aug 14, 2018

If you experience multiple exceptions, do they all appear in the same Thread?
If you could build a reproduction case it would help greatly.

@t-8ch
Copy link
Contributor

t-8ch commented Aug 14, 2018

When you run it outside of Jenkins, are you using the same SQ parameters/server and the same version of Java?

@transamericamoon
Copy link
Author

I believe this is definitely the issue:

AsyncHttpClient/async-http-client#1412

A PR was opened but is not accepted/un-mergable.
AsyncHttpClient/async-http-client#1435

@t-8ch
Copy link
Contributor

t-8ch commented Aug 15, 2018

@transamericamoon I am not sure this is the issue.
The linked issue was about in-flight requests at the point in time the client is closed.
All requests in sonar-stash are done synchronously in StashClient.performRequest().
As such there should be no way of this happening.

@t-8ch
Copy link
Contributor

t-8ch commented Aug 15, 2018

You can also try to update async-httpclient to the latest version.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@t-8ch @transamericamoon