GlassFish + NetBeans networking related bug

[matthew-a-dunlap]

This bug is not fully understood at this point, but when some developers are connected to the Harvard network, GlassFish (all recent versions) are unable to talk to NetBeans (all recent versions). Furthermore, when turning off all network connections, communication between the two also fails even though the connection should only be across the local (internal to machine) network. Loading GlassFish manually outside of NetBeans has no issues.

The exact factors that cause this issue have not been determined, as similar set-ups do not seem to have the issue. One developer (Matthew) has tested this issue by logging into their home network, and while there GlassFish & NetBeans are able to communicate correctly. This has not been tested with other networks outside of harvard for reproducibility, but multiple networks in harvard have been tested to the same effect. Furthermore, the developer wiped their machine and started over from scratch to rule out network configuration issues with the machine. One other developer (Sarah) has reported similar issues but has not had time to investigate in depth, and a third non-Harvard developer also ran into issues during our Community Meeting.

The log messages differ depending on whether the machine is disconnected from all networks or is on the Harvard Network. When disconnected entirely, an error is thrown for the JMX connector, while when the machine is on the Harvard Network it hangs during the JMX connector step.

Error when not connected to a network:

Info: Grizzly Framework 2.3.15 started in: 1ms - bound to [/0.0.0.0:7676]1
Info: Registered com.sun.enterprise.glassfish.bootstrap.osgi.EmbeddedOSGiGlassFishImpl@5a6482a9 as OSGi service registration: org.apache.felix.framework.ServiceRegistrationImpl@545607f2.
Warning: Cannot start JMX connector JmxConnector config: { name = system, Protocol = rmi_jrmp, Address = 0.0.0.0, Port = 8686, AcceptAll = false, AuthRealmName = admin-realm, SecurityEnabled = false} due to exception java.net.MalformedURLException: Bad URL path: :0:0:414f:cf47:1f07:19c%13:8686/jndi/rmi://fe80:0:0:0:414f:cf47:1f07:19c%13:8686/jmxrmi
Severe: java.net.MalformedURLException: Bad URL path: :0:0:414f:cf47:1f07:19c%13:8686/jndi/rmi://fe80:0:0:0:414f:cf47:1f07:19c%13:8686/jmxrmi at javax.management.remote.JMXServiceURL.validate(JMXServiceURL.java:406) at javax.management.remote.JMXServiceURL.validate(JMXServiceURL.java:411) at javax.management.remote.JMXServiceURL.<init>(JMXServiceURL.java:226) at org.glassfish.admin.mbeanserver.RMIConnectorStarter.start(RMIConnectorStarter.java:306) at org.glassfish.admin.mbeanserver.JMXStartupService$JMXConnectorsStarterThread.startConnector(JMXStartupService.java:313) at org.glassfish.admin.mbeanserver.JMXStartupService$JMXConnectorsStarterThread.run(JMXStartupService.java:350)

Harvard Network last lines and final error popup (5-10 minutes later):

Info: Grizzly Framework 2.3.15 started in: 0ms - bound to [/0.0.0.0:7676]
Info: Registered com.sun.enterprise.glassfish.bootstrap.osgi.EmbeddedOSGiGlassFishImpl@5a6482a9 as OSGi service registration: org.apache.felix.framework.ServiceRegistrationImpl@545607f2.
Info: JMXStartupService has started JMXConnector on JMXService URL service:jmx:rmi://140.247.116.66:8686/jndi/rmi://140.247.116.66:8686/jmxrmi

This lack of integration greatly slows down development for affected developers, as developers are unable to take advantage of auto-deployment and the ability for NetBeans to hot-update UI components.

[matthew-a-dunlap]

This issue is a major problem and cannot be avoided by switching software versions (like the other GlassFish + NetBeans integration issue). Once it is better understood, it should be reported to the NetBeans development team.

[pameyer]

@matthew-a-dunlap can you post the JMX connector sections of domain.xml from this system?

[matthew-a-dunlap]

<admin-service system-jmx-connector-name="system" type="das-and-server">
<jmx-connector address="0.0.0.0" port="8686" name="system" auth-realm-name="admin-realm" security-enabled="false"></jmx-connector>
<property name="adminConsoleContextRoot" value="/admin"></property>
<property name="adminConsoleDownloadLocation" value="${com.sun.aas.installRoot}/lib/install/applications/admingui.war"></property>
<property name="ipsRoot" value="${com.sun.aas.installRoot}/.."></property>
<das-config></das-config>
</admin-service>

<admin-service system-jmx-connector-name="system">
<jmx-connector address="0.0.0.0" port="${JMX_SYSTEM_CONNECTOR_PORT}" name="system" auth-realm-name="admin-realm" security-enabled="false"></jmx-connector>
<property name="adminConsoleDownloadLocation" value="${com.sun.aas.installRoot}/lib/install/applications/admingui.war"></property>
<das-config></das-config>
</admin-service>

[matthew-a-dunlap]

Digging into MalformedURLException when not using a network connection, this is the chunk of code:
if (url.length() > 0) {
if (!url.startsWith("/") && !url.startsWith(";"))
throw new MalformedURLException("Bad URL path: " + url);
}

[matthew-a-dunlap]

NetBeans bug filed: https://netbeans.org/bugzilla/show_bug.cgi?id=271338

[wadechandler]

It seems the errors there may be related to IPv6 (notice the URLs with issues); they do not follow https://www.ietf.org/rfc/rfc2732.txt; that is the GF log right? i.e. it is bombing out or is that in the NB log too? It is hard to tell from the inlines; attachments may be better

[wadechandler]

Do the machines and networks where this is working have IPv6 disabled by chance? Seems plausible given the logs.

[matthew-a-dunlap]

@wadechandler Answering your second question first, I had tried disabling IPv6 for all my network devices when attempting to debug this issue, to no avail. I've turned it back on, this is the section of my ifconfig:
en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
ether f4:5c:89:af:da:df
inet6 fe80::24:87a1:f00a:cf66%en0 prefixlen 64 secured scopeid 0x6
inet 192.168.50.153 netmask 0xffffff00 broadcast 192.168.50.255
nd6 options=201<PERFORMNUD,DAD>
media: autoselect
status: active

[matthew-a-dunlap]

First question: its the GlassFish logs as displayed inside NetBeans. I'm attaching 3 files, each has two parts, first a fuller ifconfig and second the GlassFish server.log files. I can get you NetBeans logs tomorrow as well.

Thanks again @wadechandler !

Dataverse 4089 - Harvard Wired.txt
Dataverse 4089 - Home Network.txt
Dataverse 4089 - No Connection.txt

[pdurbin]

@matthew-a-dunlap I see you added the "dev guide" label to this issue and I'm wondering what, if anything, you'd like me to add to pull request #4502.

[pdurbin]

I just left a comment at #4088 (comment) about closing that issue and I think we should close this one as well unless it can be reproduced.

[pdurbin]

@matthew-a-dunlap heads up that Netbeans 9 is out: "Apache NetBeans 9.0, released on the 29th of July, 2018" https://netbeans.apache.org/download/index.html

[matthew-a-dunlap]

Ooh thanks @pdurbin. I was still having issues as of the beta from 6 months ago but maybe this is resolved.