Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failure acquiring local host IP address results in 400 errors but should be 500s #2243

Closed
lmsurpre opened this issue Apr 13, 2021 · 4 comments
Closed

Failure acquiring local host IP address results in 400 errors but should be 500s #2243

lmsurpre opened this issue Apr 13, 2021 · 4 comments
Assignees
@prb112
Labels
bug Something isn't working
Milestone
Sprint 2021-05

Comments

@lmsurpre
Copy link
Member

lmsurpre commented Apr 13, 2021 •
edited
Loading

Describe the bug
Saw this one on a failed CI run:

[4/13/21 3:39:44:607 UTC] 0000001e com.ibm.fhir.core.util.handler.IPHandler                     E Failure acquiring local host IP address
java.net.UnknownHostException: fv-az8-815: fv-az8-815: Name or service not known
	at java.net.InetAddress.getLocalHost(InetAddress.java:1506)
	at com.ibm.fhir.core.util.handler.IPHandler.determineIP(IPHandler.java:45)
	at com.ibm.fhir.core.util.handler.IPHandler.<clinit>(IPHandler.java:30)
	at com.ibm.fhir.exception.FHIRException.getUniqueId(FHIRException.java:63)
	at com.ibm.fhir.exception.FHIRException.toString(FHIRException.java:85)
	at java.lang.String.valueOf(String.java:2994)
	at java.lang.StringBuilder.append(StringBuilder.java:131)
	at java.lang.Throwable.printEnclosedStackTrace(Throwable.java:696)
	at java.lang.Throwable.printEnclosedStackTrace(Throwable.java:710)
	at java.lang.Throwable.printStackTrace(Throwable.java:668)
	at java.lang.Throwable.printStackTrace(Throwable.java:722)
	at com.ibm.websphere.ras.DataFormatHelper.printStackTrace(DataFormatHelper.java:237)
	at com.ibm.websphere.ras.DataFormatHelper.throwableToString(DataFormatHelper.java:230)
	at com.ibm.ws.logging.internal.WsLogRecord.setThrown(WsLogRecord.java:411)
	at com.ibm.ws.logging.internal.WsLogger.createWsLogRecord(WsLogger.java:228)
	at com.ibm.ws.logging.internal.WsLogger.createLogRecord(WsLogger.java:187)
	at com.ibm.ws.logging.internal.WsLogger.log(WsLogger.java:298)
	at com.ibm.fhir.server.operation.FHIROperationRegistry.<init>(FHIROperationRegistry.java:79)
	at com.ibm.fhir.server.operation.FHIROperationRegistry.<clinit>(FHIROperationRegistry.java:34)
	at com.ibm.fhir.server.listener.FHIRServletContextListener.contextInitialized(FHIRServletContextListener.java:107)
	at com.ibm.ws.webcontainer.webapp.WebApp.notifyServletContextCreated(WebApp.java:2437)
	at com.ibm.ws.webcontainer31.osgi.webapp.WebApp31.notifyServletContextCreated(WebApp31.java:514)
	at com.ibm.ws.webcontainer.webapp.WebApp.initialize(WebApp.java:1041)
	at com.ibm.ws.webcontainer.webapp.WebApp.initialize(WebApp.java:6683)
	at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost.startWebApp(DynamicVirtualHost.java:470)
	at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost.startWebApplication(DynamicVirtualHost.java:465)
	at com.ibm.ws.webcontainer.osgi.WebContainer.startWebApplication(WebContainer.java:1179)
	at com.ibm.ws.webcontainer.osgi.WebContainer.startModule(WebContainer.java:968)
	at com.ibm.ws.app.manager.module.internal.ModuleHandlerBase.deployModule(ModuleHandlerBase.java:99)
	at com.ibm.ws.app.manager.module.internal.DeployedModuleInfoImpl.installModule(DeployedModuleInfoImpl.java:49)
	at com.ibm.ws.app.manager.module.internal.SimpleDeployedAppInfoBase.deployModules(SimpleDeployedAppInfoBase.java:597)
	at com.ibm.ws.app.manager.module.internal.SimpleDeployedAppInfoBase.installApp(SimpleDeployedAppInfoBase.java:511)
	at com.ibm.ws.app.manager.module.internal.DeployedAppInfoBase.deployApp(DeployedAppInfoBase.java:349)
	at com.ibm.ws.app.manager.war.internal.WARApplicationHandlerImpl.install(WARApplicationHandlerImpl.java:65)
	at com.ibm.ws.app.manager.internal.statemachine.StartAction.execute(StartAction.java:149)
	at com.ibm.ws.app.manager.internal.statemachine.ApplicationStateMachineImpl.enterState(ApplicationStateMachineImpl.java:1352)
	at com.ibm.ws.app.manager.internal.statemachine.ApplicationStateMachineImpl.run(ApplicationStateMachineImpl.java:897)
	at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:238)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.UnknownHostException: fv-az8-815: Name or service not known
	at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
	at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:929)
	at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1324)
	at java.net.InetAddress.getLocalHost(InetAddress.java:1501)
	... 40 more

Can't say I totally understand it, but we didn't handle it too well.

Specifically it leads to an initialization error, but because we use the hostname while constructing the id of the exception, this itself produces an exception.

SRVE0283E: Exception caught while initializing context: java.lang.RuntimeException: Encountered an exception while initializing the servlet context.
	at com.ibm.fhir.server.listener.FHIRServletContextListener.contextInitialized(FHIRServletContextListener.java:220)
	at com.ibm.ws.webcontainer.webapp.WebApp.notifyServletContextCreated(WebApp.java:2437)
	at com.ibm.ws.webcontainer31.osgi.webapp.WebApp31.notifyServletContextCreated(WebApp31.java:514)
	at com.ibm.ws.webcontainer.webapp.WebApp.initialize(WebApp.java:1041)
	at com.ibm.ws.webcontainer.webapp.WebApp.initialize(WebApp.java:6683)
	at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost.startWebApp(DynamicVirtualHost.java:470)
	at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost.startWebApplication(DynamicVirtualHost.java:465)
	at com.ibm.ws.webcontainer.osgi.WebContainer.startWebApplication(WebContainer.java:1179)
	at com.ibm.ws.webcontainer.osgi.WebContainer.startModule(WebContainer.java:968)
	at com.ibm.ws.app.manager.module.internal.ModuleHandlerBase.deployModule(ModuleHandlerBase.java:99)
	at com.ibm.ws.app.manager.module.internal.DeployedModuleInfoImpl.installModule(DeployedModuleInfoImpl.java:49)
	at com.ibm.ws.app.manager.module.internal.SimpleDeployedAppInfoBase.deployModules(SimpleDeployedAppInfoBase.java:597)
	at com.ibm.ws.app.manager.module.internal.SimpleDeployedAppInfoBase.installApp(SimpleDeployedAppInfoBase.java:511)
	at com.ibm.ws.app.manager.module.internal.DeployedAppInfoBase.deployApp(DeployedAppInfoBase.java:349)
	at com.ibm.ws.app.manager.war.internal.WARApplicationHandlerImpl.install(WARApplicationHandlerImpl.java:65)
	at com.ibm.ws.app.manager.internal.statemachine.StartAction.execute(StartAction.java:149)
	at com.ibm.ws.app.manager.internal.statemachine.ApplicationStateMachineImpl.enterState(ApplicationStateMachineImpl.java:1352)
	at com.ibm.ws.app.manager.internal.statemachine.ApplicationStateMachineImpl.run(ApplicationStateMachineImpl.java:897)
	at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:238)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ExceptionInInitializerError
	at com.ibm.fhir.server.listener.FHIRServletContextListener.contextInitialized(FHIRServletContextListener.java:107)
	... 21 more
Caused by: java.lang.NullPointerException
	at com.ibm.fhir.exception.FHIRException.getUniqueId(FHIRException.java:65)
	at com.ibm.fhir.exception.FHIRException.toString(FHIRException.java:85)
	at java.lang.String.valueOf(String.java:2994)
	at java.lang.StringBuilder.append(StringBuilder.java:131)
	at java.lang.Throwable.printEnclosedStackTrace(Throwable.java:696)
	at java.lang.Throwable.printEnclosedStackTrace(Throwable.java:710)
	at java.lang.Throwable.printStackTrace(Throwable.java:668)
	at java.lang.Throwable.printStackTrace(Throwable.java:722)
	at com.ibm.websphere.ras.DataFormatHelper.printStackTrace(DataFormatHelper.java:237)
	at com.ibm.websphere.ras.DataFormatHelper.throwableToString(DataFormatHelper.java:230)
	at com.ibm.ws.logging.internal.WsLogRecord.setThrown(WsLogRecord.java:411)
	at com.ibm.ws.logging.internal.WsLogger.createWsLogRecord(WsLogger.java:228)
	at com.ibm.ws.logging.internal.WsLogger.createLogRecord(WsLogger.java:187)
	at com.ibm.ws.logging.internal.WsLogger.log(WsLogger.java:298)
	at com.ibm.fhir.server.operation.FHIROperationRegistry.<init>(FHIROperationRegistry.java:79)
	at com.ibm.fhir.server.operation.FHIROperationRegistry.<clinit>(FHIROperationRegistry.java:34)

Then, when a request comes in, we hit something similar while constructing the exception to tell them that the server didn't initialize properly. After a couple other stack traces, we get this one:

[4/13/21 3:40:46:374 UTC] 0000002b com.ibm.ws.webcontainer.util.ApplicationErrorUtils           E SRVE0777E: Exception thrown by application class 'com.ibm.fhir.exception.FHIRException.getUniqueId:65'
java.lang.NullPointerException
	at com.ibm.fhir.exception.FHIRException.getUniqueId(FHIRException.java:65)
	at com.ibm.fhir.exception.FHIRException.toString(FHIRException.java:85)
	at java.lang.String.valueOf(String.java:2994)
	at java.io.PrintWriter.println(PrintWriter.java:754)
	at java.lang.Throwable$WrappedPrintWriter.println(Throwable.java:765)
	at java.lang.Throwable.printStackTrace(Throwable.java:656)
	at java.lang.Throwable.printStackTrace(Throwable.java:722)
	at com.ibm.websphere.ras.DataFormatHelper.printStackTrace(DataFormatHelper.java:237)
	at com.ibm.websphere.ras.DataFormatHelper.throwableToString(DataFormatHelper.java:230)
	at com.ibm.ws.logging.internal.WsLogRecord.setThrown(WsLogRecord.java:411)
	at com.ibm.ws.logging.internal.WsLogger.createWsLogRecord(WsLogger.java:228)
	at com.ibm.ws.logging.internal.WsLogger.createLogRecord(WsLogger.java:187)
	at com.ibm.ws.logging.internal.WsLogger.log(WsLogger.java:298)
	at com.ibm.fhir.server.resources.FHIRResource.exceptionResponse(FHIRResource.java:266)
	at com.ibm.fhir.server.resources.Operation.invoke(Operation.java:93)
	at com.ibm.fhir.server.resources.Operation$Proxy$_$$_WeldClientProxy.invoke(Unknown Source)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.ibm.ws.jaxrs20.cdi.component.JaxRsFactoryImplicitBeanCDICustomizer.serviceInvoke(JaxRsFactoryImplicitBeanCDICustomizer.java:342)
	at com.ibm.ws.jaxrs20.server.LibertyJaxRsServerFactoryBean.performInvocation(LibertyJaxRsServerFactoryBean.java:641)
	at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.performInvocation(LibertyJaxRsInvoker.java:160)
	at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:101)
	at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.invoke(LibertyJaxRsInvoker.java:273)
	at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:205)
	at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.invoke(LibertyJaxRsInvoker.java:444)
	at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:111)
	at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
	at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
	at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
	at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:123)
	at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:275)
	at com.ibm.ws.jaxrs20.endpoint.AbstractJaxRsWebEndpoint.invoke(AbstractJaxRsWebEndpoint.java:137)
	at com.ibm.websphere.jaxrs.server.IBMRestServlet.handleRequest(IBMRestServlet.java:146)
	at com.ibm.websphere.jaxrs.server.IBMRestServlet.doHead(IBMRestServlet.java:141)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:703)
	at com.ibm.websphere.jaxrs.server.IBMRestServlet.service(IBMRestServlet.java:96)
	at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1257)
	at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:745)
	at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:442)
	at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:182)
	at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:93)
	at com.ibm.fhir.server.filter.rest.FHIRRestServletFilter.doFilter(FHIRRestServletFilter.java:142)
	at javax.servlet.http.HttpFilter.doFilter(HttpFilter.java:127)
	at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:201)
	at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:90)
	at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:1001)
	at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1139)
	at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:5057)
	at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost$2.handleRequest(DynamicVirtualHost.java:314)
	at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:1006)
	at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost$2.run(DynamicVirtualHost.java:279)
	at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink$TaskWrapper.run(HttpDispatcherLink.java:1146)
	at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.wrapHandlerAndExecute(HttpDispatcherLink.java:427)
	at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.ready(HttpDispatcherLink.java:386)
	at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:566)
	at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleNewRequest(HttpInboundLink.java:500)
	at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.processRequest(HttpInboundLink.java:360)
	at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.ready(HttpInboundLink.java:327)
	at com.ibm.ws.channel.ssl.internal.SSLConnectionLink.determineNextChannel(SSLConnectionLink.java:1100)
	at com.ibm.ws.channel.ssl.internal.SSLConnectionLink.readyInboundPostHandshake(SSLConnectionLink.java:757)
	at com.ibm.ws.channel.ssl.internal.SSLConnectionLink$MyHandshakeCompletedCallback.complete(SSLConnectionLink.java:427)
	at com.ibm.ws.channel.ssl.internal.SSLUtils.handleHandshake(SSLUtils.java:954)
	at com.ibm.ws.channel.ssl.internal.SSLHandshakeIOCallback.complete(SSLHandshakeIOCallback.java:85)
	at com.ibm.ws.tcpchannel.internal.WorkQueueManager.requestComplete(WorkQueueManager.java:504)
	at com.ibm.ws.tcpchannel.internal.WorkQueueManager.attemptIO(WorkQueueManager.java:574)
	at com.ibm.ws.tcpchannel.internal.WorkQueueManager.workerRun(WorkQueueManager.java:958)
	at com.ibm.ws.tcpchannel.internal.WorkQueueManager$Worker.run(WorkQueueManager.java:1047)
	at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:238)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Eventually we get down to this one:

[4/13/21 3:40:46:375 UTC] 0000002b com.ibm.fhir.server.filter.rest.FHIRRestServletFilter        I Error while setting request context or processing request
java.lang.NullPointerException
	at com.ibm.fhir.exception.FHIRException.getUniqueId(FHIRException.java:65)
	at com.ibm.fhir.exception.FHIRException.toString(FHIRException.java:85)
	at java.lang.String.valueOf(String.java:2994)
	at java.io.PrintWriter.println(PrintWriter.java:754)
	at java.lang.Throwable$WrappedPrintWriter.println(Throwable.java:765)
	at java.lang.Throwable.printStackTrace(Throwable.java:656)
	at java.lang.Throwable.printStackTrace(Throwable.java:722)
	at com.ibm.websphere.ras.DataFormatHelper.printStackTrace(DataFormatHelper.java:237)
	at com.ibm.websphere.ras.DataFormatHelper.throwableToString(DataFormatHelper.java:230)
	at com.ibm.ws.logging.internal.WsLogRecord.setThrown(WsLogRecord.java:411)
	at com.ibm.ws.logging.internal.WsLogger.createWsLogRecord(WsLogger.java:228)
	at com.ibm.ws.logging.internal.WsLogger.createLogRecord(WsLogger.java:187)
	at com.ibm.ws.logging.internal.WsLogger.log(WsLogger.java:298)
	at com.ibm.fhir.server.resources.FHIRResource.exceptionResponse(FHIRResource.java:266)
	at com.ibm.fhir.server.resources.Operation.invoke(Operation.java:93)
	at com.ibm.fhir.server.resources.Operation$Proxy$_$$_WeldClientProxy.invoke(Unknown Source)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.ibm.ws.jaxrs20.cdi.component.JaxRsFactoryImplicitBeanCDICustomizer.serviceInvoke(JaxRsFactoryImplicitBeanCDICustomizer.java:342)
	at com.ibm.ws.jaxrs20.server.LibertyJaxRsServerFactoryBean.performInvocation(LibertyJaxRsServerFactoryBean.java:641)
	at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.performInvocation(LibertyJaxRsInvoker.java:160)
	at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:101)
	at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.invoke(LibertyJaxRsInvoker.java:273)
	at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:205)
	at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.invoke(LibertyJaxRsInvoker.java:444)
	at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:111)
	at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
	at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
	at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
	at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:123)
	at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:275)
	at com.ibm.ws.jaxrs20.endpoint.AbstractJaxRsWebEndpoint.invoke(AbstractJaxRsWebEndpoint.java:137)
	at com.ibm.websphere.jaxrs.server.IBMRestServlet.handleRequest(IBMRestServlet.java:146)
	at com.ibm.websphere.jaxrs.server.IBMRestServlet.doHead(IBMRestServlet.java:141)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:703)
	at com.ibm.websphere.jaxrs.server.IBMRestServlet.service(IBMRestServlet.java:96)
	at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1257)
	at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:745)
	at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:442)
	at com.ibm.ws.webcontainer.filter.WebAppFilterChain.invokeTarget(WebAppFilterChain.java:182)
	at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:93)
	at com.ibm.fhir.server.filter.rest.FHIRRestServletFilter.doFilter(FHIRRestServletFilter.java:142)
	at javax.servlet.http.HttpFilter.doFilter(HttpFilter.java:127)
	at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:201)
	at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:90)
	at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:1001)
	at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1139)
	at com.ibm.ws.webcontainer.webapp.WebApp.handleRequest(WebApp.java:5057)
	at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost$2.handleRequest(DynamicVirtualHost.java:314)
	at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:1006)
	at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost$2.run(DynamicVirtualHost.java:279)
	at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink$TaskWrapper.run(HttpDispatcherLink.java:1146)
	at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.wrapHandlerAndExecute(HttpDispatcherLink.java:427)
	at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.ready(HttpDispatcherLink.java:386)
	at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:566)
	at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleNewRequest(HttpInboundLink.java:500)
	at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.processRequest(HttpInboundLink.java:360)
	at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.ready(HttpInboundLink.java:327)
	at com.ibm.ws.channel.ssl.internal.SSLConnectionLink.determineNextChannel(SSLConnectionLink.java:1100)
	at com.ibm.ws.channel.ssl.internal.SSLConnectionLink.readyInboundPostHandshake(SSLConnectionLink.java:757)
	at com.ibm.ws.channel.ssl.internal.SSLConnectionLink$MyHandshakeCompletedCallback.complete(SSLConnectionLink.java:427)
	at com.ibm.ws.channel.ssl.internal.SSLUtils.handleHandshake(SSLUtils.java:954)
	at com.ibm.ws.channel.ssl.internal.SSLHandshakeIOCallback.complete(SSLHandshakeIOCallback.java:85)
	at com.ibm.ws.tcpchannel.internal.WorkQueueManager.requestComplete(WorkQueueManager.java:504)
	at com.ibm.ws.tcpchannel.internal.WorkQueueManager.attemptIO(WorkQueueManager.java:574)
	at com.ibm.ws.tcpchannel.internal.WorkQueueManager.workerRun(WorkQueueManager.java:958)
	at com.ibm.ws.tcpchannel.internal.WorkQueueManager$Worker.run(WorkQueueManager.java:1047)
	at com.ibm.ws.threading.internal.ExecutorServiceImpl$RunnableWrapper.run(ExecutorServiceImpl.java:238)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

And then we return a 400 Bad Request. For every request.

Environment
main branch (circa 4.7.0)

To Reproduce
You'd need to reproduce/simulate the Failure acquiring local host IP address condition, but I'm not sure how.

Expected behavior
Ideally, we'd obtain the current host/IP in a more reliable way.
If thats not feasible, then we should at least be more robust to this situation.
Its reasonable to me that this would result in an initialization error, but a server in this state should definitely be returning HTTP 500 errors, and not 400s.

Additional context
Possibly related to #1805 ?
@lmsurpre lmsurpre added the bug Something isn't working label Apr 13, 2021
@prb112 prb112 self-assigned this Apr 13, 2021
@prb112
Copy link
Contributor

prb112 commented Apr 13, 2021

I'll look at this one shortly.

prb112 added a commit that referenced this issue Apr 15, 2021
@prb112
Failure acquiring local host IP address results in 400 #2243
aadbdb8 
- Specific to some setups where the name lookup fails.

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>
@prb112
Copy link
Contributor

prb112 commented Apr 15, 2021

PR #2250

@prb112 prb112 added this to the Sprint 2021-05 milestone Apr 15, 2021
prb112 added a commit that referenced this issue Apr 20, 2021
@prb112 @lmsurpre
Failure acquiring local host IP address results in 400 #2243 (#2250)
bf2d0bf 
* Failure acquiring local host IP address results in 400 #2243

- Specific to some setups where the name lookup fails.

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

* Apply suggestions from code review

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

Co-authored-by: Lee Surprenant <lmsurpre@us.ibm.com>

* Update fhir-core/src/main/java/com/ibm/fhir/core/util/handler/IPHandler.java

Signed-off-by: Paul Bastide <pbastide@us.ibm.com>

Co-authored-by: Lee Surprenant <lmsurpre@us.ibm.com>
@lmsurpre
Copy link
Member Author

test suggestion from robin: turn off wifi and start the server

@lmsurpre
Copy link
Member Author

I tried the suggestion and wasn't able to reproduce the issue. I invoked a request against the server running on localhost and it worked just fine. I also purposefully misconfigured the server (with wifi off) to force an exception, and still no issue.

Lets close this one and we can consider reopening if we ever see this in the CI again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@prb112 prb112

Labels
bug Something isn't working
Projects
None yet
Milestone
Sprint 2021-05
Development

No branches or pull requests
2 participants
@prb112 @lmsurpre