New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RESTful service is not completed yet #78

Closed
Yongyao opened this Issue Dec 22, 2016 · 9 comments

Comments

Projects
None yet
3 participants
@Yongyao
Collaborator

Yongyao commented Dec 22, 2016

@lewismc

When you get a chance, could you please finish the Restful service you started a while ago? I have made some changes to the Ranking module. I need to make sure the master branch is running okay before merging it with my local branch. Same thing with @quintinali

Right now, when I start the MUDROD-SERVICE in master and do a search. The result is like below
screen shot 2016-12-22 at 4 28 18 pm

@lewismc lewismc self-assigned this Jan 18, 2017

@lewismc lewismc added this to the 02/01/17 milestone Jan 18, 2017

@lewismc

This comment has been minimized.

Show comment
Hide comment
@lewismc

lewismc Jan 18, 2017

Collaborator

@Yongyao do you have a stack trace please?

Collaborator

lewismc commented Jan 18, 2017

@Yongyao do you have a stack trace please?

@lewismc lewismc added this to GUI + Webservices Development in AIST Master Schedule Feb 1, 2017

@lewismc

This comment has been minimized.

Show comment
Hide comment
@lewismc

lewismc Feb 2, 2017

Collaborator

Partially fixed in #83

Collaborator

lewismc commented Feb 2, 2017

Partially fixed in #83

@lewismc lewismc modified the milestones: 02/01/17, 02/15/2017 Feb 2, 2017

@fgreg

This comment has been minimized.

Show comment
Hide comment
@fgreg

fgreg Feb 6, 2017

Collaborator

Here's a stacktrace I get when I try to run http://localhost:8080/mudrod-service/services/metadata/search?query=temperature&operator=phrase. One caveat to this example though: I have not run any log ingestion on my local install yet so this failure could be because there is no data/indexes are missing?

2017-02-06 10:03:53,345 WARN  phase.PhaseInterceptorChain (LogUtils.java:doLog(449)) - Application {http://search.services.mudrod.esiptestbed/}SearchMetadataResource has thrown exception, unwinding now
org.apache.cxf.interceptor.Fault: all shards failed
	at org.apache.cxf.service.invoker.AbstractInvoker.createFault(AbstractInvoker.java:162)
	at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:128)
	at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:189)
	at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:99)
	at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
	at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
	at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
	at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
	at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:254)
	at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234)
	at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208)
	at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160)
	at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:180)
	at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:299)
	at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:223)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
	at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:274)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1041)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:603)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: Failed to execute phase [query], all shards failed; shardFailures {[nUShbJdnSbu_L3IczOvkXQ][mudrod][0]: RemoteTransportException[[nUShbJd][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: QueryShardException[No mapping found for [weight] in order to sort on]; }{[nUShbJdnSbu_L3IczOvkXQ][mudrod][1]: RemoteTransportException[[nUShbJd][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: QueryShardException[No mapping found for [weight] in order to sort on]; }{[nUShbJdnSbu_L3IczOvkXQ][mudrod][2]: RemoteTransportException[[nUShbJd][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: QueryShardException[No mapping found for [weight] in order to sort on]; }{[nUShbJdnSbu_L3IczOvkXQ][mudrod][3]: RemoteTransportException[[nUShbJd][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: QueryShardException[No mapping found for [weight] in order to sort on]; }{[nUShbJdnSbu_L3IczOvkXQ][mudrod][4]: RemoteTransportException[[nUShbJd][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: QueryShardException[No mapping found for [weight] in order to sort on]; }
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onFirstPhaseResult(AbstractSearchAsyncAction.java:208)
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.access$100(AbstractSearchAsyncAction.java:52)
	at org.elasticsearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:143)
	at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:51)
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1024)
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1126)
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1104)
	at org.elasticsearch.transport.TransportService$7.onFailure(TransportService.java:621)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.onFailure(ThreadContext.java:581)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:39)
	... 3 more
Caused by: RemoteTransportException[[nUShbJd][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: QueryShardException[No mapping found for [weight] in order to sort on];
Caused by: [mudrod/vxxUo48vQXuqiDsY8H-vPw] QueryShardException[No mapping found for [weight] in order to sort on]
	at org.elasticsearch.search.sort.FieldSortBuilder.build(FieldSortBuilder.java:262)
	at org.elasticsearch.search.sort.SortBuilder.buildSort(SortBuilder.java:156)
	at org.elasticsearch.search.SearchService.parseSource(SearchService.java:688)
	at org.elasticsearch.search.SearchService.createContext(SearchService.java:540)
	at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:516)
	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:251)
	at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:298)
	at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:295)
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69)
	at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:610)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:596)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Collaborator

fgreg commented Feb 6, 2017

Here's a stacktrace I get when I try to run http://localhost:8080/mudrod-service/services/metadata/search?query=temperature&operator=phrase. One caveat to this example though: I have not run any log ingestion on my local install yet so this failure could be because there is no data/indexes are missing?

2017-02-06 10:03:53,345 WARN  phase.PhaseInterceptorChain (LogUtils.java:doLog(449)) - Application {http://search.services.mudrod.esiptestbed/}SearchMetadataResource has thrown exception, unwinding now
org.apache.cxf.interceptor.Fault: all shards failed
	at org.apache.cxf.service.invoker.AbstractInvoker.createFault(AbstractInvoker.java:162)
	at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:128)
	at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:189)
	at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:99)
	at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59)
	at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96)
	at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
	at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121)
	at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:254)
	at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234)
	at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208)
	at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160)
	at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:180)
	at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:299)
	at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:223)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:621)
	at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:274)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:100)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:953)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1041)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:603)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
Caused by: Failed to execute phase [query], all shards failed; shardFailures {[nUShbJdnSbu_L3IczOvkXQ][mudrod][0]: RemoteTransportException[[nUShbJd][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: QueryShardException[No mapping found for [weight] in order to sort on]; }{[nUShbJdnSbu_L3IczOvkXQ][mudrod][1]: RemoteTransportException[[nUShbJd][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: QueryShardException[No mapping found for [weight] in order to sort on]; }{[nUShbJdnSbu_L3IczOvkXQ][mudrod][2]: RemoteTransportException[[nUShbJd][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: QueryShardException[No mapping found for [weight] in order to sort on]; }{[nUShbJdnSbu_L3IczOvkXQ][mudrod][3]: RemoteTransportException[[nUShbJd][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: QueryShardException[No mapping found for [weight] in order to sort on]; }{[nUShbJdnSbu_L3IczOvkXQ][mudrod][4]: RemoteTransportException[[nUShbJd][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: QueryShardException[No mapping found for [weight] in order to sort on]; }
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onFirstPhaseResult(AbstractSearchAsyncAction.java:208)
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.access$100(AbstractSearchAsyncAction.java:52)
	at org.elasticsearch.action.search.AbstractSearchAsyncAction$1.onFailure(AbstractSearchAsyncAction.java:143)
	at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:51)
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1024)
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1126)
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1104)
	at org.elasticsearch.transport.TransportService$7.onFailure(TransportService.java:621)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.onFailure(ThreadContext.java:581)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:39)
	... 3 more
Caused by: RemoteTransportException[[nUShbJd][127.0.0.1:9300][indices:data/read/search[phase/query]]]; nested: QueryShardException[No mapping found for [weight] in order to sort on];
Caused by: [mudrod/vxxUo48vQXuqiDsY8H-vPw] QueryShardException[No mapping found for [weight] in order to sort on]
	at org.elasticsearch.search.sort.FieldSortBuilder.build(FieldSortBuilder.java:262)
	at org.elasticsearch.search.sort.SortBuilder.buildSort(SortBuilder.java:156)
	at org.elasticsearch.search.SearchService.parseSource(SearchService.java:688)
	at org.elasticsearch.search.SearchService.createContext(SearchService.java:540)
	at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:516)
	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:251)
	at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:298)
	at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:295)
	at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69)
	at org.elasticsearch.transport.TransportService$7.doRun(TransportService.java:610)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:596)
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
@fgreg

This comment has been minimized.

Show comment
Hide comment
@fgreg

fgreg Feb 7, 2017

Collaborator

I think my exception above was due to no data. I was able to get a partial log ingestion and now the query returns results without exceptions.

Collaborator

fgreg commented Feb 7, 2017

I think my exception above was due to no data. I was able to get a partial log ingestion and now the query returns results without exceptions.

@lewismc

This comment has been minimized.

Show comment
Hide comment
@lewismc

lewismc Feb 9, 2017

Collaborator

ack @fgreg can you please verify if any of the other services are working? You can see the REST documentation on master branch by executing mvn miredot I believe.

Collaborator

lewismc commented Feb 9, 2017

ack @fgreg can you please verify if any of the other services are working? You can see the REST documentation on master branch by executing mvn miredot I believe.

@lewismc

This comment has been minimized.

Show comment
Hide comment
@lewismc

lewismc Feb 10, 2017

Collaborator

Closing this issue off for the time being. If there are individual bugs or improvements which we need to make then we can do so under separate tickets.

Collaborator

lewismc commented Feb 10, 2017

Closing this issue off for the time being. If there are individual bugs or improvements which we need to make then we can do so under separate tickets.

@lewismc lewismc closed this Feb 10, 2017

@lewismc lewismc removed this from GUI + Webservices Development in AIST Master Schedule Feb 23, 2017

@Yongyao Yongyao reopened this Mar 2, 2017

@Yongyao

This comment has been minimized.

Show comment
Hide comment
@Yongyao

Yongyao Mar 2, 2017

Collaborator

@lewismc @fgreg I just tested the master branch with the data we provided to you a few weeks ago. The mudrod-core works fine, but there is still a problem with the mudrod-service.

I am able to start the service and see the landing page, but once I type in some keywords, it takes forever to return the search results. I can see the results printed out in my eclipse console,

2017-03-01 21:51:14,608 INFO search.SearchMetadataResource (SearchMetadataResource.java:searchMetadata(76)) - Response received: "{\"PDResults\":[{\"Short Name\":\"SPURS1_DRIFTER\",\"Long Name\":\"Drifter data for the SPURS-1 N. Atlantic field campaign\",\"Topic\":\"Temperature Profiles, Salini...

However, I found an error on the browser,

screen shot 2017-03-01 at 9 53 18 pm

Collaborator

Yongyao commented Mar 2, 2017

@lewismc @fgreg I just tested the master branch with the data we provided to you a few weeks ago. The mudrod-core works fine, but there is still a problem with the mudrod-service.

I am able to start the service and see the landing page, but once I type in some keywords, it takes forever to return the search results. I can see the results printed out in my eclipse console,

2017-03-01 21:51:14,608 INFO search.SearchMetadataResource (SearchMetadataResource.java:searchMetadata(76)) - Response received: "{\"PDResults\":[{\"Short Name\":\"SPURS1_DRIFTER\",\"Long Name\":\"Drifter data for the SPURS-1 N. Atlantic field campaign\",\"Topic\":\"Temperature Profiles, Salini...

However, I found an error on the browser,

screen shot 2017-03-01 at 9 53 18 pm

@lewismc

This comment has been minimized.

Show comment
Hide comment
@lewismc

lewismc Mar 2, 2017

Collaborator

@Yongyao this is disappointing, would have been nice to have known about this a while back when I posted the message to our mailing list... never mind, we can sort it out now!
Can you try end-to-end to end ingestion with the data that Chris Finch made available. We are working on data from PO.DAAC now not the test data in Google Drive. If you cannot locate the data that Chris made available, then let me know here and I will make it available for you. Thank you

Collaborator

lewismc commented Mar 2, 2017

@Yongyao this is disappointing, would have been nice to have known about this a while back when I posted the message to our mailing list... never mind, we can sort it out now!
Can you try end-to-end to end ingestion with the data that Chris Finch made available. We are working on data from PO.DAAC now not the test data in Google Drive. If you cannot locate the data that Chris made available, then let me know here and I will make it available for you. Thank you

@lewismc

This comment has been minimized.

Show comment
Hide comment
@lewismc

lewismc Mar 2, 2017

Collaborator

@Yongyao please open a new ticket and tailor the issue to the error you are experiencing.
Bug reports are important, that's why I asked everyone to confirm... if no-one tries this out, and no-one files bug reports, then we don't know stuff is not functional. Thanks for re-opening the issue. Please close it and open a new issue with a detailed description of the bug you've found. Also please describe how to reproduce the bug.

@Yongyao @clarkgogogo @quintinali if you would like to see how bug reports should look, please see the following documentation, it will really help to understand the importance of why they are crucial to team development and good software.

http://blog.testlio.com/the-ideal-bug-report

Collaborator

lewismc commented Mar 2, 2017

@Yongyao please open a new ticket and tailor the issue to the error you are experiencing.
Bug reports are important, that's why I asked everyone to confirm... if no-one tries this out, and no-one files bug reports, then we don't know stuff is not functional. Thanks for re-opening the issue. Please close it and open a new issue with a detailed description of the bug you've found. Also please describe how to reproduce the bug.

@Yongyao @clarkgogogo @quintinali if you would like to see how bug reports should look, please see the following documentation, it will really help to understand the importance of why they are crucial to team development and good software.

http://blog.testlio.com/the-ideal-bug-report

@Yongyao Yongyao closed this Mar 2, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment