Skip to content
This repository has been archived by the owner on Dec 24, 2019. It is now read-only.

Fail to connect to /83.212.75.210:8080 #445

Closed
phkrief opened this issue Dec 5, 2019 · 16 comments
Closed

Fail to connect to /83.212.75.210:8080 #445

phkrief opened this issue Dec 5, 2019 · 16 comments
Assignees
@jdirocco @gertom
Labels
bug high priority
Milestone
1.18.0.rev0

Comments

@phkrief
Copy link

phkrief commented Dec 5, 2019

When I try to run a search I get an error. I cannot use our instance since then...

@md2manoppello is it the same issue as before?

image

Here are my preferences, if you have a doubt:
image

Thx
@phkrief phkrief added this to the 1.18.0.rev0 milestone Dec 5, 2019
@blueoly
Copy link
Contributor

blueoly commented Dec 5, 2019

Hi @phkrief

Can you try again?

@blueoly
Copy link
Contributor

blueoly commented Dec 5, 2019

Or not try again right now.

I am running tests to find out the source of the memory leak of the metric platform and I am the KB service crashed, even if I am not using it. I restarted it when I saw your issue but it crashed again after a few seconds.

@geryxyz geryxyz removed their assignment Dec 5, 2019
@phkrief
Copy link
Author

phkrief commented Dec 5, 2019

Thank you so much for taking the time to investigate, @blueoly. I really appreciate.

Keep me posted and I will will try again.
Thanks

@jdirocco
Copy link
Member

jdirocco commented Dec 6, 2019

Hi @blueoly , @phkrief ,
I investigated why the server crashes, and the issue depends on the deployment process. The kb-db MongoDB container doesn't import the dump because of a mongo crash (see the listing below). The services have been rebuilt and started, but they still crash during the import step.
So, the kb-db service is still alive but the DB doesn't work. Consequently, the kb-service stops with a timeout error.
I installed the docker infrastructure on different machines and I didn't get this exception. @blueoly can you help me?

kb-db_1           | 2019-12-06T09:31:11.865+0000
kb-db_1           | 2019-12-06T09:31:12.150+0000	error: read tcp 127.0.0.1:57124->127.0.0.1:27017: read: connection reset by peer
kb-db_1           | 2019-12-06T09:31:12.162+0000	error: read tcp 127.0.0.1:57124->127.0.0.1:27017: read: connection reset by peer
kb-db_1           | 2019-12-06T09:31:12.175+0000	error: read tcp 127.0.0.1:57124->127.0.0.1:27017: read: connection reset by peer
kb-db_1           | 2019-12-06T09:31:12.180+0000	CROSSMINER.relation  1.59GB
kb-db_1           | panic: send on closed channel
kb-db_1           |
kb-db_1           | goroutine 9 [running]:
kb-db_1           | github.com/mongodb/mongo-tools/common/archive.(*RegularCollectionReceiver).Write(0xc0000a4fa0, 0xc00023c010, 0xf3, 0x1000000, 0xc0001423e8, 0x1, 0x0)
kb-db_1           | 	/data/mci/fd89fd3f26dd46ab2b08185b565d7ede/src/src/mongo/gotools/src/github.com/mongodb/mongo-tools/common/archive/demultiplexer.go:328 +0x5d
kb-db_1           | github.com/mongodb/mongo-tools/common/archive.(*Demultiplexer).BodyBSON(0xc001284000, 0xc00023c010, 0xf3, 0x1000000, 0x0, 0x0)
kb-db_1           | 	/data/mci/fd89fd3f26dd46ab2b08185b565d7ede/src/src/mongo/gotools/src/github.com/mongodb/mongo-tools/common/archive/demultiplexer.go:208 +0x1f5
kb-db_1           | github.com/mongodb/mongo-tools/common/archive.(*Parser).ReadBlock(0xc00023c000, 0x55f79349c720, 0xc001284000, 0x0, 0x0)
kb-db_1           | 	/data/mci/fd89fd3f26dd46ab2b08185b565d7ede/src/src/mongo/gotools/src/github.com/mongodb/mongo-tools/common/archive/parser.go:141 +0x150
kb-db_1           | github.com/mongodb/mongo-tools/common/archive.(*Parser).ReadAllBlocks(0xc00023c000, 0x55f79349c720, 0xc001284000, 0x0, 0x0)
kb-db_1           | 	/data/mci/fd89fd3f26dd46ab2b08185b565d7ede/src/src/mongo/gotools/src/github.com/mongodb/mongo-tools/common/archive/parser.go:105 +0x4b
kb-db_1           | github.com/mongodb/mongo-tools/common/archive.(*Demultiplexer).Run(0xc001284000, 0x0, 0x0)
kb-db_1           | 	/data/mci/fd89fd3f26dd46ab2b08185b565d7ede/src/src/mongo/gotools/src/github.com/mongodb/mongo-tools/common/archive/demultiplexer.go:61 +0xa0
kb-db_1           | github.com/mongodb/mongo-tools/mongorestore.(*MongoRestore).Restore.func1(0xc00008bc70, 0xc000134480, 0xc000148300)
kb-db_1           | 	/data/mci/fd89fd3f26dd46ab2b08185b565d7ede/src/src/mongo/gotools/src/github.com/mongodb/mongo-tools/mongorestore/mongorestore.go:399 +0x38
kb-db_1           | created by github.com/mongodb/mongo-tools/mongorestore.(*MongoRestore).Restore
kb-db_1           | 	/data/mci/fd89fd3f26dd46ab2b08185b565d7ede/src/src/mongo/gotools/src/github.com/mongodb/mongo-tools/mongorestore/mongorestore.go:398 +0x5f1
kb-db_1           | panic: close of closed channel
kb-db_1           |
kb-db_1           | goroutine 67 [running]:
kb-db_1           | github.com/mongodb/mongo-tools/common/archive.(*RegularCollectionReceiver).Read(0xc0000a4fa0, 0xc003286000, 0x4, 0x1000000, 0x0, 0x0, 0x0)
kb-db_1           | 	/data/mci/fd89fd3f26dd46ab2b08185b565d7ede/src/src/mongo/gotools/src/github.com/mongodb/mongo-tools/common/archive/demultiplexer.go:263 +0x548
kb-db_1           | io.ReadAtLeast(0x7f7783b8e000, 0xc0000a4fa0, 0xc003286000, 0x4, 0x1000000, 0x4, 0x4, 0xc00004bf48, 0xc001240d80)
kb-db_1           | 	/opt/golang/go1.11/src/io/io.go:310 +0x8a
kb-db_1           | github.com/mongodb/mongo-tools/common/db.(*BSONSource).LoadNext(0xc00012e600, 0x55f792ccd665, 0xc0000982a0, 0xc00004bf48)
kb-db_1           | 	/data/mci/fd89fd3f26dd46ab2b08185b565d7ede/src/src/mongo/gotools/src/github.com/mongodb/mongo-tools/common/db/bson_stream.go:87 +0xcc
kb-db_1           | github.com/mongodb/mongo-tools/common/db.(*DecodedBSONSource).Next(0xc0000a20a0, 0x55f793411c20, 0xc00013a120, 0xc00869f900)
kb-db_1           | 	/data/mci/fd89fd3f26dd46ab2b08185b565d7ede/src/src/mongo/gotools/src/github.com/mongodb/mongo-tools/common/db/bson_stream.go:62 +0x39
kb-db_1           | github.com/mongodb/mongo-tools/mongorestore.(*MongoRestore).RestoreCollectionToDB.func1(0xc0000a20a0, 0xc00008bc70, 0xc000094ac0, 0xa, 0xc000094acb, 0x8, 0xc0001da000, 0xc0000982a0, 0xc0001e4038)
kb-db_1           | 	/data/mci/fd89fd3f26dd46ab2b08185b565d7ede/src/src/mongo/gotools/src/github.com/mongodb/mongo-tools/mongorestore/restore.go:254 +0xc8
kb-db_1           | created by github.com/mongodb/mongo-tools/mongorestore.(*MongoRestore).RestoreCollectionToDB
kb-db_1           | 	/data/mci/fd89fd3f26dd46ab2b08185b565d7ede/src/src/mongo/gotools/src/github.com/mongodb/mongo-tools/mongorestore/restore.go:252 +0x39a
kb-db_1           | /home/mongo-restore.sh: line 25:    11 Killed                  /entrypoint.sh mongod --logpath /var/log/mongodb.log --logappend

@phkrief
Copy link
Author

phkrief commented Dec 6, 2019

Dear @md2manoppello and @blueoly,

Thank you so much, both of you, for investigating.
I really bless the decision we made when we decided to switch to an instance maintained by somebody else but us.
At least, it shows that the issues we are facing are not coming from our incompetency but from the complexity of the project and the data it has to analyze.

It was so frustrating to fill guilty when something was not working in our ci4 instance. Now, at least, we know that we have the same configuration as anybody else and all of us will be able to take advantage of what we learn.

So, thank you again for your great support
Kind regards

FYI @davidediruscio @tdegueul @borisbaldassari

@blueoly
Copy link
Contributor

blueoly commented Dec 6, 2019

@phkrief As I finished with my experiments, I restarted the whole instance.

The issue is not addressed but now you will be able to connect to the instance.

@phkrief
Copy link
Author

phkrief commented Dec 6, 2019

Dear @blueoly ,
Actually, when I try to launch a simple search, I get a timeout:
image
On the other hand, I can reach http://83.212.75.210:8080/swagger-ui.html
So I don't know how to interpret this error.
Thanks

@jdirocco
Copy link
Member

jdirocco commented Dec 6, 2019 via email

@phkrief
Copy link
Author

phkrief commented Dec 6, 2019 •
edited
Loading

Actually, I just installed the new version @md2manoppello and I get another error:
image

and the log says:

io.swagger.client.ApiException:
at io.swagger.client.ApiClient.handleResponse(ApiClient.java:923)
at io.swagger.client.ApiClient.execute(ApiClient.java:839)
at io.swagger.client.api.ArtifactsRestControllerApi.getProjectUsingGETWithHttpInfo(ArtifactsRestControllerApi.java:930)
at io.swagger.client.api.ArtifactsRestControllerApi.getProjectUsingGET(ArtifactsRestControllerApi.java:913)
at org.eclipse.scava.plugin.projectsearch.search.tab.search.ExpressionSearchTabModel.getNextPageResults(ExpressionSearchTabModel.java:33)
at org.eclipse.scava.plugin.projectsearch.search.tab.search.SearchTabController.loadNextPage(SearchTabController.java:50)
at org.eclipse.scava.plugin.projectsearch.search.tab.search.SearchTabController.init(SearchTabController.java:44)
at org.eclipse.scava.plugin.projectsearch.search.SearchController.onSearch(SearchController.java:63)
at org.eclipse.scava.plugin.projectsearch.search.SearchView.lambda$2(SearchView.java:229)
at org.eclipse.scava.plugin.mvc.event.direct.EventManager.invoke(EventManager.java:43)
at org.eclipse.scava.plugin.projectsearch.search.SearchView.search(SearchView.java:229)
at org.eclipse.scava.plugin.projectsearch.search.SearchView.access$0(SearchView.java:223)
at org.eclipse.scava.plugin.projectsearch.search.SearchView$3.widgetSelected(SearchView.java:119)
at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:252)
at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:89)
at org.eclipse.swt.widgets.Display.sendEvent(Display.java:4406)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1512)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1535)
at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1520)
at org.eclipse.swt.widgets.Widget.notifyListeners(Widget.java:1324)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:4193)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3810)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine$5.run(PartRenderingEngine.java:1160)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338)
at org.eclipse.e4.ui.internal.workbench.swt.PartRenderingEngine.run(PartRenderingEngine.java:1049)
at org.eclipse.e4.ui.internal.workbench.E4Workbench.createAndRunUI(E4Workbench.java:155)
at org.eclipse.ui.internal.Workbench.lambda$3(Workbench.java:633)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:338)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:557)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:150)
at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:150)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:137)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:107)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:400)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:660)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:597)
at org.eclipse.equinox.launcher.Main.run(Main.java:1468)

Any idea ?

@phkrief
Copy link
Author

phkrief commented Dec 6, 2019

The message associated is:

Swagger client exception details:
Error code: 500
Response headers:
Content-Type
application/json;charset=UTF-8
Transfer-Encoding
chunked
Date
Fri, 06 Dec 2019 17:59:39 GMT
Connection
close
OkHttp-Sent-Millis
1575655148833
OkHttp-Received-Millis
1575655179196
Response body:
{
"timestamp": 1575655179141,
"status": 500,
"error": "Internal Server Error",
"exception": "org.springframework.dao.DataAccessResourceFailureException",
"message": "Timed out after 30000 ms while waiting for a server that matches ReadPreferenceServerSelector{readPreference\u003dprimary}. Client view of cluster state is {type\u003dSTANDALONE, servers\u003d[{address\u003dkb-db:27017, type\u003dUNKNOWN, state\u003dCONNECTING, exception\u003d{com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused (Connection refused)}}]; nested exception is com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting for a server that matches ReadPreferenceServerSelector{readPreference\u003dprimary}. Client view of cluster state is {type\u003dSTANDALONE, servers\u003d[{address\u003dkb-db:27017, type\u003dUNKNOWN, state\u003dCONNECTING, exception\u003d{com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused (Connection refused)}}]",
"path": "/api/artifacts/search/json"
}

@jdirocco
Copy link
Member

jdirocco commented Dec 6, 2019 via email

@phkrief
Copy link
Author

phkrief commented Dec 6, 2019

Do we have idea from which component it comes from?

@jdirocco
Copy link
Member

jdirocco commented Dec 6, 2019 via email

@phkrief
Copy link
Author

phkrief commented Dec 6, 2019

I still don't understand why some instances works and some others don'. And why we cannot compare both configurations to solve the differences.

Do you have an idea of the main differences between your instance and AEUB but yours is sitting in Italy and Konstantinos' is sitting in Greece ;-)?

@blueoly
Copy link
Contributor

blueoly commented Dec 7, 2019

I pulled the latest scava-deployment repo with the latest changes from @md2manoppello , and after that the kb-service seems to work without crashes.

@phkrief
Copy link
Author

phkrief commented Dec 8, 2019

I works nicely now... Thanks a lot, dear @blueoly

@phkrief phkrief closed this as completed Dec 8, 2019
@phkrief phkrief mentioned this issue Dec 8, 2019
Open
@davidediruscio davidediruscio mentioned this issue Jan 6, 2020
Closed
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@jdirocco jdirocco

@gertom gertom

Labels
bug high priority
Projects
None yet
Milestone
1.18.0.rev0
Development

No branches or pull requests
5 participants
@geryxyz @blueoly @phkrief @jdirocco @gertom