Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could not fetch system information, we expected 200, got 500 #1747

Closed
fraser-m opened this issue Feb 1, 2016 · 3 comments
Closed

Could not fetch system information, we expected 200, got 500 #1747

fraser-m opened this issue Feb 1, 2016 · 3 comments
Assignees
Milestone

Comments

@fraser-m
Copy link

@fraser-m fraser-m commented Feb 1, 2016

Hi,

I've put up a graylog2 setup (web interface and server on same host) with an external elasticsearch cluster and external MongoDB server.

The server will start, however when i enter the system page i get the following error and stacktrace, it states the collection already exists(this happened when there was a blank db)

However, more interestingly, if i was to restart the server it will instantly crash and the only way i can get around this is to either delete the mongodb database or change the config to use a new Database.

Any ideas here? All machines are running on amazon linux and I am using Elasticsearch 1.7.4, mongodb 3.2 and most recent graylog2 version. The error points towards the issue being on MongoDB, so it it possibly a bug in mongodb 3.2?

Thanks for all and any help in advance

error:

(You caused a org.graylog2.restclient.lib.APIException. API call failed GET http://@127.0.0.1:12900/system/indexer/failures/count?since=2016-01-31T15:15:49.574Z returned 500 Internal Server Error body: Uncaught exception! A MultiException has 3 exceptions. They are: 1. com.google.inject.ProvisionException: Guice provision errors: 1) Error injecting constructor, com.mongodb.CommandFailureException: { "serverUsed" : "<<REDACTED MONGODB SERVER IP>:27017" , "ok" : 0.0 , "errmsg" : "collection already exists" , "code" : 48} at org.graylog2.indexer.IndexFailureServiceImpl.<init>(IndexFailureServiceImpl.java:35) while locating org.graylog2.indexer.IndexFailureServiceImpl while locating org.graylog2.indexer.IndexFailureService 1 error 2. java.lang.IllegalArgumentException: While attempting to resolve the dependencies of org.graylog2.rest.resources.system.indexer.FailuresResource errors were found 3. java.lang.IllegalStateException: Unable to perform operation: resolve on org.graylog2.rest.resources.system.indexer.FailuresResource )

stacktrace:

org.graylog2.restclient.lib.ApiClientImpl$ApiRequestBuilder#execute (ApiClientImpl.java:436)
org.graylog2.restclient.models.ClusterService#getIndexerFailureCountLast24Hours (ClusterService.java:103)
controllers.SystemController#index (SystemController.java:68)
Routes$$anonfun$routes$1$$anonfun$applyOrElse$43$$anonfun$apply$473#apply (routes_routing.scala:1606)
Routes$$anonfun$routes$1$$anonfun$applyOrElse$43$$anonfun$apply$473#apply (routes_routing.scala:1606)
play.core.Router$HandlerInvokerFactory$$anon$4#resultCall (Router.scala:264)
play.core.Router$HandlerInvokerFactory$JavaActionInvokerFactory$$anon$15$$anon$1#invocation (Router.scala:255)
play.core.j.JavaAction$$anon$1#call (JavaAction.scala:55)
play.GlobalSettings$1#call (GlobalSettings.java:67)
play.mvc.Security$AuthenticatedAction#call (Security.java:44)
play.core.j.JavaAction$$anonfun$11#apply (JavaAction.scala:82)
play.core.j.JavaAction$$anonfun$11#apply (JavaAction.scala:82)
scala.concurrent.impl.Future$PromiseCompletingRunnable#liftedTree1$1 (Future.scala:24)
scala.concurrent.impl.Future$PromiseCompletingRunnable#run (Future.scala:24)
play.core.j.HttpExecutionContext$$anon$2#run (HttpExecutionContext.scala:40)
play.api.libs.iteratee.Execution$trampoline$#execute (Execution.scala:46)
play.core.j.HttpExecutionContext#execute (HttpExecutionContext.scala:32)
scala.concurrent.impl.Future$#apply (Future.scala:31)
scala.concurrent.Future$#apply (Future.scala:485)
play.core.j.JavaAction$class#apply (JavaAction.scala:82)
play.core.Router$HandlerInvokerFactory$JavaActionInvokerFactory$$anon$15$$anon$1#apply (Router.scala:252)
play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4$$anonfun$apply$5#apply (Action.scala:130)
play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4$$anonfun$apply$5#apply (Action.scala:130)
play.utils.Threads$#withContextClassLoader (Threads.scala:21)
play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4#apply (Action.scala:129)
play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4#apply (Action.scala:128)
scala.Option#map (Option.scala:145)
play.api.mvc.Action$$anonfun$apply$1#apply (Action.scala:128)
play.api.mvc.Action$$anonfun$apply$1#apply (Action.scala:121)
play.api.libs.iteratee.Iteratee$$anonfun$mapM$1#apply (Iteratee.scala:483)
play.api.libs.iteratee.Iteratee$$anonfun$mapM$1#apply (Iteratee.scala:483)
play.api.libs.iteratee.Iteratee$$anonfun$flatMapM$1#apply (Iteratee.scala:519)
play.api.libs.iteratee.Iteratee$$anonfun$flatMapM$1#apply (Iteratee.scala:519)
play.api.libs.iteratee.Iteratee$$anonfun$flatMap$1$$anonfun$apply$14#apply (Iteratee.scala:496)
play.api.libs.iteratee.Iteratee$$anonfun$flatMap$1$$anonfun$apply$14#apply (Iteratee.scala:496)
scala.concurrent.impl.Future$PromiseCompletingRunnable#liftedTree1$1 (Future.scala:24)
scala.concurrent.impl.Future$PromiseCompletingRunnable#run (Future.scala:24)
akka.dispatch.TaskInvocation#run (AbstractDispatcher.scala:41)
akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask#exec (AbstractDispatcher.scala:393)
scala.concurrent.forkjoin.ForkJoinTask#doExec (ForkJoinTask.java:260)
scala.concurrent.forkjoin.ForkJoinPool$WorkQueue#runTask (ForkJoinPool.java:1339)
scala.concurrent.forkjoin.ForkJoinPool#runWorker (ForkJoinPool.java:1979)
scala.concurrent.forkjoin.ForkJoinWorkerThread#run (ForkJoinWorkerThread.java:107)
@joschi
Copy link
Contributor

@joschi joschi commented Feb 1, 2016

@fraser-m Could you please delete the index_failures collection in the MongoDB database used by Graylog and check if the error is still triggered when opening the "System" page?

Please also check the error log of your Graylog server node.

@fraser-m
Copy link
Author

@fraser-m fraser-m commented Feb 1, 2016

@joschi That seems to have done the trick for the first click on the page, however if i click on the 'system' button again the error returns, same error and stacktrace.

Any ideas?

joschi pushed a commit that referenced this issue Feb 4, 2016
Jochen Schalanda
Fixes #1747
@joschi joschi closed this in d4b4f9f Feb 4, 2016
@joschi
Copy link
Contributor

@joschi joschi commented Feb 4, 2016

@fraser-m I couldn't reproduce the issue with MongoDB 3.2.1 on my development machine.

$ mongod --version
db version v3.2.1
git version: a14d55980c2cdc565d4704a7e3ad37e4e535c1b2
allocator: system
modules: none
build environment:
    distarch: x86_64
    target_arch: x86_64

I have identified a part in our code that could've lead to Graylog trying to create the capped collection concurrently multiple times, though. This has been fixed in Graylog 2.0.0 and we'll maybe release another (last) Graylog 1.3.x patch release.

@joschi joschi added this to the 2.0.0 milestone Feb 4, 2016
@joschi joschi self-assigned this Feb 4, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.