Could Not retrieve Inputs (Fetching inputs failed with status: 500)

[123dev]

Problem description

I was editing one of the inputs to set the source value when it failed to save (can't remember the error, a red popup from the bottom of the screen was seen)
Afterwards the input was stopped.
Stopping Graylog, and restarting it back, or even rebooting the system, hasn't helped.
Since this error, I'm unable to access any of the inputs.
Although one of my browser windows was open on the extractors page of one the inputs, which is still accessible
http://10.1.1.94:9000/system/inputs/00c2670c-064b-415a-8768-104eb0e7ce08/56e021bade20c60caec10dde/extractors

But http://10.1.1.94:9000/system/inputs/00c2670c-064b-415a-8768-104eb0e7ce08/56e021bade20c60caec10dde is not

I have more inputs,

I see the following messages in log

2016-03-13 07:36:57,777 INFO : org.graylog2.bootstrap.ServerBootstrap - Graylog server up and running.
2016-03-13 07:36:58,019 WARN : org.graylog2.inputs.PersistedInputsImpl - Cannot instantiate persisted input. Exception caught:
java.lang.ClassCastException: com.mongodb.BasicDBObject cannot be cast to com.mongodb.BasicDBList
        at org.graylog2.inputs.InputImpl.getStaticFields(InputImpl.java:120) ~[graylog.jar:?]
        at org.graylog2.inputs.InputServiceImpl.getMessageInput(InputServiceImpl.java:337) ~[graylog.jar:?]
        at org.graylog2.inputs.PersistedInputsImpl.iterator(PersistedInputsImpl.java:54) [graylog.jar:?]
        at org.graylog2.shared.inputs.InputLauncher.launchAllPersisted(InputLauncher.java:115) [graylog.jar:?]
        at org.graylog2.shared.initializers.InputSetupService.run(InputSetupService.java:92) [graylog.jar:?]
        at com.google.common.util.concurrent.AbstractExecutionThreadService$1$2.run(AbstractExecutionThreadService.java:60) [graylog.jar:?]
        at com.google.common.util.concurrent.Callables$3.run(Callables.java:100) [graylog.jar:?]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_65]
2016-03-13 07:36:58,095 INFO : org.graylog2.shared.inputs.InputStateListener - Input [Raw/Plaintext UDP/56db757cde20c6076e325ce0] is now STARTING
2016-03-13 07:36:58,111 INFO : org.graylog2.shared.inputs.InputStateListener - Input [Syslog UDP/56dc3a45de20c6076e332b8b] is now STARTING
2016-03-13 07:36:58,119 INFO : org.graylog2.shared.inputs.InputStateListener - Input [GELF UDP/56dc3ae1de20c6076e332c33] is now STARTING
2016-03-13 07:36:58,130 INFO : org.graylog2.shared.inputs.InputStateListener - Input [Raw/Plaintext UDP/56dc4fbede20c6076e3342ba] is now STARTING
2016-03-13 07:36:58,162 INFO : org.graylog2.shared.inputs.InputStateListener - Input [Syslog UDP/56e021bade20c60caec10dde] is now STARTING
2016-03-13 07:36:58,324 WARN : org.graylog2.plugin.inputs.transports.NettyTransport - receiveBufferSize (SO_RCVBUF) for input SyslogUDPInput{title=Syslog UDP [Router], type=org.graylog2.inputs.syslog.udp.SyslogUDPInput, nodeId=00c2670c-064b-415a-8768-104eb0e7ce08} should be 262144 but is 212992.
2016-03-13 07:36:58,324 INFO : org.graylog2.shared.inputs.InputStateListener - Input [Syslog UDP/56e021bade20c60caec10dde] is now RUNNING
2016-03-13 07:36:58,330 WARN : org.graylog2.plugin.inputs.transports.NettyTransport - receiveBufferSize (SO_RCVBUF) for input SyslogUDPInput{title=Syslog UDP, type=org.graylog2.inputs.syslog.udp.SyslogUDPInput, nodeId=00c2670c-064b-415a-8768-104eb0e7ce08} should be 262144 but is 212992.
2016-03-13 07:36:58,331 WARN : org.graylog2.plugin.inputs.transports.NettyTransport - receiveBufferSize (SO_RCVBUF) for input RawUDPInput{title=Cisco Catalyst, type=org.graylog2.inputs.raw.udp.RawUDPInput, nodeId=00c2670c-064b-415a-8768-104eb0e7ce08} should be 262144 but is 212992.
2016-03-13 07:36:58,331 INFO : org.graylog2.shared.inputs.InputStateListener - Input [Syslog UDP/56dc3a45de20c6076e332b8b] is now RUNNING
2016-03-13 07:36:58,332 WARN : org.graylog2.plugin.inputs.transports.NettyTransport - receiveBufferSize (SO_RCVBUF) for input RawUDPInput{title=Raw/Plaintext UDP, type=org.graylog2.inputs.raw.udp.RawUDPInput, nodeId=00c2670c-064b-415a-8768-104eb0e7ce08} should be 262144 but is 212992.
2016-03-13 07:36:58,334 WARN : org.graylog2.plugin.inputs.transports.NettyTransport - receiveBufferSize (SO_RCVBUF) for input GELFUDPInput{title=GELF UDP, type=org.graylog2.inputs.gelf.udp.GELFUDPInput, nodeId=00c2670c-064b-415a-8768-104eb0e7ce08} should be 262144 but is 212992.
2016-03-13 07:36:58,343 INFO : org.graylog2.shared.inputs.InputStateListener - Input [Raw/Plaintext UDP/56dc4fbede20c6076e3342ba] is now RUNNING
2016-03-13 07:36:58,347 INFO : org.graylog2.shared.inputs.InputStateListener - Input [GELF UDP/56dc3ae1de20c6076e332c33] is now RUNNING
2016-03-13 07:36:58,353 INFO : org.graylog2.shared.inputs.InputStateListener - Input [Raw/Plaintext UDP/56db757cde20c6076e325ce0] is now RUNNING
2016-03-13 07:37:12,943 INFO : org.apache.shiro.session.mgt.AbstractValidatingSessionManager - Enabling session validation scheduler...
2016-03-13 07:37:13,274 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:4, serverValue:4}] to localhost:27017
2016-03-13 07:37:26,215 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
java.lang.ClassCastException: com.mongodb.BasicDBObject cannot be cast to com.mongodb.BasicDBList
        at org.graylog2.inputs.InputImpl.getStaticFields(InputImpl.java:120) ~[graylog.jar:?]
        at org.graylog2.rest.resources.system.inputs.InputsResource.getInputSummary(InputsResource.java:208) ~[graylog.jar:?]
        at org.graylog2.rest.resources.system.inputs.InputsResource.lambda$list$0(InputsResource.java:108) ~[graylog.jar:?]
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_65]
        at java.util.Iterator.forEachRemaining(Iterator.java:116) ~[?:1.8.0_65]
        at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801) ~[?:1.8.0_65]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_65]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_65]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_65]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_65]
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_65]
        at org.graylog2.rest.resources.system.inputs.InputsResource.list(InputsResource.java:109) ~[graylog.jar:?]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_65]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_65]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_65]
        at java.lang.reflect.Method.invoke(Method.java:497) ~[?:1.8.0_65]
        at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[graylog.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:315) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:297) [graylog.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:267) [graylog.jar:?]
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) [graylog.jar:?]
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) [graylog.jar:?]
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) [graylog.jar:?]
        at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) [graylog.jar:?]
        at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:224) [graylog.jar:?]
        at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176) [graylog.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_65]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_65]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_65]
2016-03-13 07:37:28,602 INFO : org.mongodb.driver.connection - Opened connection [connectionId{localValue:5, serverValue:5}] to localhost:27017
2016-03-13 07:38:07,068 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource
java.lang.ClassCastException: com.mongodb.BasicDBObject cannot be cast to com.mongodb.BasicDBList
        at org.graylog2.inputs.InputImpl.getStaticFields(InputImpl.java:120) ~[graylog.jar:?]
        at org.graylog2.rest.resources.system.inputs.InputsResource.getInputSummary(InputsResource.java:208) ~[graylog.jar:?]
        at org.graylog2.rest.resources.system.inputs.InputsResource.lambda$list$0(InputsResource.java:108) ~[graylog.jar:?]
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_65]

Steps to reproduce the problem

Don't know how to reproduce the problem, as I have other inputs that I managed to edit the source without issues,
The Input in question was the Netflow plugin input.
https://marketplace.graylog.org/addons/29bcffe3-816e-402f-89c8-06a8e2657c34

The alpha 5 was in general working very nicely (great job folks) aside from few glitches here and there which I will report (if they are not already reported) I haven't had any serious issue.
However now it's quite broken

Thanks

Environment

• Graylog Version: 2.0 alpha 5
• Elasticsearch Version: 2.2.0
• MongoDB Version: 3.2.4
• Operating System: ubuntu 14.04 TLS
• Browser version: Chrome x64

[123dev]

I Managed to fix the problem.

I looked at the MongoDB entries for the inputs and noticed the following.

• The source value was properly overwritten
• However during saving, the static field that was previously added, was badly saved.

Instead of saving it in an array such as

    "static_fields" : [
        {
            "value" : "Netflow_UDP", 
            "key" : "GraylogInput"
        }
    ]

It was saved like this:

    "static_fields" : {
        "GraylogInput" : "Netflow_UDP"
    }

Deleting the input from MongoDB and adding it back from the UI fixed the problem.

Though I suppose there could be a glitch in the saving / update code that caused the corruption.

[kroepke]

Actually it looks like this is related to 03927cb

This might be a bug regarding backwards compatibility

[joschi]

Related to Graylog2/graylog2-images#118.

[bernd]

This has been fixed in b612c97. Thank you for the report!