Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Invalid Zigbee2MQTT payload causes a cascading failure #303

Closed
climategadgets opened this issue Dec 27, 2023 · 1 comment
Closed

Invalid Zigbee2MQTT payload causes a cascading failure #303

climategadgets opened this issue Dec 27, 2023 · 1 comment
Labels
FATAL fault tolerance JSON MQTT Zigbee

Comments

@climategadgets
Copy link
Member

Expected Behavior

No matter what, the system recovers from invalid input.

Existing Behavior

Invalid Zigbee2MQTT payload causes a system wide failure, propagating to more than one sensor. There is no workaround, the application or the container must be restarted.

Context

Invalid payload (one or more) was issued by Zigbee2MQTT at some point when devices were being removed from the hub and added to it.

Exception Trace

Observed at rev. 959638e:

 java.lang.IllegalStateException: Can't parse JSON: 
        at net.sf.dz3r.device.z2m.v1.Z2MJsonListener.convert(Z2MJsonListener.java:108) ~[dz3r-mqtt-4.2.0-SNAPSHOT.jar:?]
        Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Assembly trace from producer [reactor.core.publisher.FluxMapFuseable] :
        reactor.core.publisher.Flux.map
        net.sf.dz3r.device.z2m.v1.Z2MJsonListener.getFlux(Z2MJsonListener.java:86)
Error has been observed at the following site(s):
        *_______Flux.map ⇢ at net.sf.dz3r.device.z2m.v1.Z2MJsonListener.getFlux(Z2MJsonListener.java:86)
        |_               ⇢ at net.sf.dz3r.device.z2m.v1.Z2MJsonListener.getFlux(Z2MJsonListener.java:38)
        |_ Flux.doOnNext ⇢ at net.sf.dz3r.signal.filter.TimeoutGuard.compute(TimeoutGuard.java:122)
        |_ Flux.doOnNext ⇢ at net.sf.dz3r.signal.filter.TimeoutGuard.compute(TimeoutGuard.java:123)
Original Stack Trace:
                at net.sf.dz3r.device.z2m.v1.Z2MJsonListener.convert(Z2MJsonListener.java:108) ~[dz3r-mqtt-4.2.0-SNAPSHOT.jar:?]
                at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onNext(FluxFilterFuseable.java:118) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxFlatMap$FlatMapMain.drainLoop(FluxFlatMap.java:716) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxFlatMap$FlatMapMain.drain(FluxFlatMap.java:592) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxFlatMap$FlatMapInner.onNext(FluxFlatMap.java:975) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxReplay$SizeAndTimeBoundReplayBuffer.replayFused(FluxReplay.java:300) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxReplay$SizeAndTimeBoundReplayBuffer.replay(FluxReplay.java:494) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxReplay$ReplaySubscriber.onNext(FluxReplay.java:1345) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.SinkManyEmitterProcessor.drain(SinkManyEmitterProcessor.java:476) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.SinkManyEmitterProcessor.tryEmitNext(SinkManyEmitterProcessor.java:273) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.SinkManySerialized.tryEmitNext(SinkManySerialized.java:100) ~[reactor-core-3.6.0.jar:3.6.0]
                at net.sf.dz3r.device.mqtt.v2async.MqttListenerImpl.callback(MqttListenerImpl.java:119) ~[dz3r-mqtt-4.2.0-SNAPSHOT.jar:?]
                at net.sf.dz3r.device.mqtt.v2async.MqttListenerImpl.lambda$createFlux$4(MqttListenerImpl.java:89) ~[dz3r-mqtt-4.2.0-SNAPSHOT.jar:?]
                at com.hivemq.client.internal.mqtt.MqttAsyncClient$CallbackSubscriber.onNext(MqttAsyncClient.java:303) [hivemq-mqtt-client-1.3.3.jar:?]
                at com.hivemq.client.internal.mqtt.MqttAsyncClient$CallbackSubscriber.onNext(MqttAsyncClient.java:288) [hivemq-mqtt-client-1.3.3.jar:?]
                at com.hivemq.client.rx.FlowableWithSingle$SingleFutureSubscriber.onNext(FlowableWithSingle.java:406) [hivemq-mqtt-client-1.3.3.jar:?]
                at com.hivemq.client.internal.rx.operators.FlowableWithSingleCombine$SplitSubscriber$Default.tryOnNextActual(FlowableWithSingleCombine.java:235) [hivemq-mqtt-client-1.3.3.jar:?]
                at com.hivemq.client.internal.rx.operators.FlowableWithSingleCombine$SplitSubscriber.tryOnNext(FlowableWithSingleCombine.java:200) [hivemq-mqtt-client-1.3.3.jar:?]
                at io.reactivex.internal.operators.flowable.FlowableObserveOn$ObserveOnConditionalSubscriber.runAsync(FlowableObserveOn.java:649) [rxjava-2.2.21.jar:?]
                at io.reactivex.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.run(FlowableObserveOn.java:176) [rxjava-2.2.21.jar:?]
                at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66) [rxjava-2.2.21.jar:?]
                at io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57) [rxjava-2.2.21.jar:?]
                at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
                at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
                at java.lang.Thread.run(Thread.java:840) [?:?]
Caused by: com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
 at [Source: (String)""; line: 1, column: 0]
        at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59) ~[jackson-databind-2.15.3.jar:2.15.3]
        at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4916) ~[jackson-databind-2.15.3.jar:2.15.3]
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4818) ~[jackson-databind-2.15.3.jar:2.15.3]
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3772) ~[jackson-databind-2.15.3.jar:2.15.3]
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3740) ~[jackson-databind-2.15.3.jar:2.15.3]
        at net.sf.dz3r.device.z2m.v1.Z2MJsonListener.convert(Z2MJsonListener.java:97) ~[dz3r-mqtt-4.2.0-SNAPSHOT.jar:?]
        at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onNext(FluxFilterFuseable.java:118) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.FluxFlatMap$FlatMapMain.drainLoop(FluxFlatMap.java:716) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.FluxFlatMap$FlatMapMain.drain(FluxFlatMap.java:592) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.FluxFlatMap$FlatMapInner.onNext(FluxFlatMap.java:975) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.FluxOnAssembly$OnAssemblySubscriber.onNext(FluxOnAssembly.java:539) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.FluxReplay$SizeAndTimeBoundReplayBuffer.replayFused(FluxReplay.java:300) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.FluxReplay$SizeAndTimeBoundReplayBuffer.replay(FluxReplay.java:494) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.FluxReplay$ReplaySubscriber.onNext(FluxReplay.java:1345) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.SinkManyEmitterProcessor.drain(SinkManyEmitterProcessor.java:476) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.SinkManyEmitterProcessor.tryEmitNext(SinkManyEmitterProcessor.java:273) ~[reactor-core-3.6.0.jar:3.6.0]
        at reactor.core.publisher.SinkManySerialized.tryEmitNext(SinkManySerialized.java:100) ~[reactor-core-3.6.0.jar:3.6.0]
        at net.sf.dz3r.device.mqtt.v2async.MqttListenerImpl.callback(MqttListenerImpl.java:119) ~[dz3r-mqtt-4.2.0-SNAPSHOT.jar:?]
        at net.sf.dz3r.device.mqtt.v2async.MqttListenerImpl.lambda$createFlux$4(MqttListenerImpl.java:89) ~[dz3r-mqtt-4.2.0-SNAPSHOT.jar:?]
        at com.hivemq.client.internal.mqtt.MqttAsyncClient$CallbackSubscriber.onNext(MqttAsyncClient.java:303) [hivemq-mqtt-client-1.3.3.jar:?]
        at com.hivemq.client.internal.mqtt.MqttAsyncClient$CallbackSubscriber.onNext(MqttAsyncClient.java:288) [hivemq-mqtt-client-1.3.3.jar:?]
        at com.hivemq.client.rx.FlowableWithSingle$SingleFutureSubscriber.onNext(FlowableWithSingle.java:406) [hivemq-mqtt-client-1.3.3.jar:?]
        at com.hivemq.client.internal.rx.operators.FlowableWithSingleCombine$SplitSubscriber$Default.tryOnNextActual(FlowableWithSingleCombine.java:235) [hivemq-mqtt-client-1.3.3.jar:?]
        at com.hivemq.client.internal.rx.operators.FlowableWithSingleCombine$SplitSubscriber.tryOnNext(FlowableWithSingleCombine.java:200) [hivemq-mqtt-client-1.3.3.jar:?]
        at io.reactivex.internal.operators.flowable.FlowableObserveOn$ObserveOnConditionalSubscriber.runAsync(FlowableObserveOn.java:649) [rxjava-2.2.21.jar:?]
        at io.reactivex.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.run(FlowableObserveOn.java:176) [rxjava-2.2.21.jar:?]
        at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66) [rxjava-2.2.21.jar:?]
        at io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57) [rxjava-2.2.21.jar:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
        at java.lang.Thread.run(Thread.java:840) [?:?]
climategadgets added a commit that referenced this issue Dec 27, 2023
@climategadgets
Fixed #303
f595dcc
@climategadgets
Copy link
Member Author

More (at rev. f595dcc):

2023-12-28 22:30:13,839 ERROR TimeoutGuard RxComputationThreadPool-4 [] kitchen-temperature: errored out
 java.lang.NumberFormatException: For input string: "null"
        at jdk.internal.math.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2054) ~[?:?]
        Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Assembly trace from producer [reactor.core.publisher.FluxMapFuseable] :
        reactor.core.publisher.Flux.map
        net.sf.dz3r.device.z2m.v1.Z2MJsonListener.getFlux(Z2MJsonListener.java:86)
Error has been observed at the following site(s):
        *_______Flux.map ⇢ at net.sf.dz3r.device.z2m.v1.Z2MJsonListener.getFlux(Z2MJsonListener.java:86)
        |_               ⇢ at net.sf.dz3r.device.z2m.v1.Z2MJsonListener.getFlux(Z2MJsonListener.java:38)
        |_ Flux.doOnNext ⇢ at net.sf.dz3r.signal.filter.TimeoutGuard.compute(TimeoutGuard.java:122)
        |_ Flux.doOnNext ⇢ at net.sf.dz3r.signal.filter.TimeoutGuard.compute(TimeoutGuard.java:123)
Original Stack Trace:
                at jdk.internal.math.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2054) ~[?:?]
                at jdk.internal.math.FloatingDecimal.parseDouble(FloatingDecimal.java:110) ~[?:?]
                at java.lang.Double.parseDouble(Double.java:651) ~[?:?]
                at net.sf.dz3r.device.z2m.v1.Z2MJsonListener.convert(Z2MJsonListener.java:101) ~[dz3r-mqtt-4.2.0-SNAPSHOT.jar:?]
                at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:129) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxPeekFuseable$PeekFuseableSubscriber.onNext(FluxPeekFuseable.java:210) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onNext(FluxFilterFuseable.java:118) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxFlatMap$FlatMapMain.drainLoop(FluxFlatMap.java:716) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxFlatMap$FlatMapMain.drain(FluxFlatMap.java:592) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxFlatMap$FlatMapInner.onNext(FluxFlatMap.java:975) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxReplay$SizeAndTimeBoundReplayBuffer.replayFused(FluxReplay.java:300) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxReplay$SizeAndTimeBoundReplayBuffer.replay(FluxReplay.java:494) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.FluxReplay$ReplaySubscriber.onNext(FluxReplay.java:1345) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.SinkManyEmitterProcessor.drain(SinkManyEmitterProcessor.java:476) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.SinkManyEmitterProcessor.tryEmitNext(SinkManyEmitterProcessor.java:273) ~[reactor-core-3.6.0.jar:3.6.0]
                at reactor.core.publisher.SinkManySerialized.tryEmitNext(SinkManySerialized.java:100) ~[reactor-core-3.6.0.jar:3.6.0]
                at net.sf.dz3r.device.mqtt.v2async.MqttListenerImpl.callback(MqttListenerImpl.java:119) ~[dz3r-mqtt-4.2.0-SNAPSHOT.jar:?]
                at net.sf.dz3r.device.mqtt.v2async.MqttListenerImpl.lambda$createFlux$4(MqttListenerImpl.java:89) ~[dz3r-mqtt-4.2.0-SNAPSHOT.jar:?]
                at com.hivemq.client.internal.mqtt.MqttAsyncClient$CallbackSubscriber.onNext(MqttAsyncClient.java:303) [hivemq-mqtt-client-1.3.3.jar:?]
                at com.hivemq.client.internal.mqtt.MqttAsyncClient$CallbackSubscriber.onNext(MqttAsyncClient.java:288) [hivemq-mqtt-client-1.3.3.jar:?]
                at com.hivemq.client.rx.FlowableWithSingle$SingleFutureSubscriber.onNext(FlowableWithSingle.java:406) [hivemq-mqtt-client-1.3.3.jar:?]
                at com.hivemq.client.internal.rx.operators.FlowableWithSingleCombine$SplitSubscriber$Default.tryOnNextActual(FlowableWithSingleCombine.java:235) [hivemq-mqtt-client-1.3.3.jar:?]
                at com.hivemq.client.internal.rx.operators.FlowableWithSingleCombine$SplitSubscriber.tryOnNext(FlowableWithSingleCombine.java:200) [hivemq-mqtt-client-1.3.3.jar:?]
                at io.reactivex.internal.operators.flowable.FlowableObserveOn$ObserveOnConditionalSubscriber.runAsync(FlowableObserveOn.java:649) [rxjava-2.2.21.jar:?]
                at io.reactivex.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.run(FlowableObserveOn.java:176) [rxjava-2.2.21.jar:?]
                at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66) [rxjava-2.2.21.jar:?]
                at io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57) [rxjava-2.2.21.jar:?]
                at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
                at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?]
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
                at java.lang.Thread.run(Thread.java:840) [?:?]

climategadgets added a commit that referenced this issue Dec 29, 2023
@climategadgets
Dodged another exception (#303)
4fbb955
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
FATAL fault tolerance JSON MQTT Zigbee
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@climategadgets