Unable to parse scan record

[jobfeikens]

Sometimes, my scan operation finishes with an error because it receives a scan record with invalid data:

E/ScanRecord(21849): unable to parse scan record: [4, 9, 84, 87, 83, 2, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] .

Full stack trace

E/ScanRecord(21849): unable to parse scan record: [4, 9, 84, 87, 83, 2, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
E/ScanRecord(21849): unable to parse scan record: [4, 9, 84, 87, 83, 2, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
D/BluetoothAdapter(21849): isLeEnabled(): ON
E/flutter (21849): [ERROR:flutter/lib/ui/ui_dart_state.cc(209)] Unhandled Exception: Exception: GenericFailure<ScanFailure>(code: ScanFailure.unknown, message: "result.scanRecord.serviceData must not be null")
E/flutter (21849): #0      Result.dematerialize.<anonymous closure> (package:reactive_ble_platform_interface/src/model/result.dart:22:13)
E/flutter (21849): #1      Result.iif (package:reactive_ble_platform_interface/src/model/result.dart:34:21)
E/flutter (21849): #2      Result.dematerialize (package:reactive_ble_platform_interface/src/model/result.dart:15:28)
E/flutter (21849): #3      DeviceScannerImpl.scanForDevices.<anonymous closure>.<anonymous closure> (package:flutter_reactive_ble/src/device_scanner.dart:58:49)
E/flutter (21849): #4      _MapStream._handleData (dart:async/stream_pipe.dart:213:31)
E/flutter (21849): #5      _rootRunUnary (dart:async/zone.dart:1434:47)
E/flutter (21849): #6      _CustomZone.runUnary (dart:async/zone.dart:1335:19)
E/flutter (21849): #7      _CustomZone.runUnaryGuarded (dart:async/zone.dart:1244:7)
E/flutter (21849): #8      _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:341:11)
E/flutter (21849): #9      _BufferingStreamSubscription._add (dart:async/stream_impl.dart:271:7)
E/flutter (21849): #10     _ForwardingStreamSubscription._add (dart:async/stream_pipe.dart:123:11)
E/flutter (21849): #11     _MapStream._handleData (dart:async/stream_pipe.dart:218:10)
E/flutter (21849): #12     _rootRunUnary (dart:async/zone.dart:1434:47)
E/flutter (21849): #13     _CustomZone.runUnary (dart:async/zone.dart:1335:19)
E/flutter (21849): #14     _CustomZone.runUnaryGuarded (dart:async/zone.dart:1244:7)
E/flutter (21849): #15     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:341:11)
E/flutter (21849): #16     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:271:7)
E/flutter (21849): #17     _ForwardingStreamSubscription._add (dart:async/stream_pipe.dart:123:11)
E/flutter (21849): #18     _MapStream._handleData (dart:async/stream_pipe.dart:218:10)
E/flutter (21849): #19     _rootRunUnary (dart:async/zone.dart:1434:47)
E/flutter (21849): #20     _CustomZone.runUnary (dart:async/zone.dart:1335:19)
E/flutter (21849): #21     _CustomZone.runUnaryGuarded (dart:async/zone.dart:1244:7)
E/flutter (21849): #22     CastStreamSubscription._onData (dart:_internal/async_cast.dart:85:11)
E/flutter (21849): #23     _rootRunUnary (dart:async/zone.dart:1434:47)
E/flutter (21849): #24     _CustomZone.runUnary (dart:async/zone.dart:1335:19)
E/flutter (21849): #25     _CustomZone.runUnaryGuarded (dart:async/zone.dart:1244:7)
E/flutter (21849): #26     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:341:11)
E/flutter (21849): #27     _DelayedData.perform (dart:async/stream_impl.dart:591:14)
E/flutter (21849): #28     _StreamImplEvents.handleNext (dart:async/stream_impl.dart:706:11)
E/flutter (21849): #29     _PendingEvents.schedule.<anonymous closure> (dart:async/stream_impl.dart:663:7)
E/flutter (21849): #30     _rootRun (dart:async/zone.dart:1418:47)
E/flutter (21849): #31     _CustomZone.run (dart:async/zone.dart:1328:19)
E/flutter (21849): #32     _CustomZone.runGuarded (dart:async/zone.dart:1236:7)
E/flutter (21849): #33     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1276:23)
E/flutter (21849): #34     _rootRun (dart:async/zone.dart:1426:13)
E/flutter (21849): #35     _CustomZone.run (dart:async/zone.dart:1328:19)
E/flutter (21849): #36     _CustomZone.runGuarded (dart:async/zone.dart:1236:7)
E/flutter (21849): #37     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1276:23)
E/flutter (21849): #38     _microtaskLoop (dart:async/schedule_microtask.dart:40:21)
E/flutter (21849): #39     _startMicrotaskLoop (dart:async/schedule_microtask.dart:49:5)
E/flutter (21849): 

Maybe the field serviceData in DiscoveredDevice should be nullable or the field should be an empty Map in case of no service data.
I think its also reasonable to just ignore the scan record as the error indicates that the scan record bytes simply can't be parsed.

Platform: Android 12

[jobfeikens]

The warning is printed by parseFromBytes in Android's ScanRecord.java:

} catch (Exception e) {
    Log.e(TAG, "unable to parse scan record: " + Arrays.toString(scanRecord));
    // As the record is invalid, ignore all the parsed results for this packet
    // and return an empty record with raw scanRecord bytes in results
    return new ScanRecord(null, null, null, null, -1, Integer.MIN_VALUE, null, scanRecord);
}

[Taym95]

The warning is printed by parseFromBytes in Android's ScanRecord.java:

} catch (Exception e) {
    Log.e(TAG, "unable to parse scan record: " + Arrays.toString(scanRecord));
    // As the record is invalid, ignore all the parsed results for this packet
    // and return an empty record with raw scanRecord bytes in results
    return new ScanRecord(null, null, null, null, -1, Integer.MIN_VALUE, null, scanRecord);
}

Hi, this is an Exception coming from Android itself and not a warning! Android validates the data before sending it to RxAndroidBle then sends it to our library, this issue is related to your device, you need to check the advertisement packet and make sure they do not have any additional zeroes or anything, make sure the length of each field is correct.

[jobfeikens]

Hi I'm scanning for all devices, so I cannot decide which scan records I receive. I have no idea which device sends this scan record. I think the reason the exception is caught in the code above, is so that the scan doesn't stop on receiving invalid scan records.

[Taym95]

Yes that is the reason, unfortunately, we can not ignore an exception coming from Android/RxAndroidBle

[remonh87]

@jobfeikens you can do one trick and that is filter out the service uuid of the device you want. I am pretty sure that Android won't propagate the results in that case so you won't get the crash.

flutter_reactive_ble/packages/flutter_reactive_ble/lib/src/reactive_ble.dart

Line 228 in 8d66469

required List<Uuid> withServices,

Also I propose to leave this issue open because if this is something in the rxandroidble we may need to fix it there and keep this open for traceability.

[jobfeikens]

Thanks @remonh87, this is a good solution. Unfortunately we decided not to advertise with our proprietary service uuid since the scan record was full 😞, and so we do filtering based on manufacturer data. I'll take an attempt at fixing the issue in RxAndroidBle.

[jobfeikens]

After digging around a bit, I think I found the problem

mServiceData in ScanRecord.java is not marked with a @Nullable annotation. Yet the field is initialized with a null pointer when the scan record is not parseable.

Next, we read serviceData here:

flutter_reactive_ble/packages/reactive_ble_mobile/android/src/main/kotlin/com/signify/hue/flutterreactiveble/ble/ReactiveBleClient.kt

Line 87 in 8d66469

result.scanRecord.serviceData.mapKeys { it.key.uuid },

Here, kotlin requires a non-null value.