Crash when MQTT triggers

[charredchar]

I am really hoping this gets a little help... I updated the code to work with Python 3.8 and TensorFlow 2, I need to run within Windows to utilize my GPU so I am really trying to get this to work. So far so good, but when ever I trigger a camera:

C:\Users\char\Desktop\BIObjectDetect>python bi_alert_detect.py

======================================================
Person Detection system started
Modelling System: TF
Importing TensorFlow (may take a few seconds)
2020-10-19 22:20:58.459332: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_110.dll
Loading model from folder 'TF' (may take some time)...
2020-10-19 22:21:09.893813: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library nvcuda.dll
2020-10-19 22:21:09.957619: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:06:00.0 name: GeForce GTX 1650 computeCapability: 7.5
coreClock: 1.68GHz coreCount: 14 deviceMemorySize: 4.00GiB deviceMemoryBandwidth: 119.24GiB/s
2020-10-19 22:21:09.979684: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_110.dll
2020-10-19 22:21:10.012133: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cublas64_11.dll
2020-10-19 22:21:10.027788: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cufft64_10.dll
2020-10-19 22:21:10.040146: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library curand64_10.dll
2020-10-19 22:21:10.057489: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusolver64_10.dll
2020-10-19 22:21:10.072963: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusparse64_11.dll
2020-10-19 22:21:10.084799: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudnn64_8.dll
2020-10-19 22:21:10.095608: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-10-19 22:21:10.123131: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x254f02de460 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-10-19 22:21:10.138133: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-10-19 22:21:10.149691: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:06:00.0 name: GeForce GTX 1650 computeCapability: 7.5
coreClock: 1.68GHz coreCount: 14 deviceMemorySize: 4.00GiB deviceMemoryBandwidth: 119.24GiB/s
2020-10-19 22:21:10.179859: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_110.dll
2020-10-19 22:21:10.189596: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cublas64_11.dll
2020-10-19 22:21:10.202010: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cufft64_10.dll
2020-10-19 22:21:10.216694: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library curand64_10.dll
2020-10-19 22:21:10.227494: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusolver64_10.dll
2020-10-19 22:21:10.247724: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusparse64_11.dll
2020-10-19 22:21:10.257995: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudnn64_8.dll
2020-10-19 22:21:10.276563: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-10-19 22:22:58.273604: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-10-19 22:22:58.285055: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263]      0
2020-10-19 22:22:58.293115: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0:   N
2020-10-19 22:22:58.306141: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2909 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1650, pci bus id: 0000:06:00.0, compute capability: 7.5)
2020-10-19 22:22:58.334152: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x255390c1e30 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-10-19 22:22:58.349294: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce GTX 1650, Compute Capability 7.5
Model initalised. Connecting to mqtt server 192.168.5.158
Subscribing to mqtt topic 'BlueIris/alert'
MQTT connection established with broker

2020-10-19 22:25:23 'b'livingroom-hdt'' camera alert triggered
[DEBUG] Calling image processing function...
[DEBUG] 'do_detect' function called for camera 'b'livingroom-hdt''
[DEBUG] Camera 'b'livingroom-hdt'' not present in existing collection. Instantinating new instance
[DEBUG] Search objects for camera 'b'livingroom-hdt'': Person
Traceback (most recent call last):
  File "bi_alert_detect.py", line 835, in <module>
    mqtt_client.loop_forever()
  File "C:\Users\char\AppData\Local\Programs\Python\Python38\lib\site-packages\paho\mqtt\client.py", line 1779, in loop_forever
    rc = self.loop(timeout, max_packets)
  File "C:\Users\char\AppData\Local\Programs\Python\Python38\lib\site-packages\paho\mqtt\client.py", line 1181, in loop
    rc = self.loop_read(max_packets)
  File "C:\Users\char\AppData\Local\Programs\Python\Python38\lib\site-packages\paho\mqtt\client.py", line 1572, in loop_read
    rc = self._packet_read()
  File "C:\Users\char\AppData\Local\Programs\Python\Python38\lib\site-packages\paho\mqtt\client.py", line 2310, in _packet_read
    rc = self._packet_handle()
  File "C:\Users\char\AppData\Local\Programs\Python\Python38\lib\site-packages\paho\mqtt\client.py", line 2936, in _packet_handle
    return self._handle_publish()
  File "C:\Users\char\AppData\Local\Programs\Python\Python38\lib\site-packages\paho\mqtt\client.py", line 3216, in _handle_publish
    self._handle_on_message(message)
  File "C:\Users\char\AppData\Local\Programs\Python\Python38\lib\site-packages\paho\mqtt\client.py", line 3444, in _handle_on_message
    self.on_message(self, self._userdata, message)
  File "bi_alert_detect.py", line 738, in mqtt_on_message
    detected = ai_detector.do_detect(cam_name)
  File "bi_alert_detect.py", line 395, in do_detect
    self.cameras[cam_name] = Camera(
  File "bi_alert_detect.py", line 209, in __init__
    self.url = self.get_url()
  File "bi_alert_detect.py", line 240, in get_url
    self.url = "".join((self.url, "/image/", self.name))
TypeError: sequence item 2: expected str instance, bytes found

C:\Users\char\Desktop\BIObjectDetect>

[smar000]

As you've modified the code, it is difficult to help you as I don't know what you've done.

However, from the error thrown, it seems as if the issue may be due to the difference between Python 2 and 3 in the way it treats the non-string join. I would check the python 3 documentation for join. It may be just a case of making the empty string a byte, e.g.

self.url = b"".join((self.url, "/image/", self.name))

(note that the above has not been tested)

[charredchar]

As you've modified the code, it is difficult to help you as I don't know what you've done.

You are totally right, I wrote that up at 1am totally not expecting an answer. lol But thank you so much for replying! From this point I decided to go ahead and make a fork so I can share the modifications. Honestly though, I Do Not know Python, I just ran it and searched for each error along the way to get it working with the newer versions.

However, from the error thrown, it seems as if the issue may be due to the difference between Python 2 and 3 in the way it treats the non-string join. I would check the python 3 documentation for join. It may be just a case of making the empty string a byte, e.g.

I did come across this as the likely cause when searching for the error it returned (which seems kind of obvious when just reading the error) but sadly, because I am lacking severely in Python I could not manage to get past it after a couple of hours attempting different examples. I should have also noted that last night I changed it to the following and it kind of worked, it let me continue to find other code that needed modifying.

self.url = "".join((self.url, "/image/livingroom-hdt"))
(obviously this only works for the one camera but just testing)

It seems self.name keeps returning binary instead of a string. This is where I kept getting caught up as I don't know how to correct this. The example code you gave did not resolve this as well.

After attempting a run in Python 2.7 (which has its own issues, still) I did notice that it doesn't return the camera name the same in 3.8 versions.

DEBUG] Camera 'b'livingroom-hdt''

There is one more issue I came across after forcing the self.name, same issue that happens with Python 2.7 and I believe might have more to do with MQTT but I'm not 100% sure. During a trigger I receive the following:

======================================================
Person Detection system started
Modelling System: TF
Importing TensorFlow (may take a few seconds)
2020-10-20 18:22:07.231758: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_110.dll
Loading model from folder 'TF' (may take some time)...
2020-10-20 18:22:18.693223: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library nvcuda.dll
2020-10-20 18:22:18.758844: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:06:00.0 name: GeForce GTX 1650 computeCapability: 7.5
coreClock: 1.68GHz coreCount: 14 deviceMemorySize: 4.00GiB deviceMemoryBandwidth: 119.24GiB/s
2020-10-20 18:22:18.779084: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_110.dll
2020-10-20 18:22:18.797761: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cublas64_11.dll
2020-10-20 18:22:18.813015: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cufft64_10.dll
2020-10-20 18:22:18.825380: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library curand64_10.dll
2020-10-20 18:22:18.848300: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusolver64_10.dll
2020-10-20 18:22:18.864687: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusparse64_11.dll
2020-10-20 18:22:18.876301: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudnn64_8.dll
2020-10-20 18:22:18.887318: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-10-20 18:22:18.912604: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1bc5d1827d0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-10-20 18:22:18.927763: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-10-20 18:22:18.936531: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:06:00.0 name: GeForce GTX 1650 computeCapability: 7.5
coreClock: 1.68GHz coreCount: 14 deviceMemorySize: 4.00GiB deviceMemoryBandwidth: 119.24GiB/s
2020-10-20 18:22:18.954546: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_110.dll
2020-10-20 18:22:18.963924: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cublas64_11.dll
2020-10-20 18:22:18.974022: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cufft64_10.dll
2020-10-20 18:22:18.984056: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library curand64_10.dll
2020-10-20 18:22:18.995234: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusolver64_10.dll
2020-10-20 18:22:19.006508: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusparse64_11.dll
2020-10-20 18:22:19.017077: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudnn64_8.dll
2020-10-20 18:22:19.027021: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2020-10-20 18:23:38.156601: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-10-20 18:23:38.167920: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263]      0
2020-10-20 18:23:38.173339: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0:   N
2020-10-20 18:23:38.181015: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2909 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1650, pci bus id: 0000:06:00.0, compute capability: 7.5)
2020-10-20 18:23:38.205837: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x1bca0dc4630 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-10-20 18:23:38.221707: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce GTX 1650, Compute Capability 7.5
Model initalised. Connecting to mqtt server 192.168.5.158
Subscribing to mqtt topic 'BlueIris/alert'
MQTT connection established with broker

2020-10-20 18:23:52 'b'livingroom-hdt'' camera alert triggered
[DEBUG] Calling image processing function...
[DEBUG] 'do_detect' function called for camera 'b'livingroom-hdt''
[DEBUG] Camera 'b'livingroom-hdt'' not present in existing collection. Instantinating new instance
[DEBUG] Search objects for camera 'b'livingroom-hdt'': Person
[DEBUG] New camera 'b'livingroom-hdt'' added to dictionary
[DEBUG] Downloadinge image from BlueIris
[DEBUG] Getting image from BlueIris url: http://192.168.5.156:81/image/livingroom-hdt?user=user&pw=password&q=100&s=100
[DEBUG] [ImageFrame.get_image_from_camera] Image width: 960, height: 720
[INFO ] Image downloaded from BI (1.13 seconds) [960 x 720]
[DEBUG] Calling 'process_tensorflow'
[DEBUG] TensorFlow: Classifying frame contents from camera 'b'livingroom-hdt'' (may take some time depending on CPU/GPU)
2020-10-20 18:24:02.153384: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cublas64_11.dll
2020-10-20 18:24:03.207944: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudnn64_8.dll
2020-10-20 18:24:05.259621: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: Invoking GPU asm compilation is supported on Cuda non-Windows platforms only
Relying on driver to perform ptx compilation.
Modify $PATH to customize ptxas location.
This message will be only logged once.
2020-10-20 18:24:07.885390: W tensorflow/core/common_runtime/bfc_allocator.cc:246] Allocator (GPU_0_bfc) ran out of memory trying to allocate 3.51GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
2020-10-20 18:24:08.192920: W tensorflow/core/common_runtime/bfc_allocator.cc:246] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.93GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
[DEBUG] Frame content classification took 19.4983 seconds
[DEBUG] Reviewing classfications for detected objects (7)
[DEBUG] Completed TF processing. Returning to calling function
[DEBUG] Returned from detection API function. Processing identified objects
[INFO ] Analysis complete. No motion detected (20.64 seconds overall)
Unexpected disconnection.
[DEBUG] mqtt rc: 1, userdata: None, client: <paho.mqtt.client.Client object at 0x000001BC4C8EEB80>
MQTT connection established with broker

Near the end, the Unexpected Disconnect. I can not find what is causing it as it doesn't give me enough information. I also have to stop the program and restart it to have it accept new triggers. Scaling doesn't seem to work either... When entering the URL into a browser it seems the scaling is a percentage now but even 100 doesn't match the resolution TensorFlow receives and neither are what the camera produces. I just now enabled saving the original and processed images, noticed I only end up with the original.

I'll do some reading through the Python docs to see if I can figure out the binary thing, the rest is going to end up stopping me eventually as well... Again, I can't thank you enough for helping on this, your program is my best hope at getting TensorFlow to work on Windows with Blue Iris.

[smar000]

Sorry, I don't think I can help much further, as it could be any number of issues, especially with your migration to python 3.

However, I will upload shortly my internal python3 version, which may save you some effort. Good luck!

[charredchar]

I think I figured out what could be causing it, I'll just need to work on it some. Again thank you for helping out.

I am excited to try out your version, thanks!