Very Low FPS

[sushant1727]

When running webcam_demo.py and video_demo.py I am getting very low fps.
The output stream freezes and runs in like 10 seconds of interval.
Webcam output is frozen.

Any possible fix?

Note: I followed the guide and installed the CPU version. Running Ubuntu 16.04 VMWare 8 GB ram

[sushant1727]

I also have this issue where I am trying to implement the webcam_demo.py with tkinter. I am having a error where:
23 detection
Loading weights from weights/yolov3.weights...Done!
Exception in Tkinter callback
Traceback (most recent call last):
File "/usr/lib/python3.5/tkinter/init.py", line 1553, in call
return self.func(*args)
File "webcam_demo.py", line 11, in start_video
show_frame()
File "webcam_demo.py", line 35, in show_frame
img = Image.fromarray(cv2image)
AttributeError: type object 'pydarknet.Image' has no attribute 'fromarray'
Exception in Tkinter callback
Traceback (most recent call last):
File "/usr/lib/python3.5/tkinter/init.py", line 1553, in call
return self.func(*args)
File "webcam_demo.py", line 11, in start_video
show_frame()
File "webcam_demo.py", line 35, in show_frame
img = Image.fromarray(cv2image)
AttributeError: type object 'pydarknet.Image' has no attribute 'fromarray'

Code is:

import time
import tkinter as tk
import settings
from PIL import Image, ImageTk
import imutils
from pydarknet import Detector, Image
import cv2

def start_video():
settings.start_video = True
show_frame()

def stop_video():
settings.start_video = False
settings.start_processing = False
lmain.config(image='')

def start_process():
settings.start_processing = True

def stop_process():
settings.start_processing = False

def show_frame():
if not settings.start_video:
return None

_, frame = cap.read()
frame = imutils.resize(frame, width=400)

if settings.start_processing:
    frame = process_frame(frame)

cv2image = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
img = Image.fromarray(cv2image)
imgtk = ImageTk.PhotoImage(image=cv2image)
lmain.imgtk = imgtk
lmain.configure(image=imgtk)
lmain.after(10, show_frame)

def process_frame(img):
# grab the frame dimensions and convert it to a blob
(h, w) = img.shape[:2]
blob = cv2.dnn.blobFromImage(cv2.resize(img, (300, 300)),
0.007843, (300, 300), 127.5)

# pass the blob through the network and obtain the detections and
# predictions
#net.setInput(blob)
#detections = net.forward()

start_time = time.time()

# Only measure the time taken by YOLO and API Call overhead

dark_frame = Image(img)
results = net.detect(dark_frame)
del dark_frame

end_time = time.time()
print("Elapsed Time:",end_time-start_time)

for cat, score, bounds in results:
    x, y, w, h = bounds
    cv2.rectangle(img, (int(x-w/2),int(y-h/2)),(int(x+w/2),int(y+h/2)),(255,0,0))
    cv2.putText(img, str(cat.decode("utf-8")), (int(x), int(y)), cv2.FONT_HERSHEY_COMPLEX, 1, (255, 255, 0))

#cv2.imshow("preview", frame)

load our serialized model from disk

print("Loading model...")
net = Detector(bytes("cfg/yolov3.cfg", encoding="utf-8"), bytes("weights/yolov3.weights", encoding="utf-8"), 0,
bytes("cfg/coco.data", encoding="utf-8"))
cap = cv2.VideoCapture(0)

window = tk.Tk()
window.title("DEMO")
window.geometry('700x420')
lbl = tk.Label(window, text="Smart Machine Vision System", font=("Arial Bold", 24))
lbl.grid(column=1, row=0)
imageFrame = tk.Frame(window, width=600, height=500)
imageFrame.grid(row=1, column=1, padx=10, pady=2)
lmain = tk.Label(imageFrame, text="Press Start Video")
lmain.grid(row=1, column=1)
startVideoStreamBtn = tk.Button(window, text="Start Video", command=start_video)
startVideoStreamBtn.grid(column=0, row=2, padx=15)
stopVideoStreamBtn = tk.Button(window, text="Stop Video", command=stop_video)
stopVideoStreamBtn.grid(column=0, row=3, padx=15)
startProcessBtn = tk.Button(window, text="Start Detection", command=start_process)
startProcessBtn.grid(column=1, row=2)
stopProcessBtn = tk.Button(window, text="Stop Detection", command=stop_process)
stopProcessBtn.grid(column=1, row=3)
window.mainloop()

[madhawav]

@sushant1727 Regarding your first post, you should use the GPU version to obtain good frame rates. Can get 10+ FPS if you use a good CUDA enabled GPU. However I am not sure how you can setup GPU with VMWare.

@sushant1727 Regarding your second post, I will go through it in detail and get back to you.

[sushant1727]

Thanks @madhawav I will run it with GPU enabled and report back the performance.

To add more for the second post:
File "webcam_demo.py", line 35, in show_frame
img = Image.fromarray(cv2image)
AttributeError: type object 'pydarknet.Image' has no attribute 'fromarray'

The error pydarknet.Image' has no attribute 'fromarray. I am using Image.fromarray(which is defined in PIL(pillow)) function and I guess it is interferring with Image function in pyarknet. That's my guess.

I've this code in pastebin for better view. https://pastebin.com/2R36DWux

[madhawav]

Hi,
In your code, your expecting a response from function process_frame but it doesn't actually return a value.

In line 32 you have

  if settings.start_processing:
      frame = process_frame(frame)

but the function process_frame has no "return" keyword.

Therefore, the above code makes the value of frame None (In python, if there is no return keyword, None is returned by the function). This None is passed as input to cv2image = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB) which would make value of cv2image invalid. This is probably causing the error you have mentioned in line 35.

Can you add the return statement to process_frame function and let me know whether it corrects the issue?

[sushant1727]

Thanks for pointing out the mistake.

I added return(img), I still get the error.

'Loading weights from weights/yolov3.weights...Done!'
Exception in Tkinter callback
Traceback (most recent call last):
File "/usr/lib/python3.5/tkinter/init.py", line 1553, in call
return self.func(*args)
File "webcam_demo.py", line 11, in start_video
show_frame()
File "webcam_demo.py", line 35, in show_frame
img = Image.fromarray(cv2image)
AttributeError: type object 'pydarknet.Image' has no attribute 'fromarray'

Note:- I think in the imports Image is getting imported twice, that might be causing the problem.

I tried using this imports
import PIL.Image
from PIL import ImageTk
from pydarknet import Detector, Image

Now, I am getting this error.
TypeError: unhashable type: 'numpy.ndarray'
Exception ignored in: <bound method PhotoImage.del of <PIL.ImageTk.PhotoImage object at 0x7f7ea3fa0e80>>
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/PIL/ImageTk.py", line 118, in del
name = self.__photo.name
AttributeError: 'PhotoImage' object has no attribute '_PhotoImage__photo'

[madhawav]

Hi,
I am assuming you replaced the line Image.fromarray(cv2image) with PIL.Image.fromarray(cv2image). (In addition to adding import PIL.Image as you have mentioned.) This move should solve the first error you have mentioned in the above comment.

The second error is related with numpy and ImageTk.
In the line imgtk = ImageTk.PhotoImage(image=cv2image), you are passing a numpy array (cv2image) as input to ImageTk.PhotoImage. But the source code of PIL.ImageTk mentions that it require you to provide a PIL image instead of a Numpy Array.

For example, this is what PIL.ImageTk.py mentions for init() of PhotoImage.

class PhotoImage(object):
    """
    .....
    :param image: Either a PIL image, or a mode string.  If a mode string is
                  used, a size must also be given.

So basically, you will have to convert the numpy array to a PIL Image and then pass it to ImageTk.PhotoImage().
So, can you replace the line imgtk = ImageTk.PhotoImage(image=cv2image) with imgtk = ImageTk.PhotoImage(image=PIL.Image.fromarray(cv2image)) ? This would convert the numpy array to a PIL Image and it would be passed into the method. So it should solve the issue.

[sushant1727]

Thanks @madhawav , you were a great help and I am able to run it as expected. Appreciate your effort.
To add more while using CPU only version, I am able to get more FPS while using custom trained Mobilenet-SSD, Faster-RCNN and Caffe Pre-Trained model which is around 15.

[madhawav]

@sushant1727 Will close this issue since the problem is solved.
And thank you for the info!
Cheers!