Increase FPS

[mcdonasd]

I am using trt_pose on a system with i5 cpu and GTX 1660 Super GPU. I am currently only getting 9 fps for an 1080 x 1920 video. Any ideas for how to improve FPS?

[jaybdub]

Hi mcdonasd,

Thanks for reaching out!

Are you downscaling the image before providing it to the neural network? I would recommend downscaling to 224x224 or 256x256 if you are trying to detect people within a few meters. The output can be re-scaled to match your original image dimensions.

Higher resolutions may work, but will change the effective size of objects you detect. The higher resolution the image, the smaller the size of object you will detect, and slower the model will run. It is best to tune the input shape for your application. For example, a human-computer interaction robot probably only needs 224x224, monitoring pose of a crowd at a distance, something much higher.

Please let me know if this helps, or you run into any issues.

Best,
John

[mcdonasd]

Thank you, I will implement your recommendations and keep you posted. Since I posted, I realized implementing the recommendations in Issue #79 slowed video playback significantly. I am still working thru how to save the keypoints to a dataframe while minimizing the performance impact. I would welcomed ideas / recommendations.

Thank you!

[mcdonasd]

@jaybdub I have confirmed that I am downscaling to 224x224. In this application, we need to only detect the largest person in the image.

I have modified the DrawObjects class to map joints to a common standard I am using as follows:

class DrawObjects(object):

    def __init__(self, topology):
        self.topology = topology

    def __call__(self, image, object_counts, objects, normalized_peaks, SWING_ID, F):
        topology = self.topology
        color = (166, 85, 41)
        FILLED = -1
        AA_LINE = 16
        joint_size = 6

        # project standard joint dataframe
        data=pd.DataFrame({"SwingId": [SWING_ID], "FrameNumber": [F], "X1": [0], "Y1": [0], "X2": [0], "Y2": [0], "X3": [0], "Y3": [0], "X4": [0], "Y4": [0], "X5": [0], "Y5": [0], "X6": [0], "Y6": [0], "X7": [0], "Y7": [0], "X8": [0], "Y8": [0], "X9": [0], "Y9": [0], "X10": [0], "Y10": [0], "X11": [0], "Y11": [
            0], "X12": [0], "Y12": [0], "X13": [0], "Y13": [0], "X14": [0], "Y14": [0], "X15": [0], "Y15": [0], "X16": [0], "Y16": [0], "X17": [0], "Y17": [0], "X18": [0], "Y18": [0], "X19": [0], "Y19": [0], "X20": [0], "Y20": [0], "X21": [0], "Y21": [0], "X22": [0], "Y22": [0], "X23": [0], "Y23": [0], "X24": [0], "Y24": [0], "X25": [0], "Y25": [0]})

        # LUT for mapping to project standard joint references
        joint_map = [17, 20, 18, 21, 19, 14, 13, 15,  12, 16, 11, 4, 3, 5, 2, 6, 1, 9]


        # declare our empty dictionary
        dict_XY= {}

        height= image.shape[0]
        width= image.shape[1]

        K= topology.shape[0]
        count= int(object_counts[0])
        K= topology.shape[0]
        for i in range(count):

            obj= objects[0][i]
            C= obj.shape[0]
            for j in range(C):
                k= int(obj[j])
                if k >= 0:
                    peak= normalized_peaks[0][j][k]
                    x= round(float(peak[1]) * width)
                    y= round(float(peak[0]) * height)
                    # make nose large and open and remove eyes, ears
                    if j == 0:
                        cv2.circle(image, (x, y), 40, color, 2, AA_LINE)
                    if j > 4 and j < 18:
                        cv2.circle(image, (x, y), joint_size,
                                   color, FILLED, AA_LINE)

                    # updated code 
                    x_j=float(peak[1])
                    y_j=float(peak[0])

                    #key=human_pose["keypoints"][j]

                    # assigning column names (X1... and so on)
                    X_var="X" + str(joint_map[j])
                    Y_var="Y" + str(joint_map[j])

                    # add each key, value pair in to the dictionary
                    dict_XY[X_var]=x_j
                    dict_XY[Y_var]=y_j
                    


            for k in range(K):
                c_a=topology[k][2]
                c_b=topology[k][3]

                if c_a > 4 and c_a != 17:
                    if obj[c_a] >= 0 and obj[c_b] >= 0:
                        peak0=normalized_peaks[0][c_a][obj[c_a]]
                        peak1=normalized_peaks[0][c_b][obj[c_b]]
                        x0=round(float(peak0[1]) * width)
                        y0=round(float(peak0[0]) * height)
                        x1=round(float(peak1[1]) * width)
                        y1=round(float(peak1[0]) * height)
                        cv2.line(image, (x0, y0), (x1, y1), color, 2,  AA_LINE)


        df=pd.DataFrame.from_dict([dict_XY], orient = 'columns')
        data.update(df.where(df>0))

        return data, image

[mcdonasd]

Speed was significantly improved when I used a dictionary instead of dataframe to remap the joints.

[tucachmo2202]

Speed was significantly improved when I used a dictionary instead of dataframe to remap the joints.

Hi @mcdonasd,
Could you share your code using dictionary? Tks.