jtop causes high CPU usage on Jetson Nano after upgrade to 3.0.0

[SteffenBauer]

Describe the bug
Since I upgraded to jetson_stats version 3.0.0, running jtop shows constant CPU utilization of > 50% in all active cores.
Furthermore, shown total power consumption in jtop is now around 2000 mW, before it was at 940 mW shown in jtop when the board was idling.

To Reproduce

1. Open terminal
2. Run jtop. Observe significant CPU utilization
3. Quit jtop. Run tegrastats. CPU utilization is now back at only a few percent.

Screenshots

NVIDIA Jetson Nano (Developer Kit Version) - Jetpack 4.4 [L4T 32.4.3]
CPU1 [||||||||||||||||||||||||||||||||||||||||||||Schedutil -  80%] 921MHz
CPU2 [||||||||||||||||||||||||||||||||||||||||||||Schedutil -  84%] 921MHz
CPU3 [  OFF                                                              ]
CPU4 [  OFF                                                              ]

Mem [|||||||||||                                                                                                             0.4G/4.1GB] (lfb 857x4MB)
Imm [                                                                                                                        0.0k/252.0kB] (lfb 252kB)
Swp [                                                                                                                        0.0GB/2.0GB] (cached 0MB)
EMC [|                                                                                                                                      1%] 1.6GHz

GPU [                                                                                                                                       0%] 76 MHz
Dsk [##########################################################################                                                         49.3GB/94.4GB]
┌───────────────────── [info] ──────────────────────┬──────────── [Sensor] ── [Temp] ─────────────┬──────────── [Power/mW] ─ [Cur]  [Avr] ────────────┐
│UpT: 0 days 0:4:49                                 │             AO          35.00C              │             5V CPU       551    551               │
│FAN [|||||||||                         25%] Ta= 25%│             CPU         29.00C              │             5V GPU       42     42                │
│Jetson Clocks: inactive                            │             GPU         27.00C              │             ALL          2079   2079              │
│NV Power[1]: 5W                                    │             PLL         25.00C              │                                                   │
│────────────────── [HW engines] ───────────────────│             iwlwifi     37.00C              │                                                   │
│APE: 25MHz                                         │             thermal     28.50C              │                                                   │
│NVENC: [OFF]              NVDEC: [OFF]             │                                             │                                                   │
│NVJPG: [OFF]                                       │                                             │                                                                

On parallel terminal output of tegrastats:

RAM 246/3964MB (lfb 857x4MB) SWAP 0/1982MB (cached 0MB) CPU [54%@921,55%@921,off,off] EMC_FREQ 0% GR3D_FREQ 0% PLL@25C CPU@29C iwlwifi@33C PMIC@100C GPU@27.5C AO@35C thermal@28C POM_5V_IN 2082/2111 POM_5V_GPU 0/37 POM_5V_CPU 552/556
RAM 246/3964MB (lfb 857x4MB) SWAP 0/1982MB (cached 0MB) CPU [58%@921,52%@921,off,off] EMC_FREQ 0% GR3D_FREQ 0% PLL@25C CPU@29C iwlwifi@33C PMIC@100C GPU@27.5C AO@35.5C thermal@28.5C POM_5V_IN 2000/2101 POM_5V_GPU 0/34 POM_5V_CPU 552/555

[ ... here I quit jtop, CPU utilization and power consumption immediately goes down: ]

RAM 245/3964MB (lfb 857x4MB) SWAP 0/1982MB (cached 0MB) CPU [39%@102,37%@102,off,off] EMC_FREQ 0% GR3D_FREQ 0% PLL@24.5C CPU@28.5C iwlwifi@33C PMIC@100C GPU@27.5C AO@35C thermal@28.5C POM_5V_IN 986/2008 POM_5V_GPU 0/31 POM_5V_CPU 128/520
RAM 245/3964MB (lfb 857x4MB) SWAP 0/1982MB (cached 0MB) CPU [2%@102,0%@102,off,off] EMC_FREQ 0% GR3D_FREQ 0% PLL@24.5C CPU@28.5C iwlwifi@33C PMIC@100C GPU@27.5C AO@35C thermal@28C POM_5V_IN 943/1926 POM_5V_GPU 0/29 POM_5V_CPU 128/490
RAM 245/3964MB (lfb 857x4MB) SWAP 0/1982MB (cached 0MB) CPU [4%@102,1%@102,off,off] EMC_FREQ 0% GR3D_FREQ 0% PLL@25C CPU@28.5C iwlwifi@33C PMIC@100C GPU@27.5C AO@35C thermal@28C POM_5V_IN 944/1856 POM_5V_GPU 0/27 POM_5V_CPU 128/464
RAM 245/3964MB (lfb 857x4MB) SWAP 0/1982MB (cached 0MB) CPU [1%@102,0%@102,off,off] EMC_FREQ 0% GR3D_FREQ 0% PLL@24.5C CPU@28.5C iwlwifi@33C PMIC@100C GPU@27.5C AO@35C thermal@28C POM_5V_IN 944/1795 POM_5V_GPU 0/25 POM_5V_CPU 128/441

Expected behavior
Very low CPU on an idle board running only jtop

Additional context

I'm running the Jetson nano board in headless mode and without X-Windows running, to reduce processor and memory load as much as possible. Above examples were recorded logged in via ssh.

Bug occurs in both nvpmodel modes. In mode 0, all four cores are at ~ 30% during running jtop. Bug also regardless of running under root or user.

Output of jetson_release -v:

 - NVIDIA Jetson Nano (Developer Kit Version)
   * Jetpack 4.4 [L4T 32.4.3]
   * NV Power Mode: 5W - Type: 1
   * jetson_stats.service: active
 - Board info:
   * Type: Nano (Developer Kit Version)
   * SOC Family: tegra210 - ID:33
   * Module: P3448-0000 - Board: P3449-0000
   * Code Name: porg
   * Boardids: 3448
   * CUDA GPU architecture (ARCH_BIN): 5.3
   * Serial Number: 1421919084211
 - Libraries:
   * CUDA: 10.2.89
   * cuDNN: 8.0.0.180
   * TensorRT: 7.1.3.0
   * Visionworks: 1.6.0.501
   * OpenCV: 4.1.1 compiled CUDA: NO
   * VPI: 0.3.7
   * Vulkan: 1.2.70
 - jetson-stats:
   * Version 3.0.0
   * Works on Python 3.6.9

As CPU utilization is at ~50% on two cores, and ~ 30% on four cores, it looks like the program loop is constantly running and not sleeping correctly during idle.

[rbonghi]

Hi @SteffenBauer ,

Thank you for your issue, clear and complete!
I caught this bug :-)
There is a thread that not sleep (for Nano and Xavier NX), but read without stop "jetson_clocks --show" I will release a new package before the end of this week to fix this issue.

[rbonghi]

I'm testing on a jetson xavier nx with 2 cores without jetson_clocks

Right now I cannot decrease the speed lower than an average of 18%, because I don't have enough experience with python profilers.
I will keep you posted when I will improve the performance of jtop, anyway is in plan to don't use anymore tegrastats and jetson_clocks. Only running these sub processes I lose a lot of performances.

I'm releasing now the new version 3.0.1

With jtop

Without jtop