MongoDB CPU usage increasing over time

[vsaw]

In situations where ODC is running for a long period of time, the CPU usage from MongoDB increases over time.

In my setup, within 10 minutes, the CPU usage of Mongo doubled from 5% to 10%. I've had "longer term" scenarios where Mongo was running at 100% CPU power.

I believe it's coming from the way we store the counter history in the recording which requires an update of the document. I will try to run some more benchmarks and see if my suspicion is correct. If so the fix would be to move the counter history into its own collection, which will allow us to simply add new entries to the collection. This should be a quite cheap and quick operation and fix the CPU usage.

Loading of recordings from the DB will be a bit more expensive, but I don't think this will be a problem in the end.

P.S. By the time I finished writing this, CPU usage has increased by another 1%.

Update 1
P.P.S. CPU usage just jumped to 95% for this recording

{
  "_id": "6453b7ddc8bb1f933bcd5b0e",
  "dateStart": "2023-05-04T13:49:17.694Z",
  "dateEnd": "2023-05-04T14:25:12.121Z",
  "videoResolution": {
    "w": 1280,
    "h": 720
  },
  "filename": "",
  "id": "6fe6d39d-64d3-45e1-815f-15494d281c23",
  "counterSummary": {
    "07b765a7-f05c-47b7-988b-7be06cbe3e53": {
      "_total": 11189,
      "car": 11189
    }
  },
  "trackerSummary": {
    "totalItemsTracked": 28887
  }
}

Update 2
I've ran two quick benchmarks.

1. Disable Counter History + Disable Tracker History: CPU usage stayed < 5%, often as low as 3%
2. Disable Counter History + Enable Tracker History: CPU usage < 6%. This is relevant because the tracker has it's own collection that I also want to do for the CounterHistory.

[vsaw]

I managed to keep MongoDB on my machine below 6% now with this new change 🎉