Skip to content

Insertion of messages in capped collection is slow #280

@odesenfans

Description

@odesenfans

Running some benchmarks on DB insertion time, I noticed that the insertion time in the CappedMessage collection is two magnitudes of order slower than insertions in the Message collection.

2022-05-19 11:59:33 [INFO] root: Wrote 880 documents to collection Message in 0.5942 seconds
2022-05-19 11:59:33 [INFO] root: Wrote 880 documents to collection PendingMessage in 0.1174 seconds
2022-05-19 11:59:41 [INFO] root: Wrote 880 documents to CappedMessage in 7.9019 seconds

This slows down the message processing job quite a bit. Given that capped collections have other painful limitations (ex: no updates that increase the size, e.g. for confirmations), we should investigate whether this capped collection can be dropped in favor of a better solution for the websocket.

An aggregate could be a good solution. Ex: aggregate([{$sort: {"reception_time": -1}}, {$limit: 5000}, {$match: <query-filters>}]) using the future reception_time field introduced in #267.

Metadata

Metadata

Assignees

No one assigned

    Labels

    performance issueFixing this issue would improve the performance of the application

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions