Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add multiple queues for MongoDB #11

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

vkost
Copy link
Contributor

@vkost vkost commented Jan 27, 2020

I didn't like creating multiple tables for multiple queues so I added queue_name parameter, similar to the one in default symfony/messenger DBAL transport. Lets you store multiple queues in one MongoDB table - very simple.

…ike DBAL to help sort messages with identical published_at time, so if you push a lot of messages in one second, the order of the execution can not be guaranteed. This way it can.
@arendjantetteroo
Copy link
Contributor

Interesting. Can you check the unit test failure?

@alekitto
Copy link
Owner

alekitto commented Feb 2, 2020

Honestly, I don't really like it. Multiple queues was left on separate collections (and tables for dbal) on purpose: having multiple queues on the same table/collection implies the creation of a new index and, in case of mongo, a very inefficient queue data retrieval.

For example: i'm using a count operation to collect metrics about the number of messages in the queues of my system. In this context, a document count of the whole collection is optimized by mongo, but a filtered count could be very very slow (mongo will perform the query and than advances the cursor to the last element couting documents one-by-one).

In any case, adding some user-defined data and let the user to check that data while querying the queue is a good idea, but I think should be done in another way.
This should allow any kind of data to be added/queried at will. What do you think about it?

@alekitto alekitto force-pushed the master branch 5 times, most recently from 74a8d7b to 2bbe009 Compare April 17, 2024 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants