Skip to content

Conversation

tcnichol
Copy link
Contributor

This moves the listeners to an async service so that there is no longer a need for the time.sleep hack after indexing a file to get it to work.

To fix this, I used an async service and added a different type of event - a 'file indexed' event that the message_listener is listening to.

In order to make all this work, I did change the rabbitmq dependency from a blockingChannel to an AbstractChannel. I tested to make sure that files trigger extractors when added/indexed and also, that manual submission of datasets and files still works.

To test this, do the following.

First, comment out in docker-compose.dev.yml the part related to the message_listener.

Second, run the dependencies using docker-dev.sh up as usual.

Third, run the message_listener using pycharm, in debug mode.

Fourth, run the backend in debug mode. Run the front end as usual.

Test the following:

Create a dataset and add a file - it should create an event that is picked up by the message_listener when the file is indexed, and this should trigger the check_feed_listeners and any extractors that should run on the file.

Also, to verify that everything extractor related works as before, manually submit files and datasets to extractors.

@tcnichol tcnichol linked an issue Sep 22, 2025 that may be closed by this pull request
@tcnichol tcnichol requested review from max-zilla and lmarini and removed request for lmarini and max-zilla September 22, 2025 20:26
@lmarini lmarini added this to the v2.1 milestone Oct 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Move check_feed_listeners to async service
2 participants