Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ASR and NLP jobs not running when adding new source #74

Open
datadonK23 opened this issue Dec 9, 2021 · 1 comment
Open

ASR and NLP jobs not running when adding new source #74

datadonK23 opened this issue Dec 9, 2021 · 1 comment

Comments

@datadonK23
Copy link
Contributor

Describe the bug
Using the current state of main, ASR and NLP worker jobs are not getting triggered when importing new source (e.g. served devset feed), although sliders in Importer are set correctly.

To Reproduce
OAS started. Worker started (using poetry) and is running correctly (single jobs are possible). When importing new source into Importer and set the sliders to trigger ASR and NLP, no job gets into worker.

Expected behavior
Worker jobs should get triggered to compute ASR and NLP jobs.

Additional context
Worked correctly using old job queue.

@Frando
Copy link
Member

Frando commented Dec 10, 2021

I tried to reproduce and it seems to work OK for me. Please try the following and see if it works for you:

# Start OAS services with an empty data dir to have a clean instance to start with
OAS_DATA_DIR=/tmp/oas_test docker-compose -f docker-compose.dev.yml up
# Start the server
RUST_LOG=debug cargo run -- run
# Import a feed with curl or another HTTP client
# I uploaded the current devset feed to make this easier, 
# replace the URL with a local URL when serving the devset feed locally)
curl -X POST -v -u admin:password http://localhost:8080/api/v1/feed -d '{"url":"https://dl.arso.xyz/oas/devset-cba-0.1.rss", "mediaJobs":{"asr":{}},"postJobs":{"nlp":{}}}'
# Check if jobs were created
curl -u admin:password http://localhost:8080/api/v1/jobs
# this should be a big array of jobs, with one NLP job per post and one ASR job per media
# Start a worker
cd oas_worker && poetry run python worker.py
# This should now start both ASR and NLP processing. New NLP jobs should be created after the ASR finishes also.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants