Skip to content

Route log parsing tasks by collection size#119

Merged
pitangainnovare merged 2 commits intoscieloorg:mainfrom
pitangainnovare:codex/celery-parse-routing-cleanup
May 4, 2026
Merged

Route log parsing tasks by collection size#119
pitangainnovare merged 2 commits intoscieloorg:mainfrom
pitangainnovare:codex/celery-parse-routing-cleanup

Conversation

@pitangainnovare
Copy link
Copy Markdown
Contributor

Summary

  • Route validation-triggered parse orchestration per collection, using the existing parse_<size> queue mapping.
  • Preserve parse queue routing across parse wave retries and re-executions.
  • Clean up task imports and remove unused legacy core/libs/chkcsv.py.
  • Persist seeded translator_class values for log manager directories and normalize the default to classic.

Validation

  • docker compose -f local.yml run --rm django pytest log_manager/tests.py metrics/tests/test_tasks.py metrics/tests/test_cleanup.py
  • docker compose -f local.yml run --rm django python manage.py makemigrations --check --dry-run log_manager_config

@pitangainnovare pitangainnovare self-assigned this May 4, 2026
@pitangainnovare pitangainnovare added the bug Something isn't working label May 4, 2026
@pitangainnovare pitangainnovare marked this pull request as ready for review May 4, 2026 22:09
@pitangainnovare pitangainnovare merged commit 7968f99 into scieloorg:main May 4, 2026
2 checks passed
@pitangainnovare pitangainnovare deleted the codex/celery-parse-routing-cleanup branch May 4, 2026 22:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Corrigir delegação de tasks para filas Celery ao adotar task única para pipeline inteira

1 participant