Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run CRS update + CSV as a nightly Celery task #363

Closed
aih opened this issue May 13, 2021 · 1 comment
Closed

Run CRS update + CSV as a nightly Celery task #363

aih opened this issue May 13, 2021 · 1 comment
Assignees

Comments

@aih
Copy link
Collaborator

aih commented May 13, 2021

The CSV should be produced automatically after the CRS scraper is run. The CRS scraper should only get new files (if possible) and the CSV should be updated to include the new records (if possible). It's ok to re-run the CSV script, which takes ~ 5 minutes to run.

@aih
Copy link
Collaborator Author

aih commented May 21, 2021

This is now implemented:
https://github.com/aih/FlatGov/blob/main/server_py/flatgov/flatgov/celery.py#L42

'crs_scraper_daily': {
'task': 'bills.tasks.crs_task',
'schedule': crontab(minute=0, hour=1),
'options': {'queue': 'bill'}
},

@aih aih closed this as completed May 21, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants