Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make TimeDelineatedProviderDataIngester handle date ranges larger than a single day #3534

Open
stacimc opened this issue Dec 15, 2023 · 0 comments
Open

Make TimeDelineatedProviderDataIngester handle date ranges larger than a single day #3534

stacimc opened this issue Dec 15, 2023 · 0 comments
Labels
💻 aspect: code Concerns the software code in the repository ✨ goal: improvement Improvement to an existing user-facing feature 🟩 priority: low Low priority and doesn't need to be rushed 🧱 stack: catalog Related to the catalog and Airflow DAGs

Comments

@stacimc
Copy link
Contributor

stacimc commented Dec 15, 2023

Problem

The TimeDelineatedProviderDataIngester is only used to break down a 24-hour period.

Description

It would be nice if the ingester was able to handle larger date ranges, so it could be used for non-dated DAGs or for doing one-off, longer ingestion periods for DAGs like Flickr. (See context in #3490 comments).
@stacimc stacimc added 🟩 priority: low Low priority and doesn't need to be rushed ✨ goal: improvement Improvement to an existing user-facing feature 💻 aspect: code Concerns the software code in the repository 🧱 stack: catalog Related to the catalog and Airflow DAGs labels Dec 15, 2023
@stacimc stacimc mentioned this issue Dec 15, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
💻 aspect: code Concerns the software code in the repository ✨ goal: improvement Improvement to an existing user-facing feature 🟩 priority: low Low priority and doesn't need to be rushed 🧱 stack: catalog Related to the catalog and Airflow DAGs
Projects
Openverse Backlog
Status: 📋 Backlog
Milestone
No milestone
Development

No branches or pull requests
1 participant
@stacimc