You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
VMois
changed the title
Add partitioning to raw realtime vehicle position based on timestamp of the event
Add partitioning and partition expiry to raw realtime vehicle position based on timestamp of the event
Apr 21, 2024
…om GCS to BQ
- Use DATE from timestamp for partitioning raw vehicle positions tables, set partition expiry to 30 days;
- Put merge query from Python to standalone SQL file for better visibility.
closes#12#14
…om GCS to BQ
- Use DATE from timestamp for partitioning raw vehicle positions tables, set partition expiry to 30 days;
- Put merge query from Python to standalone SQL file for better visibility.
closes#12#14
As discussed in https://vmois.dev/data-flow-bigquery-miwaitway/, vehicle location data must be partitioned and set to expire after 30 days.
The text was updated successfully, but these errors were encountered: