Waze data is not up to date

[Tomkernow]

Hi,

Waze mobility data has not been updated for 10 days.

How can we fix this issue?

Cheers

[ActiveConclusion]

Hi @Tomkernow!
Thank you for reporting the issue. Probably it's an issue with the Waze scraper https://github.com/ActiveConclusion/waze_mobility_scraper
I wanted to refactor this scraper from a selenium solution to a few simple POST requests, however, I don't have so much time lately :( I will try to fix it as soon as I can.
Contributions are welcome.

[Tomkernow]

Hi!

Thanks for your quick response.
I would love to hlep but do not have the quant skills to be of much use....
Do you think you will have time to fix it by the end of the week? If not, I can ask around for some help.

Best regards

[ActiveConclusion]

If everything in Ukraine will be fine this week and I will be alive (I hope for that 😬), I will try to fix it by the end of the week. I will inform you about the progress here.

[ActiveConclusion]

Manually updated on 15.02.22

[Tomkernow]

I appreciate you taking the time to update the values! Big help!
I hope you stay safe and well....

[ActiveConclusion]

Manually updated 18.02.22

[oleh-kulik]

I need more time to investigate how to rework scraper. I will update data manually at the moment.

[ActiveConclusion]

Manually updated 22.02.22 (twosday)

[Tomkernow]

Thanks a lot for updating everything manually! Please continue to do so if possible......
Hope you are staying safe!

[ActiveConclusion]

Thank you, I am holding on and I am sure that we'll win this war. I can update the data during breaks between bombardments. But also I think it will be a good idea to provide instructions/accesses for someone else for updating the data. Volunteers are welcome.

Glory to Ukraine! Glory to Heroes!

[Tomkernow]

Hi. Hope all is okay on your side.... Can you update the data? Instructions for the update are welcome - please post them here when possible - I can give it a go.
Cheers!

[ActiveConclusion]

Thanks, I am fine. Good, I will post instructions as soon as I can. Checked for updates, but it seems that Waze didn't update the data on their source https://datastudio.google.com/reporting/fe8a3c7d-9303-4e70-8acb-4e042714fa76/page/bhuOB

[Tomkernow]

Hi! I have contacted Waze and they have updated their data

[ActiveConclusion]

Hi! Yes, I saw that, I updated the data yesterday.
So I am posting instructions here on how to update the data manually.

1. Open Waze dashboard. https://datastudio.google.com/reporting/fe8a3c7d-9303-4e70-8acb-4e042714fa76/page/bhuOB
2. Select in filter all countries/cities
   
3. Click on three dots -> Export
   
4. Input the name of the file. For country-level data it should be "Waze_Country-Level_Data", for city-level - "Waze_City-Level_Data"
5. Export files.
   If you have enough raw data files, then you are done 😃 But if you want merged and preprocessed data like in this repo, you should do the following steps.
6. Open repo for Waze mobility scraper. https://github.com/ActiveConclusion/waze_mobility_scraper
7. Upload these files. I will give you access to this repo.
   
   
8. The data will be updated here in a few hours. https://github.com/ActiveConclusion/COVID19_mobility/tree/master/waze_reports. Every 3 hours there is a job here that automatically updates the data. https://github.com/ActiveConclusion/COVID19_mobility/actions

If you have any questions, feel free to ask 😃

Thank you!

[ActiveConclusion]

Manually updated 12.03.22

[ActiveConclusion]

Manually updated 21.03.22
P.S. Shopping mall in Kyiv before and after russian demilitarisation 😡

[Tomkernow]

Damn that's absolutely crazy....

[Tomkernow]

I don't think I have access to the repo in order to update files manually

[ActiveConclusion]

Hi @Tomkernow! I sent you an invite to the repo: https://github.com/ActiveConclusion/waze_mobility_scraper
You should accept it here:

Please let me know if you will have any other issues.
Thank you!

[Tomkernow]

Ah... it has expired. Please send again if possible

[ActiveConclusion]

@Tomkernow Sent again

[ActiveConclusion]

Manually updated 29.03.22

[Tomkernow]

CHEERS - please make sure is is updated again.
If possible, please post instruction on how to write new code to automise this.

[ActiveConclusion]

Implemented a draft version of the new scraper. Testing results.

[ActiveConclusion]

Successfully updated with scraper 11.04. Closing this issue. Will reopen if needed.