Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add population values to geo #23

Closed
jeanpaulrsoucy opened this issue May 2, 2022 · 14 comments
Closed

Add population values to geo #23

jeanpaulrsoucy opened this issue May 2, 2022 · 14 comments
Labels
data New data sources or updates to existing data sources. enhancement New feature or request

Comments

@jeanpaulrsoucy
Copy link
Member

No description provided.

@jeanpaulrsoucy jeanpaulrsoucy added enhancement New feature or request timeline data New data sources or updates to existing data sources. labels May 2, 2022
@jeanpaulrsoucy jeanpaulrsoucy mentioned this issue May 2, 2022
Closed
@apiology
Copy link

Hi @jeanpaulrsoucy! The API change disabled data updates at microCOVID.org and I'm trying to (unofficially) help out getting the new format ingested.

It looks like microCOVID relied on population data in order to calculate prevalence by health region.

If you happen to be aware of another upstream source for population, that'd be helpful in the interim.

@jeanpaulrsoucy
Copy link
Member Author

Hi @apiology, thanks for letting me know. What a cool project! At minimum, I can add population data for the provinces to pt.csv later today.

As for dealing with the new data format, see the note in the README giving links to case and death CSV files in the legacy format (i.e., the same format used for Covid19Canada).

@apiology
Copy link

@jeanpaulrsoucy: Appreciate it - we have province-level population and timeseries data for Canada already, so I think we're really looking for things at the health-region level. Where were you gathering health-region population from previously?

I did see that note, thank you--the legacy format doesn't contain the health-region-level population, so I don't think it would help us with this. I'll keep it in mind as I get further into this problem, though, perhaps there's another application.

@jeanpaulrsoucy
Copy link
Member Author

Ah okay, in that case I can add the HR-level population data too. I didn't see the HR data being used in my quick scan of the site.

@apiology
Copy link

It's not currently - it had to be yanked because of this issue. It'll be added back once we can get your data ingested again.

@apiology
Copy link

Ah, it looks like I can grab this from the legacy repo at https://github.com/ccodwg/Covid19Canada/blob/master/other/hr_map.csv in the interim (the existing code was using https://api.opencovid.ca/other?stat=hr).

jeanpaulrsoucy added a commit that referenced this issue May 22, 2022
@jeanpaulrsoucy
Add population data for PTs and HRs #23
ce08b8f 
- Quote columns of pt.csv and health_regions.csv to be consistent with data files
@jeanpaulrsoucy
Copy link
Member Author

Hi @apiology, see geo/pt.csv and geo/health_regions.csv for the latest population data. In particular, use the pop column for the latest population data available for the geography. Data notes are here.

Let me know if you have any questions or issues!

@apiology
Copy link

Thanks so much! I've gotten a lot farther in other areas of the transition in the meantime. I'll give it a shot and give you a shout if there are any problems!

@apiology
Copy link

apiology commented May 28, 2022 •
edited

@jeanpaulrsoucy: I am seeing a potential issue while doing QC on microCOVID/microCOVID#1452

When microCOVID.org fetches a vaccination report for health region 595 (Vancouver Coastal, BC) from api.covid19tracker.ca here, I see it claim that there are 1.1million people vaccinated (see total_vaccinated in the data).

However, your data claims a total population much less than that here - 300,970.

Which do you think is correct/incorrect?

@jeanpaulrsoucy
Copy link
Member Author

Hi @apiology, thanks for catching this! The labels on some of the BC health regions got switched. This has been fixed in health_regions.csv, health_regions.geojson and all of the data files. #55

@apiology
Copy link

apiology commented May 29, 2022 •
edited

@jeanpaulrsoucy - thanks for the fast turnaround! However, the population for hruid 595 (now labeled Northern Health), is still out of line with the total_vaccinated data I'm seeing from api.covid19tracker.ca here, which is fetched via the hruid 595.

Thoughts? I see the same issue for hruid 592.

@jeanpaulrsoucy
Copy link
Member Author

jeanpaulrsoucy commented May 29, 2022 •
edited

Hmm, may be an error on their end then. (Link to BC health regions IDs from StatCan)

Some screenshots from the BC dashboard:

Northern (595):

595

Total doses are 537,461, which makes sense with a total population ~300k.

Fraser (592):

592

Total doses are 4,199,845, which makes sense with a total population ~2 million.

This is also ~consistent with what is reported on the covid19tracker.ca page of BC vaccination. So it seems this is just a problem of HRUID labels in the API.

@jeanpaulrsoucy
Copy link
Member Author

Okay, I figured it out.

The health region IDs for BC used by covid19tracker.ca are wrong: https://api.covid19tracker.ca/province/BC/regions
...but this traces back to the health region IDs for BC used by us in Covid19Canada being wrong: https://github.com/ccodwg/Covid19Canada/blob/master/other/hr_map.csv
...but this traces back to the health region IDs for BC used by ESRI (for their map file) being wrong: https://resources-covid19canada.hub.arcgis.com/datasets/covid19canada::health-region-summaries

However, our current HRUIDs for BC are now correct, according to both StatCan and BC itself.

@jeanpaulrsoucy
Copy link
Member Author

I've sent a note to @noahlittle so hopefully we can get aligned on this shortly. :)

This was referenced May 29, 2022
Merged
Closed
jeanpaulrsoucy added a commit that referenced this issue Apr 14, 2024
@jeanpaulrsoucy
Update geo population files
f4caf08 
- Update CAN and PT populations (#23, #112)
- HR populations are not updated, since latest version of annual population dataset for 2018 HR boundaries is 2022
- New 2022 HR boundary files are available (https://www150.statcan.gc.ca/n1/pub/82-402-x/82-402-x2023001-eng.htm); however, the most relevant fixes (i.e., changing SK PHUs and merging two ON PHUs) have already been handled in the maps downloaded from the PHAC dashboard (#50) and in the population data scripts (#23). There also doesn't seem to be an annual population table for 2022 HR boundaries yet like there are for the 2018 boundaries (https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1710013401), so I will not be using the new boundary files.

What's new for 2022 HR boundaries (https://www150.statcan.gc.ca/n1/pub/82-402-x/82-402-x2023001-eng.htm):
- Saskatchewan introduced extensive, province-wide changes to the health regions, reducing the number of health regions from 13 to 7.
- Ontario merged Huron County (3539) and Perth District Health (3554) Units to form Huron Perth Health Unit (3550).
- Ontario renamed Local Health Integration Networks (LHINs) to Home and Community Care Support Services.
- British Columbia reassigned 36 dissemination blocks and 5 dissemination areas to refine the precision of census correspondence to health regions. In addition to that, two boundary adjustments were carried out in Alberta, and one in New Brunswick.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
data New data sources or updates to existing data sources. enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@apiology @jeanpaulrsoucy