Skip to content

Conversation

@nmdefries
Copy link
Contributor

@nmdefries nmdefries commented Jul 15, 2022

Description

A couple fixes to the "amend microdata" script to deal with geo edge cases.

  • Local zips info file was missing two FIPS codes. Switch to covidcast to generate state column instead.
  • Blank invalid (low population) zip codes instead of blanking everything but valid zip codes. Makes later assertion easier and preserves nonsense/invalid A3 values that have NA zip codes.
  • Filter out territories by zip code in addition to filter on state in case state field is not available.

Changelog

  • amend_monthly_microdata.R

Keeping only zips in a valid zips list ends up dropping obs with missing
(NA) A3/zip5. Add message to assert.

Check if the script is trying to
blank (based on low population) zip code for any number of obs. Low
population filtering was initially implemented in May 2020, so almost
all files should see no blanking. If that's not true, there might be an
error upstream.
@nmdefries nmdefries requested a review from capnrefsmmat July 15, 2022 18:14
@nmdefries
Copy link
Contributor Author

@krivard This is ready to merge.

@krivard krivard merged commit 99e63c4 into main Jul 15, 2022
@krivard krivard deleted the ndefries/amend-microdata-zips-fix branch July 15, 2022 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants