Some sites cannot be retrieved and cause call to WQP to fail

[limnoliver]

Certain characters/spaces in either the organization or siteid cause calls to WQP to fail. This resulted in many (50ish) targets to not build. If we can identify patterns in these issues, we can create a filter on the WQP inventory step that will help limit these issues.

Some sites/patterns that I've been able to narrow in on that fail:

• organization id COE/ISU (several sites) -- forward slash in the organization ID may be problem
• organization id ALABAMACOUSHATTATRIBE.TX_WQX (several sites) -- period in the organization id may be problem
• organization ids RCE WRP and SAN PASQUAL -- space in organization id may be the problem

[jordansread]

Are we posting by Org ID or monitoring ID? Perhaps we can exclude the Org ID and just use the monitoring ID to avoid some of these?

[limnoliver]

We're doing almost everything monitoring ID -- but the issues all seem to be related to the Org ID, and it's sometimes hard to parse what's the site and what's the org from the monitoring ID string. The monitoring ID is the Org ID + site ID.

[limnoliver]

After making the above fixes, I'm still getting a 404 error message from 9 calls to WQP through dataRetrieval. From the partitions file, the follow pull tasks are failing

failed_pulltasks <- c("190412203340_020", "190412203340_037", "190412203340_048", "190412203340_078", "190412203340_114", "190412203340_146", "190412203340_161", "190412203340_170", "190412203340_195")

I think these will be fixed if dataRetrieval moves towards using POST for downloading data.

[limnoliver]

Need to filter out bad sites + use Jordan's POST script + reduce batch size to hopefully resolve WQP issues.

[limnoliver]

This is been resolved with the POST script.