Skip to content

Retry Socrata API requests in case of failure in vignettes#98

Merged
jeancochrane merged 2 commits into
masterfrom
jeancochrane/retry-socrata-api-requests-in-case-of-failure
May 14, 2026
Merged

Retry Socrata API requests in case of failure in vignettes#98
jeancochrane merged 2 commits into
masterfrom
jeancochrane/retry-socrata-api-requests-in-case-of-failure

Conversation

@jeancochrane
Copy link
Copy Markdown
Member

Damon noticed that the tax rate vignette sometimes fails to render because the Socrata API closes the connection during a query. See these logs for an example:

! Failed to render 'vignettes/tax_rates.Rmd'.
✖ Quitting from tax_rates.Rmd:65-94 [unnamed-chunk-3]
✖ Warning message:
✖ call dbDisconnect() when finished working with a connection
Caused by error:
! Failure when receiving data from the peer [datacatalog.cookcountyil.gov]:
Recv failure: Connection reset by peer
Backtrace:
    ▆
 1. ├─httr::GET(...)
 2. │ └─httr:::request_perform(req, hu$handle$handle)
 3. │   ├─httr:::request_fetch(req$output, req$url, handle)
 4. │   └─httr:::request_fetch.write_memory(req$output, req$url, handle)
 5. │     └─curl::curl_fetch_memory(url, handle = handle)
 6. └─curl:::raise_libcurl_error(...)
Execution halted

Our Socrata API work in data-architecture has revealed that closed connections are common when working with the Socrata API, so this PR tweaks our vignettes to use the RETRY() httr function to retry requests to Socrata in case of closed connections.

There's no definitive way to test that this change resolves the sporadic errors, since they are non-deterministic and so not reproducible. But I don't think this change will hurt, and we can re-evaluate it later if we notice builds continuing to fail.

@jeancochrane jeancochrane marked this pull request as ready for review May 12, 2026 22:36
@jeancochrane jeancochrane requested a review from Damonamajor May 12, 2026 22:36
Copy link
Copy Markdown
Contributor

@Damonamajor Damonamajor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes perfect sense to me!

Copy link
Copy Markdown
Member

@kyrasturgill kyrasturgill left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching and addressing this!

@jeancochrane jeancochrane merged commit 2872123 into master May 14, 2026
9 checks passed
@jeancochrane jeancochrane deleted the jeancochrane/retry-socrata-api-requests-in-case-of-failure branch May 14, 2026 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants