fix: handle unknown Rusoto errors with 404 status

[alukach]

What I'm changing

Added handling for unknown Rusoto errors that return a 404 status, mapping them to a custom BackendError::ObjectNotFound.

I'm honestly not sure why Rusoto is throwing RusotoError::Unknown errors when an object is not found, however based on this API response I believe that is indeed the error being thrown:

▶ curl --verbose https://data.source.coop/joshmoore/idr-ome-ngff-samples/this-does-not-exist                 
> GET /joshmoore/idr-ome-ngff-samples/this-does-not-exist HTTP/2
> Host: data.source.coop
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
< HTTP/2 502 
< date: Wed, 28 May 2025 18:15:33 GMT
< content-length: 86
< access-control-allow-credentials: true
< vary: Origin, Access-Control-Request-Method, Access-Control-Request-Headers
< x-version: 0.1.27
< 
Internal Server Error: s3 error: HeadObject Unknown Error: status 404 Not Found, body

Note that the text matches the error generated here:

data.source.coop/src/utils/errors.rs

Lines 149 to 154 in 9da5253

	RusotoError::Unknown(e) => format!(
	"{} Unknown Error: status {}, body {}",
	operation,
	e.status,
	e.body_as_str()
	),

How I did it

Add a match for unknown errors where status == 404, cast those errors as BackendError::ObjectNotFound.

How to test it

This fix is currently deployed to https://data.dev.source.coop.

Dev API running on main

▶ curl --silent --verbose https://data.dev.source.coop/amitbajaj/ecovicerepo/this-does-not-exist         
* Host data.dev.source.coop:443 was resolved.
* IPv6: (none)
* IPv4: 34.196.190.40, 34.198.203.6
*   Trying 34.196.190.40:443...
* Connected to data.dev.source.coop (34.196.190.40) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-AES128-GCM-SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=*.dev.source.coop
*  start date: Nov  6 00:00:00 2024 GMT
*  expire date: Dec  5 23:59:59 2025 GMT
*  subjectAltName: host "data.dev.source.coop" matched cert's "*.dev.source.coop"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M03
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://data.dev.source.coop/amitbajaj/ecovicerepo/this-does-not-exist
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: data.dev.source.coop]
* [HTTP/2] [1] [:path: /amitbajaj/ecovicerepo/this-does-not-exist]
* [HTTP/2] [1] [user-agent: curl/8.7.1]
* [HTTP/2] [1] [accept: */*]
> GET /amitbajaj/ecovicerepo/this-does-not-exist HTTP/2
> Host: data.dev.source.coop
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
< HTTP/2 502 
< date: Wed, 28 May 2025 18:22:11 GMT
< content-length: 86
< x-version: 0.1.27
< access-control-allow-credentials: true
< vary: Origin, Access-Control-Request-Method, Access-Control-Request-Headers
< 
* Connection #0 to host data.dev.source.coop left intact
Internal Server Error: s3 error: HeadObject Unknown Error: status 404 Not Found, body %    

Dev API running on fix/s3-404s

▶ curl --silent --verbose https://data.dev.source.coop/amitbajaj/ecovicerepo/this-does-not-exist
* Host data.dev.source.coop:443 was resolved.
* IPv6: (none)
* IPv4: 34.196.190.40, 34.237.166.164, 98.85.134.39, 34.198.203.6
*   Trying 34.196.190.40:443...
* Connected to data.dev.source.coop (34.196.190.40) port 443
* ALPN: curl offers h2,http/1.1
* (304) (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/cert.pem
*  CApath: none
* (304) (IN), TLS handshake, Server hello (2):
* (304) (IN), TLS handshake, Unknown (8):
* (304) (IN), TLS handshake, Certificate (11):
* (304) (IN), TLS handshake, CERT verify (15):
* (304) (IN), TLS handshake, Finished (20):
* (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-AES128-GCM-SHA256 / [blank] / UNDEF
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=*.dev.source.coop
*  start date: Nov  6 00:00:00 2024 GMT
*  expire date: Dec  5 23:59:59 2025 GMT
*  subjectAltName: host "data.dev.source.coop" matched cert's "*.dev.source.coop"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M03
*  SSL certificate verify ok.
* using HTTP/2
* [HTTP/2] [1] OPENED stream for https://data.dev.source.coop/amitbajaj/ecovicerepo/this-does-not-exist
* [HTTP/2] [1] [:method: GET]
* [HTTP/2] [1] [:scheme: https]
* [HTTP/2] [1] [:authority: data.dev.source.coop]
* [HTTP/2] [1] [:path: /amitbajaj/ecovicerepo/this-does-not-exist]
* [HTTP/2] [1] [user-agent: curl/8.7.1]
* [HTTP/2] [1] [accept: */*]
> GET /amitbajaj/ecovicerepo/this-does-not-exist HTTP/2
> Host: data.dev.source.coop
> User-Agent: curl/8.7.1
> Accept: */*
> 
* Request completely sent off
< HTTP/2 404 
< date: Wed, 28 May 2025 18:30:48 GMT
< content-length: 26
< x-version: 0.1.27
< vary: Origin, Access-Control-Request-Method, Access-Control-Request-Headers
< access-control-allow-credentials: true
< 
* Connection #0 to host data.dev.source.coop left intact
object not found: Some("")

PR Checklist

• This PR has no breaking changes.
• I have updated or added new tests to cover the changes in this PR.
• This PR affects the Source Cooperative Frontend & API,
  and I have opened issue/PR #XXX to track the change.

Related Issues