Browsing the naciscdn download URLs directly

[joelostblom]

If I go to https://www.naturalearthdata.com/downloads/10m-cultural-vectors/ and right click the "Download states and provinces" link to copy the URL:

I get a url that looks like this:

https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/50m/cultural/ne_50m_admin_1_states_provinces.zip

I have noticed that this url does not work for programmatic access (for example, attempting to download the file via geopandas raises HTTPError: HTTP Error 406: Not Acceptable). Instead I need to use a URL in this format:

https://naciscdn.org/naturalearth/50m/cultural/ne_50m_admin_1_states_provinces.zip

I noticed that I can get this URL by checking the redirects in the Network section of the browser console, and right clicking to copy the value of the naciscdn entry:

However, I wonder if there is anywhere I can browse the naciscdn URLs directly instead of going through the browser console/manually editing the url? I really like the clarity and description of the web page, and it would be convenient is simply right clicking the download link would give a URL that also works for programmatic access, but I understand there might be some technical limitations hindering that.

[jonjab]

We're having this issue also.

[SomeoneElseOSM]

I'm seeing the same thing. I'm trying to download NE data on a server (which is surely what most people do, right?)/ At https://www.naturalearthdata.com/downloads/110m-cultural-vectors/ I can right click the URL and copy it, and then try and download it, this happens:

wget https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip
--2024-04-25 19:37:57--  https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip
Resolving www.naturalearthdata.com (www.naturalearthdata.com)... 50.87.253.14
Connecting to www.naturalearthdata.com (www.naturalearthdata.com)|50.87.253.14|:443... connected.
HTTP request sent, awaiting response... 500 Internal Server Error
2024-04-25 19:37:58 ERROR 500: Internal Server Error.

This is "at best unexpected". Why not just link to the actual download location? For completeness this works:

wget https://naciscdn.org/naturalearth/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip
--2024-04-25 19:49:13--  https://naciscdn.org/naturalearth/110m/cultural/ne_110m_admin_0_boundary_lines_land.zip
Resolving naciscdn.org (naciscdn.org)... 108.156.22.25, 108.156.22.88, 108.156.22.58, ...
Connecting to naciscdn.org (naciscdn.org)|108.156.22.25|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 57325 (56K) [application/zip]
Saving to: ‘ne_110m_admin_0_boundary_lines_land.zip’

ne_110m_admin_0_bou 100%[===================>]  55.98K   353KB/s    in 0.2s

2024-04-25 19:49:14 (353 KB/s) - ‘ne_110m_admin_0_boundary_lines_land.zip’ saved [57325/57325]

[QuLogic]

If you know what you're looking for programmatically-speaking, the format of the URL is described here: #246 (comment) There's also the S3 bucket at https://naturalearth.s3.amazonaws.com/ which lists everything (which isn't easily clickable, but may be useful with some S3-ish library), except as noted in the other issue, it doesn't have the latest release specifically tagged.

[joelostblom]

Thanks @QuLogic ! Ideally I would like to have someway of seeing the useful information of the website together with the correct link. The main reason is that I use this site when teaching geoviz to students and it's a bit confusing for them that the links on the web page don't work programmatically. They don't know enough at this point to manually specify the URL.