Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

hotfix: backend tests failing due to Nominatim server error:403 #6304

Merged
merged 1 commit into from
Mar 26, 2024
Merged

hotfix: backend tests failing due to Nominatim server error:403 #6304

merged 1 commit into from
Mar 26, 2024

Conversation

mahesh-naxa
Copy link
Contributor

@mahesh-naxa mahesh-naxa commented Mar 26, 2024 •
edited

Context

The unit test tests/backend/unit/models/postgis/test_project.py's test_set_country_info method was performing a GET from nominatim's reverse geocode api ie: https://nominatim.openstreetmap.org

This api was failing with requests.exceptions.HTTPError: 403 Client Error: Forbidden for url.

Response (Before)

<html>\n<head>\n<title>Access blocked</title>\n</head>\n<body>\n<h1>Access blocked</h1>\n\n<p>You have been blocked because you have violated the\n<a href="https://operations.osmfoundation.org/policies/nominatim/">usage policy</a>\nof OSM\'s Nominatim geocoding service. Please be aware that OSM\'s resources are\nlimited and shared between many users. The usage policy is there to ensure that\nthe service remains usable for everybody.</p>\n\n<p>Please review the terms and make sure that your\nsoftware adheres to the terms. You should in particular verify that you have set a\n<b>custom HTTP referrer or HTTP user agent</b> that identifies your application, and\nthat you are not overusing the service with massive bulk requests.</p>\n\n<p>If you feel that this block is unjustified or remains after you have adopted\nyour usage, you may contact the Nominatim system administrator at\nnominatim@openstreetmap.org to have this block lifted.</p>\n</body>\n</head>\n

Fix

I added a generic user agent headers to requests, which solved the issue.

        headers = {
            "User-Agent": (
                "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                "AppleWebKit/537.36 (KHTML, like Gecko) "
                "Chrome/58.0.3029.110 Safari/537.3"
            )
        }

Response (After)

{"place_id":268185681,"licence":"Data © OpenStreetMap contributors, ODbL 1.0. http://osm.org/copyright","osm_type":"way","osm_id":98588855,"lat":"56.1189493","lon":"-3.9187826284963583","category":"man_made","type":"wastewater_plant","place_rank":30,"importance":9.99999999995449e-06,"addresstype":"man_made","name":"","display_name":"Forthside Way, Sports Village, Stirling, Scotland, FK8 1QZ, United Kingdom","address":{"road":"Forthside Way","suburb":"Sports Village","city":"Stirling","county":"Stirling","ISO3166-2-lvl6":"GB-STG","state":"Scotland","ISO3166-2-lvl4":"GB-SCT","postcode":"FK8 1QZ","country":"United Kingdom","country_code":"gb"},"boundingbox":["56.1180820","56.1199930","-3.9213894","-3.9162879"]}

@mahesh-naxa
hotfix: backend tests failing due to Nominatim server error:403
bfcb43b 
Causing to fail with requests.exceptions.HTTPError: 403 Client Error: Forbidden for url. Solved with appropriate User agent headers
@sonarcloud SonarCloud
Copy link

sonarcloud bot commented Mar 26, 2024

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

@mahesh-naxa mahesh-naxa self-assigned this Mar 26, 2024
@ramyaragupathy ramyaragupathy added this to the v4.7.1 milestone Mar 26, 2024
ramyaragupathy
ramyaragupathy approved these changes Mar 26, 2024
View reviewed changes
Copy link
Member

@ramyaragupathy ramyaragupathy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving this after verifying with @mahesh-naxa that the fix has been implemented to suit the requirement mentioned here: https://operations.osmfoundation.org/policies/nominatim/

@dakotabenjamin dakotabenjamin merged commit ea0b579 into develop Mar 26, 2024
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dakotabenjamin dakotabenjamin dakotabenjamin approved these changes

@ramyaragupathy ramyaragupathy ramyaragupathy approved these changes

Assignees

@mahesh-naxa mahesh-naxa

Labels
scope: backend
Projects
None yet
Milestone
v4.7.1
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@mahesh-naxa @dakotabenjamin @ramyaragupathy