`Accept: application/geo+json` returning `text/html` response #838

morikplay · 2025-10-01T18:06:31Z

morikplay
Oct 1, 2025

Hello,
Please accept my apologies if the question is either untimely or irrelevant.

I'm trying to consume api.weather.gov data via python code for a home weather monitoring app. But, encountering a weird behavior where in:

Requesting gridpoint data via command line (linux, macOS, windows), browsers (Mozilla, chrome, safari), and postman result in expected response (geo+json data in responses' Content-Type)

curl --location 'https://api.weather.gov/gridpoints/SGX/60,31/forecast' --header 'Feature-Flags: forecast_temperature_qv,forecast_wind_speed_qv' --header 'Accept: application/geo+json'

Requesting on the same url in python code results in 200 OK w/ text/html but no actual data. Please note, User-Agent header, per this link, is included in the GET request.

Response headers

2025-10-01 09:10:47.160623 INFO nws: https://api.weather.gov/gridpoints/SGX/60,31/forecast
2025-10-01 09:10:47.329915 INFO nws: Status Code: 200
2025-10-01 09:10:47.330312 INFO nws:   Server: nginx/1.20.1
2025-10-01 09:10:47.330683 INFO nws:   Content-Type: text/html; charset=UTF-8
2025-10-01 09:10:47.331051 INFO nws:   Access-Control-Allow-Origin: *
2025-10-01 09:10:47.331347 INFO nws:   X-Request-ID: 391f0c65-4ac3-4a45-94ac-5b79cee6259a
2025-10-01 09:10:47.331638 INFO nws:   X-Correlation-ID: e7c35d6
2025-10-01 09:10:47.331957 INFO nws:   X-Server-ID: vm-bldr-api-ops-app14.ncep.noaa.gov
2025-10-01 09:10:47.332256 INFO nws:   Pragma: no-cache
2025-10-01 09:10:47.332551 INFO nws:   Content-Encoding: gzip
2025-10-01 09:10:47.332856 INFO nws:   Content-Length: 2081
2025-10-01 09:10:47.333142 INFO nws:   Cache-Control: private, must-revalidate, max-age=0
2025-10-01 09:10:47.333417 INFO nws:   Expires: Wed, 01 Oct 2025 16:10:47 GMT
2025-10-01 09:10:47.333699 INFO nws:   Date: Wed, 01 Oct 2025 16:10:47 GMT
2025-10-01 09:10:47.333979 INFO nws:   Connection: keep-alive
2025-10-01 09:10:47.334247 INFO nws:   Vary: Accept-Encoding, Accept,Feature-Flags,Accept-Language
2025-10-01 09:10:47.334537 INFO nws:   X-Edge-Request-ID: 11d44f9
2025-10-01 09:10:47.334845 INFO nws:   Strict-Transport-Security: max-age=31536000 ; includeSubDomains ; preload
2025-10-01 09:10:47.335158 INFO nws: Response BODY does not contain json ...
2025-10-01 09:10:47.336269 INFO nws: Response BODY = <!DOCTYPE html>

Response body

<html>
    <head>
                    <title>api.weather.gov</title>
                                <link rel="stylesheet" href="/build/app.addd834c.css">
     <link rel="stylesheet" href="/build/swagger-ui/swagger-ui.ef52a1d9.css" />
                                <script src="/build/runtime.38e076bf.js"></script><script src="/build/app.48df84fd.js"></script>
    <script src="/build/swagger-ui/swagger-ui-bundle.ac392a99.js"></script>
            </head>
    <body>
        <div class="wrapper">
            <section class="usa-banner" aria-label="Official website of the United States government">
    <div class="usa-accordion">
        <header class="usa-banner__header">
            <div class="usa-banner__inner">
                <div class="grid-col-auto">
                    <img aria-hidden="true" class="usa-banner__header-flag" src="/build/images/uswds/usa-banner/us_flag_small.cb0b79fb.png" alt="" />
                </div>
                <div class="grid-col-fill tablet:grid-col-auto" aria-hidden="true">
                    <p class="usa-banner__header-text">
                        An official website of the United States government
                    </p>
                    <p class="usa-banner__header-action">Here���s how you know</p>
                </div>
                <button type="button" class="usa-accordion__button usa-banner__button" aria-expanded="false" aria-controls="gov-banner-default">
                    <span class="usa-banner__button-text">Here���s how you know</span>
                </button>
            </div>
        </header>
        <div class="usa-banner__content usa-accordion__content" id="gov-banner-default">
            <div class="grid-row grid-gap-lg">
                <div class="usa-banner__guidance tablet:grid-col-6">
                    <img class="usa-banner__icon usa-media-block__img" src="/build/images/uswds/usa-banner/icon-dot-gov.8f8f40ed.svg" role="img" alt="" aria-hidden="true" />
                    <div class="usa-media-block__body">
                        <p>
                            <strong>Official websites use .gov</strong><br />A
                            <strong>.gov</strong> website belongs to an official government
                            organization in the United States.
                        </p>
                    </div>
                </div>
                <div class="usa-banner__guidance tablet:grid-col-6">
                    <img class="usa-banner__icon usa-media-block__img" src="/build/images/uswds/usa-banner/icon-https.42f84b9c.svg" role="img" alt="" aria-hidden="true" />
                    <div class="usa-media-block__body">
                        <p>
                            <strong>Secure .gov websites use HTTPS</strong><br />A
                            <strong>lock</strong> (
                            <span class="icon-lock">
                                <svg xmlns="http://www.w3.org/2000/svg" width="52" height="64" viewBox="0 0 52 64" class="usa-banner__lock-image" role="img" aria-labelledby="banner-lock-description-default" focusable="false">
                                    <title id="banner-lock-title-default">Lock</title>
                                    <desc id="banner-lock-description-default">Locked padlock icon</desc>
                                    <path fill="#000000" fill-rule="evenodd" d="M26 0c10.493 0 19 8.507 19 19v9h3a4 4 0 0 1 4 4v28a4 4 0 0 1-4 4H4a4 4 0 0 1-4-4V32a4 4 0 0 1 4-4h3v-9C7 8.507 15.507 0 26 0zm0 8c-5.979 0-10.843 4.77-10.996 10.712L15 19v9h22v-9c0-6.075-4.925-11-11-11z" />
                                </svg>
                            </span>) or <strong>https://</strong> means you���ve safely connected to
                            the .gov website. Share sensitive information only on official,
                            secure websites.
                        </p>
                    </div>
                </div>
            </div>
        </div>
    </div>
</section>
            <div id="content">    <div id="openapi" data-definition-url="https://api.weather.gov/openapi.json"></div>
</div>
            <footer role="contentinfo">
    <div class="usa-identifier padding-y-1">
        <nav class="usa-identifier__section usa-identifier__section--required-links" aria-label="Important links,">
            <div class="grid-container">
                <div class="grid-row">
                    <div class="desktop:grid-col-4">
                        <div>
                            <a href="https://noaa.gov/" class="site-logo display-inline-block">
                                <img class="height-9" src="/build/images/logo-noaa.3a4f2da6.svg" alt="National Oceanic and Atmospheric Administration logo" />
                            </a>
                            <a href="https://weather.gov/" rel="home" class="site-logo display-inline-block">
                                <img class="height-9" src="/build/images/logo-nws.5216c53b.svg" alt="National Weather Service logo" />
                            </a>
                        </div>
                        <span class="text-italic display-block">Science. Service. Stewardship.</span>
                    </div>
                    <div class="desktop:grid-col-8">
                        <div class="region region-primary-footer">
                            <nav role="navigation" aria-labelledby="block-new-weather-theme-footerprimarylinks-menu" id="block-new-weather-theme-footerprimarylinks" class="block block-menu navigation menu--footer">
                                <h2 class="visually-hidden" id="block-new-weather-theme-footerprimarylinks-menu">Footer primary links</h2>
                                <nav class="usa-identifier__section usa-identifier__section--required-links">
                                    <div class="usa-identifier__container padding-0">
                                        <ul class="usa-identifier__required-links-list">
                                            <li class="usa-identifier__required-links-item">
                                                <a href="https://www.noaa.gov/NOAA-Communications" class="usa-identifier__required-link usa-link">Media inquiries</a>
                                            </li>
                                            <li class="usa-identifier__required-links-item">
                                                <a href="https://www.noaa.gov/information-technology/foia" class="usa-identifier__required-link usa-link">Freedom of Information Act</a>
                                            </li>
                                            <li class="usa-identifier__required-links-item">
                                                <a href="https://www.noaa.gov/organization/information-technology/policy-oversight/information-quality" class="usa-identifier__required-link usa-link">Information Quality</a>
                                            </li>
                                            <li class="usa-identifier__required-links-item">
                                                <a href="https://usa.gov/" class="usa-identifier__required-link usa-link">USA.gov</a>
                                            </li>
                                        </ul>
                                    </div>
                                </nav>
                            </nav>
                        </div>
                    </div>
                </div>
            </div>
        </nav>
        <div class="region region-footer">
            <nav role="navigation" aria-labelledby="block-new-weather-theme-agencyidentificationlinks-menu" id="block-new-weather-theme-agencyidentificationlinks" class="block block-menu navigation menu--agency-identification-links">
                <h2 class="visually-hidden" id="block-new-weather-theme-agencyidentificationlinks-menu">Agency identification links</h2>
                <section class="usa-identifier__section usa-identifier__section--usagov" aria-label="U.S. government information and services,">
                    <div class="usa-identifier__container">
                        <ul class="usa-list--unstyled desktop:display-flex flex-row flex-justify">
                            <li>
                                <a class="usa-link text-bold" href="https://www.commerce.gov/">Department of Commerce</a>
                            </li>
                            <li>
                                <a class="usa-link text-bold" href="https://www.noaa.gov/">National Oceanic and Atmospheric Administration (NOAA)</a>
                            </li>
                            <li>
                                <a class="usa-link text-bold" href="https://www.weather.gov/">National Weather Service</a>
                            </li>
                        </ul>
                    </div>
                </section>
            </nav>
        </div>
    </div>
</footer>
        </div>
    </body>
</html>

Relevant python code

    def initialize(self):
        self.entity_id = "sensor.esco_nws_forecast"
        self.unique_id = "ba82093f-5964-467d-8170-f03458d86b16"
        self.friendly_name = "Esco NWS Forecast"
        self.base_url = self.args["url"]
        self.wfo = self.args["wfo"]
        self.lat = self.args["lat"]
        self.long = self.args["long"]
        self.x = self.args["point_x"]
        self.y = self.args["point_y"]

        self.uuid_forecast = self.args["uuid_forecast"]
        self.uuid_forecast_hourly = self.args["uuid_forecast_hourly"]

        # HTTP related variables
        self.header_host = "api.weather.gov"
        self.header_featureflags = "forecast_temperature_qv,forecast_wind_speed_qv"
        self.header_accept = "application/geo+json"
        self.header_accept_encoding = "gzip, deflate, br, zstd"
        self.header_user_agent = "(HomeAssistantApp, <my_email_address>)"
        
        starttime = datetime.now() + timedelta(seconds=5)
        self.run_every(self.get_forecast, starttime, self.args["refresh_interval"])
    
    def get_forecast(self, kwargs):
        try:
            # Create url
            url = f"{self.base_url}/gridpoints/{self.wfo}/{self.x},{self.y}/forecast"
            self.log(url)
            
            add_headers = {
                "Host": self.header_host,
                "Feature-Flags": self.header_featureflags,
                "Accept": self.header_accept,
                "Accept-Encoding": self.header_accept_encoding,
                "User-Agent": self.header_user_agent
            }

            try:
                response = requests.get(self.url, headers=add_headers)
                # response = requests.get(self.base_url)
            except ConnectionError as e:
                self.error(f"Connection error when fetching data from {url}: {e}")
            except Timeout as e:
                self.error(f"Timeout error when fetching data from {url}: {e}")
            except HTTPError as e:
                self.error(f"HTTP error when fetching data from {url}: {e}")
            except RequestException as e: # Catch any other requests-related exceptions
                self.error(f"An unexpected requests error occurred when fetching data from {url}: {e}")

morikplay · 2025-10-03T15:57:23Z

morikplay
Oct 3, 2025
Author

The weird behavior was most likely due to some caching issues. It resolved itself in 24 hours.

1 reply

scadergit Oct 3, 2025
Maintainer

It was returning the root path, which is the only HTML response.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

`Accept: application/geo+json` returning `text/html` response #838

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 1 comment 1 reply

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

Accept: application/geo+json returning text/html response #838

Uh oh!

Uh oh!

morikplay Oct 1, 2025

Replies: 1 comment · 1 reply

Uh oh!

morikplay Oct 3, 2025 Author

Uh oh!

scadergit Oct 3, 2025 Maintainer

`Accept: application/geo+json` returning `text/html` response #838

morikplay
Oct 1, 2025

Replies: 1 comment 1 reply

morikplay
Oct 3, 2025
Author

scadergit Oct 3, 2025
Maintainer