Updated alerts not removed

[steven-dyson]

Issue: Updated alerts are no removed. I have three active while weather.gov is only showing two (as is dark sky)

Reproduction: When an alert is updated it should show as a new alert. I did a diff on both alerts and found a few minor changes.

Expected Result: When an alert is updated, the number of alerts should remain the same. The entity is updated which can be used to trigger automatons for weather alert updates (instead of new weather alerts).

Side note: Thanks for the integration. I needed something with alerts to replace DarkSky for when the API is no longer free.

[finity69x2]

I'm not sure I follow.

I haven't had any issues with expired events not being removed. Maybe it was a glitch in the NWS system? they have a few of those fairly often.

As far as I know (unless they have changed something) every new or update to an alert should generate a new event id. So you should be able to figure out if there is an update by looking at the event id. Or maybe I've just never noticed and it's never been mentioned before.

However, I have seen alert updates and they have come thru as new alerts. So I think it's working as expected.

but there is also no way for the integration to know if it's an update to a previously issued alert or if it's a completely new alert since they both (should...) get new event ids.

[steven-dyson]

To clarify, when an update to an alert is published, new_alerts is adding it as if it is a new alert. I originally had one flooding advisory and one wind advisory. When weather.gov updated the wind advisory, nws_alerts now showed three total alerts. However, when I looked at weather.gov and DarkSky it correctly displayed only two active alerts.

I understand that each alert is given a unique ID, however the number of active alerts should match weather.gov. Perhaps DarkSky pulls the alerts when they receive an update.

[finity69x2]

I've never had that issue so I'm not sure how I could replicate it to troubleshoot it.

were all three of them all within the "expires" or "ends" timeframe?

Was this a one time thing or has it happened multiple times?

[pman07]

I'm getting this same issue currently. From what I can tell there was an updated warning still in effect with an "expiredReferences" noted on the updated warning.

Here is the raw json event ID:

{
    "path": "/alerts/urn:oid:2.49.0.1.840.0.ac8373dd2a552ac0bfef664631cc8f9b147e833d.001.1-https:/api.weather.gov/alerts/urn:oid:2.49.0.1.840.0.fb35318525ffa30ed15fbe1939a1940b8ea27af4.001.1-https:/api.weather.gov/alerts/urn:oid:2.49.0.1.840.0.5dcb16c287bec757a3ce9490be278edc3f0012ab.001.1",
    "correlationId": "9583dd62-6a22-46c0-9592-4cf2f7dd3803",
    "title": "Not Found",
    "type": "https://api.weather.gov/problems/NotFound",
    "status": 404,
    "detail": "'/alerts/urn:oid:2.49.0.1.840.0.ac8373dd2a552ac0bfef664631cc8f9b147e833d.001.1-https:/api.weather.gov/alerts/urn:oid:2.49.0.1.840.0.fb35318525ffa30ed15fbe1939a1940b8ea27af4.001.1-https:/api.weather.gov/alerts/urn:oid:2.49.0.1.840.0.5dcb16c287bec757a3ce9490be278edc3f0012ab.001.1' is not a valid resource path",
    "instance": "https://api.weather.gov/requests/9583dd62-6a22-46c0-9592-4cf2f7dd3803"
}

Here is the original event link and the updated link.

The relevant data from the updated warning raw json:

        "parameters": {
            "PIL": [
                "EAXNPWEAX"
            ],
            "NWSheadline": [
                "FREEZE WARNING REMAINS IN EFFECT FROM 1 AM TO 9 AM CDT THURSDAY"
            ],
            "BLOCKCHANNEL": [
                "EAS",
                "NWEM",
                "CMAS"
            ],
            "VTEC": [
                "/O.CON.KEAX.FZ.W.0001.210401T0600Z-210401T1400Z/"
            ],
            "eventEndingTime": [
                "2021-04-01T14:00:00+00:00"
            ],
            "expiredReferences": [
                "w-nws.webmaster@noaa.gov,urn:oid:2.49.0.1.840.0.5dcb16c287bec757a3ce9490be278edc3f0012ab.001.1,2021-03-31T04:13:00-05:00"

[steven-dyson]

I haven't had multiple alerts since. Two were almost identical. Reading the second alert I could tell it was just updated. I'll be sure to copy the entire alert entity value next time.

[steven-dyson]

Also, can someone post the entity value of their NWS alert when it has multiple alerts. It could be added to the readme for the purpose of testing.

[finity69x2]

I'm really sorry guys.

It's going to be so hard to troubleshoot this since it's not reproducible at all on my end and the issue conditions seem to happen so rarely.

I'm going to try to get some clarifying info so please don't feel I'm being condescending:

1. Are you saying that the sensor.nws_alert has a state of 1 more than it should (in this example I'll use 2 when there should be only 1) because 1 of the alerts has been removed/expired (yes/no?) or updated an old one that hasn't expired (yes/no?)?

2. do both of those (both current & updated) alerts both show up in the attributes of the sensor at the same time?

3. if they do both show up are there differences between the data displayed for both alerts in attributes?

4. could you also go to the following website to see what the API is actually returning when you see the issue:

https://api.weather.gov/alerts/active?zone=

5. do both of the alerts both show up at that webpage at the same time or only one of them?

6. at some point I assume they are both getting removed (sensor.nws_alerts goes to 0). when does that hap[pen? when both alerts are gone or when the original is removed?

7. try to note down as many details as possible (copy paste the output of the sensor state and attributes along with the results of the NWS API website) and post them in a place I have access to to look at them (public gist/dropbox/etc.)

[steven-dyson]

No problem whatever helps.

1. That is correct. I believe the alert was updated so it never actually expired.
2. Yes the alerts were in the attributes. Reading them I could tell that one was just updated with newer information.
3. Yes there is a difference. It actually stated that it was updated in the description with newer information.
4. As soon as I see it I'll grab the info from the API and the attribute info.
5. I didn't check the API site, I went to the weather.gov site and looked up the active alerts. Your NWS Alert app showed 3 active alerts when the weather.gov site showed only 2.
6. Each were removed after the expiration date/time.
7. Will do

[jojeda654]

I believe I may be currently experiencing this issue.

Currently, the entity is showing a value of 2. When I lookup my county on the alerts site, there is only 1 alert:

Last updated: 11:44 CDT on 04-20-2021
Frost Advisory
Issued: April 20 at 11:44AM CDT
Expiring: April 21 at 9:00AM CDT	Urgency: Expected
Status: Actual
Areas affected:Bosque; Comanche; Coryell; Dallas; Ellis; Erath; Hamilton; Henderson; Hill; Hood; Johnson; Kaufman; Lampasas; McLennan; Mills; Navarro; Rockwall; Somervell; Tarrant; Van Zandt

Clicking on the link in the alert directs me to this message

Event: | Frost Advisory
-- | --
Alert: | ...FROST ADVISORY REMAINS IN EFFECT FROM 3 AM TO 9 AM CDT  WEDNESDAY...    * WHAT...Temperatures as low as 33 will result in frost  formation.    * WHERE...Areas near and north of a Lampasas to Waco to Athens  line.    * WHEN...From 3 AM to 9 AM CDT Wednesday.    * IMPACTS...Frost could damage or kill sensitive outdoor  vegetation if left uncovered.
Instructions: | Take steps now to protect tender plants and other sensitive vegetation from the cold.
Target Area: | BosqueComancheCoryellDallasEllisErathHamiltonHendersonHillHoodJohnsonKaufmanLampasasMcLennanMillsNavarroRockwallSomervellTarrantVan Zandt | BosqueComancheCoryellDallasEllisErathHamiltonHendersonHillHoodJohnsonKaufmanLampasasMcLennanMillsNavarroRockwallSomervellTarrantVan Zandt
BosqueComancheCoryellDallasEllisErathHamiltonHendersonHillHoodJohnsonKaufmanLampasasMcLennanMillsNavarroRockwallSomervellTarrantVan Zandt

This is what the API is returning for my county:

{
    "@context": [
        "https://geojson.org/geojson-ld/geojson-context.jsonld",
        {
            "@version": "1.1",
            "wx": "https://api.weather.gov/ontology#",
            "@vocab": "https://api.weather.gov/ontology#"
        }
    ],
    "type": "FeatureCollection",
    "features": [
        {
            "id": "https://api.weather.gov/alerts/urn:oid:2.49.0.1.840.0.1560de1ab9de481104b119ee49d81ae353d6494a.003.1",
            "type": "Feature",
            "geometry": null,
            "properties": {
                "@id": "https://api.weather.gov/alerts/urn:oid:2.49.0.1.840.0.1560de1ab9de481104b119ee49d81ae353d6494a.003.1",
                "@type": "wx:Alert",
                "id": "urn:oid:2.49.0.1.840.0.1560de1ab9de481104b119ee49d81ae353d6494a.003.1",
                "areaDesc": "Tarrant; Dallas; Rockwall; Kaufman; Van Zandt; Erath; Hood; Somervell; Johnson; Ellis; Henderson; Comanche; Mills; Hamilton; Bosque; Hill; Navarro; Lampasas; Coryell; McLennan",
                "geocode": {
                    "SAME": [
                        "048439",
                        "048113",
                        "048397",
                        "048257",
                        "048467",
                        "048143",
                        "048221",
                        "048425",
                        "048251",
                        "048139",
                        "048213",
                        "048093",
                        "048333",
                        "048193",
                        "048035",
                        "048217",
                        "048349",
                        "048281",
                        "048099",
                        "048309"
                    ],
                    "UGC": [
                        "TXZ118",
                        "TXZ119",
                        "TXZ120",
                        "TXZ121",
                        "TXZ122",
                        "TXZ130",
                        "TXZ131",
                        "TXZ132",
                        "TXZ133",
                        "TXZ134",
                        "TXZ135",
                        "TXZ141",
                        "TXZ142",
                        "TXZ143",
                        "TXZ144",
                        "TXZ145",
                        "TXZ146",
                        "TXZ156",
                        "TXZ157",
                        "TXZ159"
                    ]
                },
                "affectedZones": [
                    "https://api.weather.gov/zones/forecast/TXZ118",
                    "https://api.weather.gov/zones/forecast/TXZ119",
                    "https://api.weather.gov/zones/forecast/TXZ120",
                    "https://api.weather.gov/zones/forecast/TXZ121",
                    "https://api.weather.gov/zones/forecast/TXZ122",
                    "https://api.weather.gov/zones/forecast/TXZ130",
                    "https://api.weather.gov/zones/forecast/TXZ131",
                    "https://api.weather.gov/zones/forecast/TXZ132",
                    "https://api.weather.gov/zones/forecast/TXZ133",
                    "https://api.weather.gov/zones/forecast/TXZ134",
                    "https://api.weather.gov/zones/forecast/TXZ135",
                    "https://api.weather.gov/zones/forecast/TXZ141",
                    "https://api.weather.gov/zones/forecast/TXZ142",
                    "https://api.weather.gov/zones/forecast/TXZ143",
                    "https://api.weather.gov/zones/forecast/TXZ144",
                    "https://api.weather.gov/zones/forecast/TXZ145",
                    "https://api.weather.gov/zones/forecast/TXZ146",
                    "https://api.weather.gov/zones/forecast/TXZ156",
                    "https://api.weather.gov/zones/forecast/TXZ157",
                    "https://api.weather.gov/zones/forecast/TXZ159"
                ],
                "references": [
                    {
                        "@id": "https://api.weather.gov/alerts/urn:oid:2.49.0.1.840.0.5d860d8f9a86fc132967d30ea0643e3ffa86630f.002.1",
                        "identifier": "urn:oid:2.49.0.1.840.0.5d860d8f9a86fc132967d30ea0643e3ffa86630f.002.1",
                        "sender": "w-nws.webmaster@noaa.gov",
                        "sent": "2021-04-20T11:23:00-05:00"
                    }
                ],
                "sent": "2021-04-20T11:44:00-05:00",
                "effective": "2021-04-20T11:44:00-05:00",
                "onset": "2021-04-21T03:00:00-05:00",
                "expires": "2021-04-21T03:00:00-05:00",
                "ends": "2021-04-21T09:00:00-05:00",
                "status": "Actual",
                "messageType": "Update",
                "category": "Met",
                "severity": "Minor",
                "certainty": "Likely",
                "urgency": "Expected",
                "event": "Frost Advisory",
                "sender": "w-nws.webmaster@noaa.gov",
                "senderName": "NWS Fort Worth TX",
                "headline": "Frost Advisory issued April 20 at 11:44AM CDT until April 21 at 9:00AM CDT by NWS Fort Worth TX",
                "description": "* WHAT...Temperatures as low as 33 will result in frost\nformation.\n\n* WHERE...Areas near and north of a Lampasas to Waco to Athens\nline.\n\n* WHEN...From 3 AM to 9 AM CDT Wednesday.\n\n* IMPACTS...Frost could damage or kill sensitive outdoor\nvegetation if left uncovered.",
                "instruction": "Take steps now to protect tender plants and other sensitive\nvegetation from the cold.",
                "response": "Prepare",
                "parameters": {
                    "PIL": [
                        "FWDNPWFWD"
                    ],
                    "NWSheadline": [
                        "FROST ADVISORY REMAINS IN EFFECT FROM 3 AM TO 9 AM CDT WEDNESDAY"
                    ],
                    "BLOCKCHANNEL": [
                        "EAS",
                        "NWEM",
                        "CMAS"
                    ],
                    "VTEC": [
                        "/O.CON.KFWD.FR.Y.0001.210421T0800Z-210421T1400Z/"
                    ],
                    "eventEndingTime": [
                        "2021-04-21T14:00:00+00:00"
                    ],
                    "expiredReferences": [
                        "w-nws.webmaster@noaa.gov,urn:oid:2.49.0.1.840.0.e6faeffbaa43517cab5e253ece255b6c2bf80f13.003.1,2021-04-20T00:52:00-05:00"
                    ]
                }
            }
        },
        {
            "id": "https://api.weather.gov/alerts/urn:oid:2.49.0.1.840.0.e6faeffbaa43517cab5e253ece255b6c2bf80f13.003.1",
            "type": "Feature",
            "geometry": null,
            "properties": {
                "@id": "https://api.weather.gov/alerts/urn:oid:2.49.0.1.840.0.e6faeffbaa43517cab5e253ece255b6c2bf80f13.003.1",
                "@type": "wx:Alert",
                "id": "urn:oid:2.49.0.1.840.0.e6faeffbaa43517cab5e253ece255b6c2bf80f13.003.1",
                "areaDesc": "Lamar; Hunt; Delta; Hopkins; Tarrant; Dallas; Rockwall; Kaufman; Van Zandt; Rains; Erath; Hood; Somervell; Johnson; Ellis; Henderson; Comanche; Mills; Hamilton; Bosque; Hill; Navarro; Lampasas; Coryell; McLennan",
                "geocode": {
                    "SAME": [
                        "048277",
                        "048231",
                        "048119",
                        "048223",
                        "048439",
                        "048113",
                        "048397",
                        "048257",
                        "048467",
                        "048379",
                        "048143",
                        "048221",
                        "048425",
                        "048251",
                        "048139",
                        "048213",
                        "048093",
                        "048333",
                        "048193",
                        "048035",
                        "048217",
                        "048349",
                        "048281",
                        "048099",
                        "048309"
                    ],
                    "UGC": [
                        "TXZ095",
                        "TXZ105",
                        "TXZ106",
                        "TXZ107",
                        "TXZ118",
                        "TXZ119",
                        "TXZ120",
                        "TXZ121",
                        "TXZ122",
                        "TXZ123",
                        "TXZ130",
                        "TXZ131",
                        "TXZ132",
                        "TXZ133",
                        "TXZ134",
                        "TXZ135",
                        "TXZ141",
                        "TXZ142",
                        "TXZ143",
                        "TXZ144",
                        "TXZ145",
                        "TXZ146",
                        "TXZ156",
                        "TXZ157",
                        "TXZ159"
                    ]
                },
                "affectedZones": [
                    "https://api.weather.gov/zones/forecast/TXZ095",
                    "https://api.weather.gov/zones/forecast/TXZ105",
                    "https://api.weather.gov/zones/forecast/TXZ106",
                    "https://api.weather.gov/zones/forecast/TXZ107",
                    "https://api.weather.gov/zones/forecast/TXZ118",
                    "https://api.weather.gov/zones/forecast/TXZ119",
                    "https://api.weather.gov/zones/forecast/TXZ120",
                    "https://api.weather.gov/zones/forecast/TXZ121",
                    "https://api.weather.gov/zones/forecast/TXZ122",
                    "https://api.weather.gov/zones/forecast/TXZ123",
                    "https://api.weather.gov/zones/forecast/TXZ130",
                    "https://api.weather.gov/zones/forecast/TXZ131",
                    "https://api.weather.gov/zones/forecast/TXZ132",
                    "https://api.weather.gov/zones/forecast/TXZ133",
                    "https://api.weather.gov/zones/forecast/TXZ134",
                    "https://api.weather.gov/zones/forecast/TXZ135",
                    "https://api.weather.gov/zones/forecast/TXZ141",
                    "https://api.weather.gov/zones/forecast/TXZ142",
                    "https://api.weather.gov/zones/forecast/TXZ143",
                    "https://api.weather.gov/zones/forecast/TXZ144",
                    "https://api.weather.gov/zones/forecast/TXZ145",
                    "https://api.weather.gov/zones/forecast/TXZ146",
                    "https://api.weather.gov/zones/forecast/TXZ156",
                    "https://api.weather.gov/zones/forecast/TXZ157",
                    "https://api.weather.gov/zones/forecast/TXZ159"
                ],
                "references": [],
                "sent": "2021-04-20T00:52:00-05:00",
                "effective": "2021-04-20T00:52:00-05:00",
                "onset": "2021-04-21T03:00:00-05:00",
                "expires": "2021-04-20T09:00:00-05:00",
                "ends": "2021-04-21T09:00:00-05:00",
                "status": "Actual",
                "messageType": "Alert",
                "category": "Met",
                "severity": "Minor",
                "certainty": "Likely",
                "urgency": "Expected",
                "event": "Frost Advisory",
                "sender": "w-nws.webmaster@noaa.gov",
                "senderName": "NWS Fort Worth TX",
                "headline": "Frost Advisory issued April 20 at 12:52AM CDT until April 21 at 9:00AM CDT by NWS Fort Worth TX",
                "description": "* WHAT...Temperatures as low as 33 will result in frost\nformation.\n\n* WHERE...Areas near and north of a Lampasas to Waco to Athens\nline.\n\n* WHEN...From 3 AM to 9 AM CDT Wednesday.\n\n* IMPACTS...Frost could damage or kill sensitive outdoor\nvegetation if left uncovered.",
                "instruction": "Take steps now to protect tender plants and other sensitive\nvegetation from the cold.",
                "response": "Prepare",
                "parameters": {
                    "PIL": [
                        "FWDNPWFWD"
                    ],
                    "NWSheadline": [
                        "FROST ADVISORY IN EFFECT FROM 3 AM TO 9 AM CDT WEDNESDAY"
                    ],
                    "BLOCKCHANNEL": [
                        "EAS",
                        "NWEM",
                        "CMAS"
                    ],
                    "VTEC": [
                        "/O.NEW.KFWD.FR.Y.0001.210421T0800Z-210421T1400Z/"
                    ],
                    "eventEndingTime": [
                        "2021-04-21T14:00:00+00:00"
                    ]
                }
            }
        }
    ],
    "title": "current watches, warnings, and advisories for Dallas County (TXC113) TX",
    "updated": "2021-04-20T16:45:15+00:00"
}

Here is the attributes content as well:

attribution: Data provided by Weather.gov
title: Frost Advisory - Frost Advisory
event_id: >-
  https://api.weather.gov/alerts/urn:oid:2.49.0.1.840.0.1560de1ab9de481104b119ee49d81ae353d6494a.003.1-https://api.weather.gov/alerts/urn:oid:2.49.0.1.840.0.e6faeffbaa43517cab5e253ece255b6c2bf80f13.003.1
display_desc: |-

  >
  Headline: FROST ADVISORY REMAINS IN EFFECT FROM 3 AM TO 9 AM CDT WEDNESDAY
  Severity: Minor
  Certainty: Likely
  Description: * WHAT...Temperatures as low as 33 will result in frost
  formation.

  * WHERE...Areas near and north of a Lampasas to Waco to Athens
  line.

  * WHEN...From 3 AM to 9 AM CDT Wednesday.

  * IMPACTS...Frost could damage or kill sensitive outdoor
  vegetation if left uncovered.
  Instruction: Take steps now to protect tender plants and other sensitive
  vegetation from the cold.

  -


  >
  Headline: FROST ADVISORY IN EFFECT FROM 3 AM TO 9 AM CDT WEDNESDAY
  Severity: Minor
  Certainty: Likely
  Description: * WHAT...Temperatures as low as 33 will result in frost
  formation.

  * WHERE...Areas near and north of a Lampasas to Waco to Athens
  line.

  * WHEN...From 3 AM to 9 AM CDT Wednesday.

  * IMPACTS...Frost could damage or kill sensitive outdoor
  vegetation if left uncovered.
  Instruction: Take steps now to protect tender plants and other sensitive
  vegetation from the cold.
spoken_desc: |-
  FROST ADVISORY REMAINS IN EFFECT FROM 3 AM TO 9 AM CDT WEDNESDAY

  -

  FROST ADVISORY IN EFFECT FROM 3 AM TO 9 AM CDT WEDNESDAY
friendly_name: NWS Alerts
icon: 'mdi:alert'

[finity69x2]

Thanks @jojeda654 for all of that information.

from all of that it definitely looks like a glitch in the NWS API.

The only thing the integration does is looks at the number of events reported in the data returned by the API. And from the info above there actually two events in the API call. And that info is updated every minute.

How the county active alert counts and the data provided by the API are cross-referenced I have no idea about. It seems like a situation where one hand doesn't fully know what the other is doing.

So with that I'm not sure there is much I can do about it.

I think the important thing is that the new alerts actually pop up when expected and the old alerts do eventually get removed when the API gets updated information.

[finity69x2]

I just added a new attribute for the sensor called "message type".

I don't know if it will help you out or not but you should be able to filter on that message type for "update" or something along those lines.