Nearest Storm Not Working

[dmccabe79]

I've been sitting through a few days of stormy weather and the "nearest storm" function has remained at zero. Have you seen this before or maybe I have this setup incorrectly.

[alexander0042]

Hi.

Thanks for using this API and filing this issue! The feedback I've gotten from people has been invaluable to improving this, so I really do appreciate it.

In this particular case, it's less of a "not working" than "does not work at all" situation unfortunately. I've spent some time thinking about this particular detail, and haven't come up with a good way to calculate this. I'd love to know how Dark Sky did it, since there's two problems to solve to get this number:

1. How do we define a storm? Precipitation would be the most intuitive, but also tricky to forecast and kind of scattered. Pressure would be easier, but doesn't correlate to storms as well.
2. Related to this, what sort of threshold would work. Either create some sort of fixed threshold (like 5 mm of rain or something) and find the distance from that, or have it relative to the intensity (would it be better to know you're 25 miles from a minor storm or 500 miles from hurricane)?
3. Once storms have been identified, there are two options: either pre-calculate the distances on the grid and then return that, or make a list of storms and calculate the distance when the API is called.

Thinking through this, I think precipitation is the most intuitive here, so maybe setting a 1 mm threshold (https://www.tropicaltidbits.com/analysis/models/?model=gfs&region=us&pkg=mslp_pcpn&runtime=2022071306&fh=24), the calculating the distance to the nearest cell that hits this threshold. Then to pre-calculate it, scipy (which is already imported) has a KDTree toolbox, which is for nearest neighbors, but is definitely going take some figuring out. I'd have an array of "storm cell" xy points, and then would need the distance from each point in the grid to the nearest "storm cell" point

[dmccabe79]

I've experimented with calculating a median distance and azimuth from Blitzortung lightning strikes. It's rough because of having errant strikes here and there, but paints a broad picture as to where the majority of the convective activity is. I should really keep working on that.

[alexander0042]

Lightning is an interesting idea! This is why I love open source projects, since there are so many great ideas and solutions to these sorts of difficult problems.

I hadn't seen that website before, so I'll take a deeper dive into it. It looks like they have the data archived somehow, which would be step 1 to getting it into the API. It's amazing that it has global coverage, since that's always a concern with weather data sources.

[github-actions]

There has been no activity on this issue for ninety days and unless you comment on the issue it will automatically close in seven days.

[AD-Wright]

What meanest thou, Bot? Forsooth, the last activity was 5 days ago!

[cloneofghosts]

Referencing an issue from a different issue doesn't count as activity on this issue. I think activity is defined as changing the label, changing who the issue is assigned to, editing a comment, editing the title or adding/removing a comment.

Since you commented on this issue it will no longer close automatically and the label will be removed the next time this action is run.

Staying on topic if lightning data was added it could be used to show a Thunderstorm icon? I know DarkSky didn't have it as an icon before but they said they might define it in the future. I think most apps would have an icon assigned in case it ever got added. Otherwise maybe any sort of precipitation over 1mm/h would probably work as stated before or maybe even both would work but the priority would be on thunderstorms.

[alexander0042]

This + text summaries are the two things that Dark Sky did that Pirate Weather doesn't, and so I've tagged this as "keep" so it stays open. Seems unrelated, but this one will actually be fixed in conjunction with the "daily max/ min" change.

After mulling it over, I think precipitation rate is probably the best way forward. Lightning would be cool, but won't work for winter storms, and there's no good/ easily accessible database. So the game plan is:

1. Mask off all cells with a precipitation intensity > x mm/h (I'm thinking 2 mm/h), probably using the GEFS results (lower res/ global, so easier to process, and better precipitation).
2. Run the xarray proximity distance function, which will give me a distance and a direction for every point on the grid.
3. Loop over every time step.

[cloneofghosts]

After mulling it over, I think precipitation rate is probably the best way forward. Lightning would be cool, but won't work for winter storms, and there's no good/ easily accessible database.

There was a comment earlier in this issue about potentially using Blitzortung lightning data to figure out where storms are. I assume you've looked into it and found that it can't easily be added to the API? If it were to be added you could use a combination of lightning strikes and precipitation rate to figure out where a storm is.

Plus during the summer storms can pop up pretty quickly and since you're using GEFS data it could miss some summer storms because it only updates every 6h.

[alexander0042]

Good points @cloneofghosts. The way it's shaping up is using the 15 minute HRRR data for this, which should capture summer storms in southern Canada/ US at least. It'll be GFS elsewhere (since that at least gives me hourly), but since it doesn't need to match precipitation probability, I don't need to fall back to GEFS

[cloneofghosts]

This issue has been fixed on the V2 development API which should be out some time this winter.

[alexander0042]

Thanks for cleaning this up- I'm pretty excited for this feature to launch!