PWA LS Cache issues

[apertome]

Multiple PWA instances are seeing a large number of stale hosts in their local LS caches; it appears that records are expiring and getting renewed with a different ID, and PWA isn't matching them up with the old records.

PWA should be using the LS's client UUID field, but I need to review what the cache is doing, as I suspect it's not doing so.

[apertome]

While troubleshooting the stale hosts problem, I noticed that the cache ignores all hosts that fail reverse dns lookups. I fixed that as part of this issue.

[apertome]

There was no expiration of stale/old hosts happening; I have added that. Still working on the details of updating a host/service record.

[ShawnMcKee]

Any update? We are seeing MOST of our hosts in psconfig.opensciencegrid.org as STALE.

[apertome]

I have a fix for this, testing in a few places

[apertome]

Fix seems to work, closing this

[ShawnMcKee]

Hi Michael, Do I need to grab new code? We are still seeing lots of Stale hosts. (See attached) Shawn

…

On Tue, Nov 6, 2018 at 2:04 PM Michael Johnson ***@***.***> wrote: Closed #60 <#60>. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#60 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AC7JZnxgYMxM8jQB06oKoTHz0Ij_Qy4Aks5useskgaJpZM4VrA-p> .

[apertome]

Hmm, it's working everywhere I'm running it. I don't see any attachments. - Michael

…

On Tue, Nov 06, 2018 at 08:12:02PM +0000, Shawn McKee wrote: Hi Michael, Do I need to grab new code? We are still seeing lots of Stale hosts. (See attached) Shawn On Tue, Nov 6, 2018 at 2:04 PM Michael Johnson ***@***.***> wrote: > Closed #60 <#60>. > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#60 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AC7JZnxgYMxM8jQB06oKoTHz0Ij_Qy4Aks5useskgaJpZM4VrA-p> > . > -- You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub: #60 (comment)

-- Michael Johnson GlobalNOC DevOps Engineer mj82@globalnoc.iu.edu

[apertome]

Hi Shawn, Looking at your PWA instance, it looks like all the hosts that are marked "Stale" are members of one or more hostgroups, but no longer in the SLS. This is by design; these records are not expired so that they won't be removed from your meshes. Perhaps we could do something differently, like change them to Ad-Hoc hosts when this happens ... or some other change. What do you think? - Michael

…

On Tue, Nov 06, 2018 at 08:12:02PM +0000, Shawn McKee wrote: Hi Michael, Do I need to grab new code? We are still seeing lots of Stale hosts. (See attached) Shawn On Tue, Nov 6, 2018 at 2:04 PM Michael Johnson ***@***.***> wrote: > Closed #60 <#60>. > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <#60 (comment)>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AC7JZnxgYMxM8jQB06oKoTHz0Ij_Qy4Aks5useskgaJpZM4VrA-p> > . > -- You are receiving this because you modified the open/close state. Reply to this email directly or view it on GitHub: #60 (comment)

-- Michael Johnson GlobalNOC DevOps Engineer mj82@globalnoc.iu.edu