fqdn: use map to dedup to reduce memory usage of dns gc job

[odinuge]

On some host we have a lot of DNS lookups, and this job allocates a lot of memory and burn a lot of cpu. We often see loglines like these; "FQDN garbage collector work deleted 16211 name entries"

This will now dedup using a map directly instead of deduping via merging slices. It doesn't seem like the code care about the order, so I think this should be fine. It would be nice to benchmark this, but its a bit difficult in the way the job is set up. Happy to iterate on that after we can test this.

Peek from a pprof, where this comes on top (running cilium v1.11.13);

Type: alloc_space
Showing nodes accounting for 24333619.24MB, 100% of 24333619.24MB total ----------------------------------------------------------+-------------
      flat  flat%   sum%        cum   cum%   calls calls% + context
----------------------------------------------------------+-------------
                                      4686095.18MB 95.24% |   github.com/cilium/cilium/daemon/cmd.(*Daemon).bootstrapFQDN.func1 /go/src/github.com/cilium/cilium/daemon/cmd/fqdn.go:236
                                       230562.07MB  4.69% |   github.com/cilium/cilium/daemon/cmd.(*Daemon).bootstrapFQDN.func1 /go/src/github.com/cilium/cilium/daemon/cmd/fqdn.go:246
                                         2609.33MB 0.053% |   github.com/cilium/cilium/pkg/fqdn.(*DNSCache).cleanupExpiredEntries /go/src/github.com/cilium/cilium/pkg/fqdn/cache.go:256
                                          997.05MB  0.02% |   github.com/cilium/cilium/daemon/cmd.(*Daemon).bootstrapFQDN.func1 /go/src/github.com/cilium/cilium/daemon/cmd/fqdn.go:250
                                          175.82MB 0.0036% |   github.com/cilium/cilium/pkg/fqdn.(*DNSCache).GC /go/src/github.com/cilium/cilium/pkg/fqdn/cache.go:339
4920439.45MB 20.22% 20.22% 4920439.45MB 20.22%                | github.com/cilium/cilium/pkg/fqdn.KeepUniqueNames /go/src/github.com/cilium/cilium/pkg/fqdn/helpers.go:127

Please ensure your pull request adheres to the following guidelines:

• For first time contributors, read Submitting a pull request
• All code is covered by unit and/or runtime tests where feasible.
• All commits contain a well written commit description including a title,
  description and a Fixes: #XXX line if the commit addresses a particular
  GitHub issue.
• If your commit description contains a Fixes: <commit-id> tag, then
  please add the commit author[s] as reviewer[s] to this issue.
• All commits are signed off. See the section Developer’s Certificate of Origin
• Provide a title or release-note blurb suitable for the release notes.
• Are you a user of Cilium? Please add yourself to the Users doc
• Thanks for contributing!

Fixes: #issue-number

fqdn: use map to dedup to reduce memory usage of dns gc job

[odinuge]

cc @christarazi mind triggering test to verify that the logic is sound?

[christarazi]

/test

Job 'Cilium-PR-K8s-1.26-kernel-4.19' failed:

Click to show.

Test Name

K8sAgentFQDNTest Validate that FQDN policy continues to work after being updated

Failure Output

FAIL: Cannot install fqdn proxy policy

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.26-kernel-4.19/169/

If it is a flake and a GitHub issue doesn't already exist to track it, comment /mlh new-flake Cilium-PR-K8s-1.26-kernel-4.19 so I can create one.

Then please upload the Jenkins artifacts to that issue.

[thorn3r]

@odinuge thanks for the PR! this seems like a pretty straight-forward improvement.
Would you mind squashing your commits and rebasing/fixing conflicts?

[odinuge]

Hi! Yes, I'll rebase and squash tomorrow morning. Thanks

[odinuge]

Should be good to go now thorn3r

[thorn3r]

/test

[thorn3r]

Looks great, thank you!