fix: If resource synchronization retry occurs, other events of the same resource will be blocked.

[gxthrj]

Please answer these questions before submitting a pull request

• Why submit this pull request?

• Bugfix

• New feature provided

• Improve performance

• Backport patches

• Related issues
  bug: If resource synchronization retry occurs, other events of the same resource will be blocked #759

---

Bugfix

• How to fix?

This bug is due to a workqueue shared under the same resource, and a ratelimit mechanism is added to this workqueue, but we only need to add the ratelimit when retrying fails, and when normal resource changes, we should immediately add the workqueue to be processed .

The new k8s event does not need to use the rate limit.

[codecov-commenter]

Codecov Report

Merging #760 (2751876) into master (4a862e2) will not change coverage.
The diff coverage is 0.00%.

@@           Coverage Diff           @@
##           master     #760   +/-   ##
=======================================
  Coverage   31.70%   31.70%           
=======================================
  Files          66       66           
  Lines        6640     6640           
=======================================
  Hits         2105     2105           
  Misses       4280     4280           
  Partials      255      255           

Impacted Files	Coverage Δ
pkg/ingress/apisix_cluster_config.go	0.00% <0.00%> (ø)
pkg/ingress/apisix_consumer.go	0.00% <0.00%> (ø)
pkg/ingress/apisix_route.go	0.00% <0.00%> (ø)
pkg/ingress/apisix_tls.go	0.00% <0.00%> (ø)
pkg/ingress/apisix_upstream.go	0.00% <0.00%> (ø)
pkg/ingress/endpoint.go	0.00% <0.00%> (ø)
pkg/ingress/endpointslice.go	0.00% <0.00%> (ø)
pkg/ingress/ingress.go	8.29% <0.00%> (ø)
pkg/ingress/namespace.go	0.00% <0.00%> (ø)
pkg/ingress/secret.go	0.00% <0.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 774077a...2751876. Read the comment docs.

[tokers]

@gxthrj Could you give some details about the fixup?

[gxthrj]

@gxthrj Could you give some details about the fixup?

This bug is due to a workqueue shared under the same resource, and a ratelimit mechanism is added to this workqueue, but we only need to add the ratelimit when retrying fails, and when normal resource changes, we should immediately add the workqueue to be processed .

The new k8s event does not need to use the rate limit.

[tangzhenhuang]

This fix cannot solve the problem in some scenarios, for example:
Suppose there are two instances of apisix-ingress-controller. When the leader goes down for some reason and another instance becomes the leader, the new leader will block the resource event due to "client-side throttling" during the list resource phase. Because apisix-ingress-controller has not completed the list stage at this time, the control loop will definitely be blocked

[tao12345666333]

@crazyMonkey1995 Thanks for your report. We can use #822 for tracking this one.