-
Notifications
You must be signed in to change notification settings - Fork 2.6k
fix: assign new_nodes of service discovery to upstream every time #12358
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Nic <qianyong@api7.ai>
Signed-off-by: Nic <qianyong@api7.ai>
f86626a
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
apisix/utils/upstream.lua
Outdated
core.log.debug("compare upstream nodes by value, ", | ||
"old: ", tostring(old_t) , " ", core.json.delay_encode(old_t, true), | ||
", new: ", tostring(new_t) , " ", core.json.delay_encode(new_t, true)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Call delay_encode
once for each log record, otherwise the content of the log records will be garbled.
core.log.debug("compare upstream nodes by value, ", | |
"old: ", tostring(old_t) , " ", core.json.delay_encode(old_t, true), | |
", new: ", tostring(new_t) , " ", core.json.delay_encode(new_t, true)) | |
core.log.debug("compare upstream nodes by value, ", | |
"old: ", tostring(old_t) , " ", core.json.delay_encode(old_t, true)) | |
core.log.debug(" new: ", tostring(new_t) , " ", core.json.delay_encode(new_t, true)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
Description
I introduced a performance optimization in #12258, and after testing, we found that the kubernetes performance optimization would invalid after some time.
The reason is that after watch progress is done in kubernetes service discovery for some time, it will re-list all data. At this point, addresses of all tables that store service discovery are changed, but the content of nodes does not change. This causes
up_conf.nodes
to never reference thenew_nodes
table atapisix/apisix/upstream.lua
Lines 305 to 317 in 9e5163c
This branch has significant performance overhead:
apisix/apisix/utils/upstream.lua
Lines 50 to 61 in 9e5163c
Checklist