help request: can i modify the config_etcd.lua's nginx timer?

[Nobilta]

Description

my apisix's etcd cluster have to many watch request(6000 connections per second),so i want to reduce the pressure of etcd cluster.if i can modify the config_etcd.lua's timer of new function timer from 0 to 0.5 or 1 like this?(i have a test,and it can reduce the number of watch useful ,but i don't know if this can due to a unexpected error)or if have better method to reduce watch?

function _M.new(key, opts)

   ...

    if automatic then
        if not key then
            return nil, "missing `key` argument"
        end

        if loaded_configuration[key] then
            local res = loaded_configuration[key]
            loaded_configuration[key] = nil -- tried to load

            log.notice("use loaded configuration ", key)

            local dir_res, headers = res.body, res.headers
            load_full_data(obj, dir_res, headers)
        end

        ngx_timer_at(1, _automatic_fetch, obj)

    else
        local etcd_cli, err = get_etcd()
        if not etcd_cli then
            return nil, "failed to start a etcd instance: " .. err
        end
        obj.etcd_cli = etcd_cli
    end

    if key then
        created_obj[key] = obj
    end

    return obj
end

Environment

• APISIX version (run apisix version):2.15
• Operating system (run uname -a):
• OpenResty / Nginx version (run openresty -V or nginx -V):
• etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info):3.5
• APISIX Dashboard version, if relevant:
• Plugin runner version, for issues related to plugin runners:
• LuaRocks version, for installation issues (run luarocks --version):

[monkeyDluffy6017]

I don't think your change will work, you just delay the action of fetch data, not the period of fetch data, it don't reduce the number of requests to etcd.
Could you tell me why you have so many requests to etcd?

[Nobilta]

I don't think your change will work, you just delay the action of fetch data, not the period of fetch data, it don't reduce the number of requests to etcd. Could you tell me why you have so many requests to etcd?

@monkeyDluffy6017 we have many apisix gateway(up to 20),and we just have 3 etcd server(as a cluster),so the pressure of etcd is too high.so do you have a better solution of this condition?

[monkeyDluffy6017]

How many workers does a apisix have ?
How many resources are you watching ?

[Nobilta]

How many workers does a apisix have ? How many resources are you watching ?

in future,we will expand our apisix cluster triple,so the etcd's pressure will be more,and i worry the Stablize of etcd cluster

[tokers]

How many workers does a apisix have ? How many resources are you watching ?

in future,we will expand our apisix cluster triple,so the etcd's pressure will be more,and i worry the Stablize of etcd cluster

A better way is adding another intermediate component to convert the http connections to grpc connections.And it's stateless so that you can scale it. You may need to use gRPC gateway feature for it .

[monkeyDluffy6017]

modify the config_etcd.lua's timer of new function timer from 0 to 0.5 or 1

How did you test ？Does this really help? @Nobilta

[Nobilta]

modify the config_etcd.lua's timer of new function timer from 0 to 0.5 or 1

How did you test ？Does this really help? @Nobilta

in our test ,although the total number of request to etcd is not reduce,the number indeed reduced in the unit time(same request but they use more time,so in the unit time,request has be reduced).so if we can came to a conclusion that it was useful and will not due to unpredictable result?

[monkeyDluffy6017]

Could you try to use a random time instead of 0.5 or 1 ? all requests was sent at the same time may have more pressure on etcd

[Nobilta]

Could you try to use a random time instead of 0.5 or 1 ? all requests was sent at the same time may have more pressure on etcd

good suggestion.we will do more test,thank you very much!

[monkeyDluffy6017]

@Nobilta Can we talk with live chat software? we are interested in your etcd problem.