There seems to be a problem with the loading priority of declarative_config and LMDB cache file in hybrid mode

[zhangxuri198]

Is there an existing issue for this?

• I have searched the existing issues

Kong version ($ kong version)

Kong 3.9.0

Current Behavior

in file：kong/kong.conf.default line 1345

#declarative_config =           # The path to the declarative configuration
                                # file which holds the specification of all
                                # entities (routes, services, consumers, etc.)
                                # to be used when the `database` is set to
                                # `off`.
                                #
                                # Entities are stored in Kong's LMDB cache,
                                # so you must ensure that enough headroom is
                                # allocated to it via the `lmdb_map_size`
                                # property.
                                #
                                # If the hybrid mode `role` is set to `data_plane`
                                # and there's no configuration cache file,
                                # this configuration is used before connecting
                                # to the control plane node as a user-controlled
                                # fallback.

It says that LMDB cache file has higher priority than declarative_config. However, during my use, I found that declarative_config has a higher priority than LMDB cache file.

Expected Behavior

LMDB cache file has higher priority than declarative_config

Steps To Reproduce

1.Build a DP-CP hybrid deployment service and mount an expired declarative configuration file.
2.Kill the CP node and restart the DP node.
3.DP pod loads the configuration in the declarative configuration file instead of the configuration in the LMDB cache

Anything else?

I read some of the code and suspected that there was something wrong with the following logic.

in file：kong/kong/init.lua line 737

      declarative_entities, err, declarative_meta, declarative_hash =
        parse_declarative_config(kong.configuration, dc)

The declarative_entities variable is initialized

in file：kong/kong/init.lua line 924-960

    if not has_declarative_config(kong.configuration) and
      declarative.get_current_hash() ~= nil then
      -- if there is no declarative config set and a config is present in LMDB,
      -- just build the router and plugins iterator
      ngx_log(ngx_INFO, "found persisted lmdb config, loading...")
      local ok, err = declarative_init_build()
      if not ok then
        stash_init_worker_error("failed to initialize declarative config: " .. err)
        return
      end

    elseif declarative_entities then

      ok, err = load_declarative_config(kong.configuration,
                                        declarative_entities,
                                        declarative_meta,
                                        declarative_hash)

      declarative_entities = nil
      declarative_meta = nil
      declarative_hash = nil

      if not ok then
        stash_init_worker_error("failed to load declarative config file: " .. err)
        return
      end

    else
      -- stream does not need to load declarative config again, just build
      -- the router and plugins iterator
      local ok, err = declarative_init_build()
      if not ok then
        stash_init_worker_error("failed to initialize declarative config: " .. err)
        return
      end
    end
  end

The above judgment logic is that LMDB will be loaded first only when there is no declarative configuration file, which is inconsistent with the description in the file.

[StarlightIbuki]

Thanks for reporting and in-detail analysis.
You're right. The logic does not match the doc and the original design.
This is a bug and it seems to have been introduced when we changed to use LMDB: #8670
I'm creating a fix.

[StarlightIbuki]

Tracked at KAG-6852