Skip to content

all_tables true crashes on startup: NoMethodError: undefined method 'corresponding_proxies' for Hash #177

Description

@Watson1978

Summary

Enabling all_tables true on the sql input crashes during #start. SQLInput#start builds each auto-discovered table by passing a plain Hash to TableElement#configure, but Fluent::Configurable#configure expects a Fluent::Config::Element. Every discovered table therefore raises NoMethodError: undefined method 'corresponding_proxies' for ...Hash, and because this happens inside an unrescued map block, the whole start fails.

As far as I can tell all_tables has been broken for the entire 2.x line — this looks like a regression from the v0.14+ plugin API migration, where Configurable#configure stopped accepting a raw Hash.

Environment

  • fluent-plugin-sql: 2.3.1 (also reproduced on current master)
  • fluentd: 1.17.1
  • activerecord: 6.1.7.x
  • ruby: 3.2.5

Steps to reproduce

Minimal, no database required:

require "fluent/plugin/in_sql"

te = Fluent::Plugin::SQLInput::TableElement.new
te.configure({ "table" => "foo", "tag" => "foo", "update_column" => nil })

Real-world trigger — any config using all_tables against a database with at least one table:

<source>
  @type sql
  host localhost
  adapter postgresql
  database appdb
  all_tables true
</source>

You can reproduce the issue with fluent-plugin-sql-issue.tar.gz

Actual behavior

NoMethodError: undefined method 'corresponding_proxies' for an instance of Hash
    .../fluent-plugin-sql-2.3.1/lib/fluent/plugin/in_sql.rb:222:in 'block in start'
    .../fluent-plugin-sql-2.3.1/lib/fluent/plugin/in_sql.rb:216:in 'start'

Expected behavior

all_tables true discovers the tables and starts tailing them (using the primary key as update_column when none is configured).

Root cause

in_sql.rb (the all_tables branch) passes a Hash to TableElement#configure:

te.configure({
  'table' => table_name,
  'tag' => table_name,
  'update_column' => nil,
})

TableElement#configure calls super, i.e. Fluent::Configurable#configure, which calls conf.corresponding_proxies — a method that exists on Fluent::Config::Element, not on Hash.

Suggested fix

Wrap the attributes in a Fluent::Config::Element (mirroring how the top-level configure passes real config elements):

             te = TableElement.new
-            te.configure({
-              'table' => table_name,
-              'tag' => table_name,
-              'update_column' => nil,
-            })
+            te.configure(Fluent::Config::Element.new('table', '', {
+              'table' => table_name,
+              'tag' => table_name,
+            }, []))
             te

(update_column defaults to nil, so it can be dropped.) I'm happy to open a PR with this fix and a regression test.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions