Summary
Enabling all_tables true on the sql input crashes during #start. SQLInput#start builds each auto-discovered table by passing a plain Hash to TableElement#configure, but Fluent::Configurable#configure expects a Fluent::Config::Element. Every discovered table therefore raises NoMethodError: undefined method 'corresponding_proxies' for ...Hash, and because this happens inside an unrescued map block, the whole start fails.
As far as I can tell all_tables has been broken for the entire 2.x line — this looks like a regression from the v0.14+ plugin API migration, where Configurable#configure stopped accepting a raw Hash.
Environment
- fluent-plugin-sql: 2.3.1 (also reproduced on current
master)
- fluentd: 1.17.1
- activerecord: 6.1.7.x
- ruby: 3.2.5
Steps to reproduce
Minimal, no database required:
require "fluent/plugin/in_sql"
te = Fluent::Plugin::SQLInput::TableElement.new
te.configure({ "table" => "foo", "tag" => "foo", "update_column" => nil })
Real-world trigger — any config using all_tables against a database with at least one table:
<source>
@type sql
host localhost
adapter postgresql
database appdb
all_tables true
</source>
You can reproduce the issue with fluent-plugin-sql-issue.tar.gz
Actual behavior
NoMethodError: undefined method 'corresponding_proxies' for an instance of Hash
.../fluent-plugin-sql-2.3.1/lib/fluent/plugin/in_sql.rb:222:in 'block in start'
.../fluent-plugin-sql-2.3.1/lib/fluent/plugin/in_sql.rb:216:in 'start'
Expected behavior
all_tables true discovers the tables and starts tailing them (using the primary key as update_column when none is configured).
Root cause
in_sql.rb (the all_tables branch) passes a Hash to TableElement#configure:
te.configure({
'table' => table_name,
'tag' => table_name,
'update_column' => nil,
})
TableElement#configure calls super, i.e. Fluent::Configurable#configure, which calls conf.corresponding_proxies — a method that exists on Fluent::Config::Element, not on Hash.
Suggested fix
Wrap the attributes in a Fluent::Config::Element (mirroring how the top-level configure passes real config elements):
te = TableElement.new
- te.configure({
- 'table' => table_name,
- 'tag' => table_name,
- 'update_column' => nil,
- })
+ te.configure(Fluent::Config::Element.new('table', '', {
+ 'table' => table_name,
+ 'tag' => table_name,
+ }, []))
te
(update_column defaults to nil, so it can be dropped.) I'm happy to open a PR with this fix and a regression test.
Summary
Enabling
all_tables trueon thesqlinput crashes during#start.SQLInput#startbuilds each auto-discovered table by passing a plainHashtoTableElement#configure, butFluent::Configurable#configureexpects aFluent::Config::Element. Every discovered table therefore raisesNoMethodError: undefined method 'corresponding_proxies' for ...Hash, and because this happens inside an unrescuedmapblock, the wholestartfails.As far as I can tell
all_tableshas been broken for the entire 2.x line — this looks like a regression from the v0.14+ plugin API migration, whereConfigurable#configurestopped accepting a rawHash.Environment
master)Steps to reproduce
Minimal, no database required:
Real-world trigger — any config using
all_tablesagainst a database with at least one table:You can reproduce the issue with fluent-plugin-sql-issue.tar.gz
Actual behavior
Expected behavior
all_tables truediscovers the tables and starts tailing them (using the primary key asupdate_columnwhen none is configured).Root cause
in_sql.rb(theall_tablesbranch) passes aHashtoTableElement#configure:TableElement#configurecallssuper, i.e.Fluent::Configurable#configure, which callsconf.corresponding_proxies— a method that exists onFluent::Config::Element, not onHash.Suggested fix
Wrap the attributes in a
Fluent::Config::Element(mirroring how the top-levelconfigurepasses real config elements):(
update_columndefaults tonil, so it can be dropped.) I'm happy to open a PR with this fix and a regression test.