loss of data

[tracyliuzw]

Environments

• fluentd version:1.5.2
• plugin version:3.0.0

Configuration

Expected Behavior

Actual Behavior

Log (if you have)

I often encounter data loss in the process of using. The error log is as follows:
2022-11-20 22:44:07 -0500 [debug]: #2 Created new chunk chunk_id="5edf2de1add7ed91680a7b7b796c5a84" metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag=nil, variables={:topic=>nil}, seq=0>
2022-11-20 22:44:07 -0500 [debug]: #5 Created new chunk chunk_id="5edf2de1ae64c54bfa55d3b729307635" metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag=nil, variables={:topic=>nil}, seq=0>
2022-11-20 22:44:07 -0500 [debug]: #0 Created new chunk chunk_id="5edf2de1b684d9f971632dd4ddb60b42" metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag=nil, variables={:topic=>nil}, seq=0>
2022-11-20 22:44:07 -0500 [debug]: #0 64 messages send.
2022-11-20 22:44:08 -0500 [warn]: #0 insert errors project_id="doomsday-last-survivors-309306" dataset="gamedata" table="event_gamelog_1120" insert_errors="[#<Google::Apis::BigqueryV2::InsertAllTableDataResponse::InsertError:0x00007fa01df22d48 @errors=[#<Google::Apis::BigqueryV2::ErrorProto:0x00007fa01df22370 @debug_info="", @location="event_time", @message="Invalid datetime string \"20827388\"", @Reason="invalid">], @Index=264>, #<Google::Apis::BigqueryV2::InsertAllTableDataResponse::InsertError:0x00007fa01df20480 @errors=[#<Google::Apis::BigqueryV2::ErrorProto:0x00007fa01df1fa58 @debug_info="", @location="event_time", @message="Invalid datetime string \"20827388\"", @Reason="invalid">], @Index=266>, #<Google::Apis::BigqueryV2::InsertAllTableDataResponse::InsertError:0x00007fa01df1dca8 @errors=[#<Google::Apis::BigqueryV2::ErrorProto:0x00007fa01df1d2a8 @debug_info="", @location="event_time", @message="Invalid datetime string \"20827388\"", @Reason="invalid">], @Index=268>, #<Google::Apis::BigqueryV2::InsertAllTableDataResponse::InsertError:0x00007fa01df1b4f8 @errors=[#<Google::Apis::BigqueryV2::ErrorProto:0x00007fa01df1aa58 @debug_info="", @location="event_time", @message="Invalid datetime string \"20827388\"", @Reason="invalid">], @Index=270>, #<Google::Apis::BigqueryV2::InsertAllTableDataResponse::InsertError:0x00007fa01df18c80 @errors=[#<Google::Apis::BigqueryV2::ErrorProto:0x00007fa01df18258 @debug_info="", @location="event_time", @message="Invalid datetime string \"20827388\"", @Reason="invalid">], @Index=272>, #<Google::Apis::BigqueryV2::InsertAllTableDataResponse::InsertError:0x00007fa01df164a8 @errors=[#<Google::Apis::BigqueryV2::ErrorProto:0x00007fa01df15ad0 @debug_info="", @location="event_time", @message="Invalid datetime string \"20827388\"", @Reason="invalid">], @Index=330>, #<Google::Apis::BigqueryV2::InsertAllTableDataResponse::InsertError:0x00007fa01df07a98 @errors=[#<Google::Apis::BigqueryV2::ErrorProto:0x00007fa01df070c0 @debug_info="", @location="event_time", @message="Invalid datetime string \"20827388\"", @Reason="invalid">], @Index=332>, #<Google::Apis::BigqueryV2::InsertAllTableDataResponse::InsertError:0x00007fa01df05270 @errors=[#<Google::Apis::BigqueryV2::ErrorProto:0x00007fa01df04848 @debug_info="", @location="event_time", @message="Invalid datetime string \"20827388\"", @Reason="invalid">], @Index=334>, #<Google::Apis::BigqueryV2::InsertAllTableDataResponse::InsertError:0x00007fa01df02778 @errors=[#<Google::Apis::BigqueryV2::ErrorProto:0x00007fa01df01d78 @debug_info="", @location="event_time", @message="Invalid datetime string \"20827388\"", @Reason="invalid">], @Index=336>, #<Google::Apis::BigqueryV2::InsertAllTableDataResponse::InsertError:0x00007fa01defd228 @errors=[#<Google::Apis::BigqueryV2::ErrorProto:0x00007fa01defbdd8 @debug_info="", @location="event_time", @message="Invalid datetime string \"20827388\"", @Reason="invalid">], @Index=338>]"
2022-11-20 22:44:08 -0500 [error]: #0 Hit limit for retries. dropping all chunks in the buffer queue. retry_times=0 records=414 error_class=Fluent::BigQuery::UnRetryableError error="failed to insert into bigquery(insert errors), and cannot retry"
2022-11-20 22:44:08 -0500 [error]: #0 suppressed same stacktrace
2022-11-20 22:44:08 -0500 [debug]: #1 Created new chunk chunk_id="5edf2de293c526e9e99cf5231cc871f7" metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag=nil, variables={:topic=>nil}, seq=0>
2022-11-20 22:44:08 -0500 [debug]: #3 119 messages send.
2022-11-20 22:44:08 -0500 [debug]: #1 170 messages send.
2022-11-20 22:44:08 -0500 [debug]: #3 Created new chunk chunk_id="5edf2de298c52e870444bb87caadf0ce" metadata=#<struct Fluent::Plugin::Buffer::Metadata timekey=nil, tag=nil, variables={:topic=>nil}, seq=0>