fluent big - output bigquery - cannot increase buffer and coult not create client socket, retrying issue

[clover1983]

Bug Report

Describe the bug
Hi fluent bit support team @Syn3rman
I am using syslog-ng agent and fowarding the logs to fluent - bit (host at GCE), then fluent try to write logs to bigquery.

the write to bigquery is never succ, keep failed. And I get the below error

[2023/05/27 00:10:16] [error] [net] coult not create client socket, retrying
[2023/05/27 00:10:16] [error] [net] coult not create client socket, retrying
[2023/05/27 00:10:16] [error] [net] coult not create client socket, retrying
[2023/05/27 00:10:16] [error] [net] coult not create client socket, retrying
[2023/05/27 00:10:17] [ warn] [http_client] cannot increase buffer: current=36960 requested=69728 max=36960
[2023/05/27 00:10:17] [ warn] [http_client] cannot increase buffer: current=36960 requested=69728 max=36960
[2023/05/27 00:10:18] [ warn] [http_client] cannot increase buffer: current=36960 requested=69728 max=36960
[2023/05/27 00:10:18] [ warn] [http_client] cannot increase buffer: current=36960 requested=69728 max=36960
fluent bit - bigquery output[2023/05/27 00:10:19] [ warn] [http_client] cannot increase buffer: current=36960 requested=69728 max=36960
[2023/05/27 00:10:19] [ warn] [http_client] cannot increase buffer: current=36960 requested=69728 max=36960
[2023/05/27 00:10:20] [ warn] [http_client] cannot increase buffer: current=36960 requested=69728 max=36960
[2023/05/27 00:10:20] [ warn] [http_client] cannot increase buffer: current=36960 requested=69728 max=36960
[2023/05/27 00:10:21] [ warn] [http_client] cannot increase buffer: current=36960 requested=69728 max=36960
[2023/05/27 00:10:21] [ warn] [http_client] cannot increase buffer: current=36960 requested=69728 max=36960
[2023/05/27 00:10:22] [ warn] [http_client] cannot increase buffer: current=36960 requested=69728 max=36960
[2023/05/27 00:10:22] [ warn] [http_client] cannot increase buffer: current=36960 requested=69728 max=36960

This is I compile from fluent bit - source. I modify the origin code (plugin/out_bigquery/bigquery.c) line 1017

flb_http_buffer_size(c, 4192)

change 4192 to 36960, but I still getting the above error (4192 have same error)

I try all the way and I don't get the root cause. Would you mind to help?

Thank you so much really appreciate your help and looking forward to hear from you.

To Reproduce

• my fluent-bit config:

cat << EOF > $ParsePath
[PARSER]
    Name   syslog-rfc5424
    Format regex
    Regex  /\A^\d+ (?<time>[^ ]+) (?<host>[^ ]+) (?<ident>[^ ]+) (?<pid>[^ ]+) (?<msgid>[^ ]+) (?<message>.+) \[ELMS_BIGQUERY_TAG\](?<extradata>.+)$\z/m
    Time_Key time
    Time_Format %Y-%m-%dT%H:%M:%S%z
EOF

 

cat << EOF > $logConfPath
[SERVICE]
    Flush                         1
    Daemon                        Off
    Log_File                      /var/log/fluent-bit.log
    Log_Level                     info
    Buffer_Workers                8
    storage.path                  /flb-storage/
    storage.sync                  normal
    storage.checksum              off
    storage.max_chunks_up         128
    storage.backlog.mem_limit     100M
    Parsers_File                  parsers.conf

 

[INPUT]
    Name                          tcp
    Listen                        0.0.0.0
    Port                          5140
    #storage.type                  filesystem
    Format                        none
    Buffer_Size                   100M

#[INPUT]
#    Name                          syslog
#    Parser                        syslog-rfc5424
#    Listen                        0.0.0.0
#    Port                          5140
#    Mode                          tcp
#    Buffer_Chunk_Size             4000

[INPUT]
    Name                           tcp
    Listen                         0.0.0.0
    Port                           8118
    #storage.type                   filesystem
    Format                         none

 

[INPUT]
    name                           fluentbit_metrics
    tag                            internal_metrics
    scrape_interval                15

 

[FILTER]
    Name                           parser
    Match                          *
    Parser                         syslog-rfc5424
    Key_Name                       message

 

[OUTPUT]
    Name         stdout
    Match        *

 

[OUTPUT]
    Name                           bigquery
    google_service_account         xxx@xxx.iam.gserviceaccount.com
    Match                          *
    project_id                     xxx
    dataset_id                     xxx
    table_id                       logs
    google_service_credentials     /tmp/xxx.json
    #storage.total_limit_size       100M

 

[OUTPUT]
    name                           prometheus_exporter
    match                          internal_metrics
    host                           0.0.0.0
    port                           24231
EOF

Expected behavior
Log succ write to bigquery table

Screenshots

Your Environment

• Version used: v2.1.4
• Configuration: as above
• Environment name and version (e.g. Kubernetes? What version?): syslog-ng PE 7.0.23, fluent-bit v2.1.4, bigquery
• Server type and version: Server host at GCE
• Operating System and version: RHEL7.9
• Filters and plugins: Parser and Bigquery Output

Additional context

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

[github-actions]

This issue was closed because it has been stalled for 5 days with no activity.