Skip to content

Forwarding from fluent bit to fluentd : Command Line working but not via config file #2191

@Shawn5141

Description

@Shawn5141

Bug Report

Describe the bug
When I tried to forward logs via fluent-bit. I used config file looks like below config file by running /opt/td-agent-bit/bin/td-agent-bit -c conf/fluentbit.conf

[SERVICE]
   
    Flush        5

    # Daemon
    # ======
    # Instruct Fluent Bit to run in foreground or background mode.
    Daemon       Off

    # Log_Level
    # =========
    # Set the verbosity level of the service, values can be:
  
    # By default 'info' is set, that means it includes 'error' and 'warning'.
    Log_Level    info

    # Parsers_File
    # ============
    # Specify an optional 'Parsers' configuration file
    # Parsers_File parsers.conf
    # Plugins_File plugins.conf

    # HTTP Server
    # ===========
    # Enable/Disable the built-in HTTP Server for metrics
    HTTP_Server  On
    HTTP_Listen  0.0.0.0
    HTTP_Port    2020

[INPUT]
    Name        tail
    Tag  log.local
    Path     /var/log/log_session.json #Need to think about how to make it generic
   
[OUTPUT]
    Name          forward
    Match         *
    Host          127.20.0.2
    Port          24224
   

my fluentd.conf

<source>
  @type  forward
  @id    input1
  @label @mainstream
  port  24224
</source>

<filter **>
  @type stdout
</filter>

<label @mainstream>
  <match docker.**>
    @type file
    @id   output_docker1
    path         /fluentd/log/docker.*.log
    symlink_path /fluentd/log/docker.log
    append       true
    time_slice_format %Y%m%d
    time_slice_wait   1m
    time_format       %Y%m%dT%H%M%S%z
  </match>
  <match **>
    @type file
    @id   output1
    path         /fluentd/log/data.*.log
    symlink_path /fluentd/log/data.log
    append       true
    time_slice_format %Y%m%d
    time_slice_wait   10m
    time_format       %Y%m%dT%H%M%S%z
  </match>
</label>

but it's not working.
Therefore, I tried to manually run command like this :
/opt/td-agent-bit/bin/td-agent-bit -i tail -p path=/var/log/heralding/log_session.json -o forward://172.20.0.2:24224
And magically it works. I wonder if there is anything in config file that I set wrong.
Thank you

To Reproduce

  • Example log message if applicable:
{"cpu: 8"}
{"cpu: 9"}
  • Steps to reproduce the problem:
  1. Because I am running on some private code, so I could not expose it here. Sorry about this.

Expected behavior
Using config file should also works

Screenshots
Running using command line

image

Running using config file
image

Your Environment

  • Version used: Fluent Bit v1.4.4
  • Configuration:
  • Environment name and version (e.g. Kubernetes? What version?):
  • Server type and version:
  • Operating System and version:
  • Filters and plugins:

Additional context

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions