[receiver/promtailreceiver] Add promtail receiver

[mar4uk]

Description:

Added new promtail receiver component

Link to tracking Issue:
#9800

Documentation:

Currently, this receiver supports scrape_config static_configs and scrape_configs loki_push_api. I didn't check other scrape_config targets (syslog, kubernetes, window_events), some additional fixes are probably needed to make them work
The config file will look like this:

receivers:
  promtail:
   config:
    positions:
      filename: /tmp/positions.yaml
      sync_period: 10s
    scrape_configs:
      - job_name: system
        static_configs:
          - labels:
              job: varlogs
              __path__: /var/log/example.log
      - job_name: push
        loki_push_api:
          server:
            http_listen_port: 3101
            grpc_listen_port: 3600
          labels:
            pushserver: push1
          use_incoming_timestamp: true
    target_config:
      sync_period: 10s

processors:
  batch:

exporters:
  logging:
    logLevel: debug
  loki:
    endpoint: "http://localhost:3100/loki/api/v1/push"
    labels:
      attributes:
        pushserver:
        foo:

service:
  pipelines:
    logs:
      receivers: [ promtail ]
      processors: [ batch ]
      exporters: [ logging, loki ]

To send logs to loki via collector we can run curl:

curl -i -XPOST -H "Content-Type: application/json" "http://localhost:3101/loki/api/v1/push" -d '{
  "streams": [
    {
      "stream": {
        "foo": "bar"
      },
      "values": [
          [ "1652782740000000000", "def" ]
      ]
    }
  ]
}'

To send logs to loki via collector we tail file example.log. It could be tested with command:

echo "May 04 11:45:56 Irinas-MBP login[87256]: USER_PROCESS: 87256 ttys012" >> example.log

The initial draft PR was closed because of uncertainty with the license. According to https://github.com/grafana/loki/blob/main/LICENSING.md#apache-20 clients/ is under Apache-2.0 license and we can use its code in collector-contrib

Important
PR grafana/loki#7320 should be merged first, it contains mapstructure support in promtail

[djaglowski]

This generally looks good. I have one big ask regarding where the code resides, and otherwise just a few minor things.

[mar4uk]

looking into failed checks..

[mar4uk]

@djaglowski could you please take another look?

[jpkrohling]

Thanks for this PR! I had a few questions but looks good.

[jpkrohling]

Could you add a comment stating that this is required because of the promtail receiver component? I read your comment that this is required for compiling this component. In that case, would the whole build fail if this component isn't present?

[mar4uk]

By promtail receiver component isn't present you mean the repo doesn't have it in the receiver directory as well as all its mentions in other places like internal/components.go, internal/components_test.go, versions.yaml. Basically how it is now in the main branch, right?

If so the build will pass with or without installing libsystemd-dev.

[jpkrohling]

We could isolate the code that depends on libsystemd-dev on a specific source, which is used only when a specific build flag is passed. The makefile could then check whether this lib exists and set the appropriate build flag. For our CI, we should always enable this, but I wouldn't require people on Mac or Windows systems to have this library to build contrib locally.

This brings another question: would the promtail receiver work on Windows and Darwin builds?

[mar4uk]

Promtail receiver will work on Windows and Darwin builds because journal target is built only for linux and when cgo is enabled: https://github.com/grafana/loki/blob/488e64e3ff1bbce70d856c3daed150b061c5614d/clients/pkg/promtail/targets/journal/journaltarget.go#L1-L2

journal target will be ignored by non-linux OS and the collector build will be successful

We could isolate the code that depends on libsystemd-dev on a specific source, which is used only when a specific build flag is passed. The makefile could then check whether this lib exists and set the appropriate build flag.

promtail depends on libsystemd-dev, and promtail receiver depends on promtail. Could you please reveal the idea in more detail? What do you mean by a specific flag? Do you mean to have a custom build flag like linux-with-libsystemd-dev? To make it work I will need to add comment //go:build linux-with-libsystemd-dev to all promtail receiver files, right? And to run build we will pass flag go build -tags linux-with-libsystemd-dev That will work. Or if Loki team is agree we can add specfic flag linux-with-libsystemd-dev to https://github.com/grafana/loki/blob/main/clients/pkg/promtail/targets/journal/journaltarget.go to isolate that code and exclude it from compilation in case linux without libsystemd-dev. It will allow exclude exactly journald part from promtail without excluding the whole promtail receiver

[mar4uk]

my last comment could be ignored

[jpkrohling]

Looks like the Loki team does not need this PR anymore. Can you check with them what the status is on this? The replace statement is a good temporary solution, but if that PR isn't being merged, this isn't temporary anymore.

[mar4uk]

The Loki team needs the support of LogpullReceived and LogpullFields for Loki right now. They are going to switch to Logpush in the future, but currently, this replace is needed for Loki

[jpkrohling]

Is this library abandoned? No commits since January, no commits in 2021 or 2020, and I see that you haven't received any answers on your PR yet :-/

[mar4uk]

yeah, the library looks dead :(

[mar4uk]

I received answers and did fixes. Waiting for the second round of review. Will keep an eye on this

[jpkrohling]

Do we need this?

[mar4uk]

without this comment, golangci-lint throws an error:

error-return: error should be the last type when returning multiple items (revive)

This code isn't mine actually, I copied it from prometheusreceiver/internal/logger.go and didn't refactor it.
I was thinking to move this code somewhere outside the components and import to promtail and prometheus receivers, but not sure if it is a good approach.
If I move the logger somewhere (to internal directory maybe?) it will be there even if promtail and prometheus components are not present in the custom setup of collector contrib, so it will be an extra code user have to have that is not used anywhere

[jpkrohling]

LGTM, thank you @mar4uk!