Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support stdin target via flag instead of automatic detection. #1935

Merged
merged 1 commit into from Apr 15, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
10 changes: 5 additions & 5 deletions docs/clients/promtail/troubleshooting.md
Expand Up @@ -13,22 +13,22 @@ In dry run mode, Promtail still support reading from a [positions](configuration
To start Promtail in dry run mode use the flag `--dry-run` as shown in the example below:

```bash
cat my.log | promtail --dry-run --client.url http://127.0.0.1:3100/loki/api/v1/push
cat my.log | promtail --stdin --dry-run --client.url http://127.0.0.1:3100/loki/api/v1/push
```

## Pipe data to Promtail

Promtail supports piping data for sending logs to Loki. This is a very useful way to troubleshooting your configuration.
Promtail supports piping data for sending logs to Loki (via the flag `--stdin`). This is a very useful way to troubleshooting your configuration.
Once you have promtail installed you can for instance use the following command to send logs to a local Loki instance:

```bash
cat my.log | promtail --client.url http://127.0.0.1:3100/loki/api/v1/push
cat my.log | promtail --stdin --client.url http://127.0.0.1:3100/loki/api/v1/push
```

You can also add additional labels from command line using:

```bash
cat my.log | promtail --client.url http://127.0.0.1:3100/loki/api/v1/push --client.external-labels=k1=v1,k2=v2
cat my.log | promtail --stdin --client.url http://127.0.0.1:3100/loki/api/v1/push --client.external-labels=k1=v1,k2=v2
```

This will add labels `k1` and `k2` with respective values `v1` and `v2`.
Expand Down Expand Up @@ -122,7 +122,7 @@ The following table shows an example of the total delay applied by the backoff a
with `minbackoff: 100ms` and `maxbackoff: 10s`:

| Retry | Min delay | Max delay | Total min delay | Total max delay |
| ----- | --------- | --------- | --------------- | --------------- |
|-------|-----------|-----------|-----------------|-----------------|
| 1 | 100ms | 200ms | 100ms | 200ms |
| 2 | 200ms | 400ms | 300ms | 600ms |
| 3 | 400ms | 800ms | 700ms | 1.4s |
Expand Down
2 changes: 2 additions & 0 deletions pkg/promtail/targets/filetarget.go
Expand Up @@ -56,11 +56,13 @@ const (
// Config describes behavior for Target
type Config struct {
SyncPeriod time.Duration `yaml:"sync_period"`
Stdin bool `yaml:"stdin"`
}

// RegisterFlags register flags.
func (cfg *Config) RegisterFlags(flags *flag.FlagSet) {
flags.DurationVar(&cfg.SyncPeriod, "target.sync-period", 10*time.Second, "Period to resync directories being watched and files being tailed.")
flags.BoolVar(&cfg.Stdin, "stdin", false, "Set to true to pipe logs to promtail.")
}

// FileTarget describes a particular set of logs.
Expand Down
4 changes: 2 additions & 2 deletions pkg/promtail/targets/manager.go
Expand Up @@ -38,8 +38,8 @@ func NewTargetManagers(
var journalScrapeConfigs []scrape.Config
var syslogScrapeConfigs []scrape.Config

if isStdinPipe() {
level.Debug(util.Logger).Log("msg", "detected pipe from stdin")
if targetConfig.Stdin {
level.Debug(util.Logger).Log("msg", "configured to read from stdin")
stdin, err := newStdinTargetManager(app, client, scrapeConfigs)
if err != nil {
return nil, err
Expand Down
9 changes: 0 additions & 9 deletions pkg/promtail/targets/stdin_target_manager.go
Expand Up @@ -47,15 +47,6 @@ var (
}
)

func isStdinPipe() bool {
info, err := stdIn.Stat()
if err != nil {
level.Warn(util.Logger).Log("err", err)
return false
}
return info.Mode()&os.ModeCharDevice == 0
}

type Shutdownable interface {
Shutdown()
}
Expand Down
18 changes: 0 additions & 18 deletions pkg/promtail/targets/stdin_target_manager_test.go
Expand Up @@ -148,24 +148,6 @@ func Test_StdinConfigs(t *testing.T) {
require.Equal(t, defaultStdInCfg, getStdinConfig([]scrape.Config{}))
}

type mockFileInfo struct{}

func (mockFileInfo) Name() string { return "" }
func (mockFileInfo) Size() int64 { return 1 }
func (mockFileInfo) Mode() os.FileMode { return 1 }
func (mockFileInfo) ModTime() time.Time { return time.Now() }
func (mockFileInfo) Sys() interface{} { return nil }
func (mockFileInfo) IsDir() bool { return false }

func Test_isPipe(t *testing.T) {
fake := newFakeStin("line")
fake.FileInfo = &mockFileInfo{}
stdIn = fake
require.Equal(t, true, isStdinPipe())
stdIn = os.Stdin
require.Equal(t, false, isStdinPipe())
}

var stagesConfig = `
pipeline_stages:
- template:
Expand Down