Add the `logs` subcommand to the CLI #2745

rdner · 2023-05-30T10:17:45Z

What does this PR do?

It adds the logs subcommand.

Following is supported:

Print the last -n lines of the logs (10 by default)
Follow the logs after printing
Filter logs by a component ID

Why is it important?

Improves development and troubleshooting experience.

Checklist

My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
~~- [ ] I have made corresponding change to the default configuration files~~
I have added tests that prove my fix is effective or that my feature works
I have added an entry in ./changelog/fragments using the changelog tool
~~- [ ] I have added an integration test or an E2E test~~

How to test this PR locally

Unit tests with a very good coverage (all the successful paths and edge cases are covered)
Install the agent built from this branch and then use elastic-agent logs -f

Related issues

Fixes Implement a sub-command to show or follow the Elastic Agent logs #114

Screenshots

Screen.Recording.2023-05-30.at.15.27.23.mov

Following is supported: * Print the last `-n` lines of the logs (10 by default) * Follow the logs after printing * Filter logs by a component ID

elasticmachine · 2023-05-30T10:29:53Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-05-30T13:08:36.941+0000
Duration: 21 min 4 sec

Test stats 🧪

Test	Results
Failed	0
Passed	5803
Skipped	19
Total	5822

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.
/package : Generate the packages.
run integration tests : Run the Elastic Agent Integration tests.
run end-to-end tests : Generate the packages and run the E2E Tests.
run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2023-05-30T10:30:04Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	98.611% (`71/72`)	👍
Files	68.4% (`171/250`)	👍 0.127
Classes	67.161% (`317/472`)	👍 0.14
Methods	53.902% (`967/1794`)	👍 0.224
Lines	39.781% (`11200/28154`)	👍 0.261
Conditionals	100.0% (`0/0`)	💚

rdner · 2023-05-30T12:09:55Z

internal/pkg/agent/cmd/logs.go

+		filter = createComponentFilter(component)
+	}
+
+	logsDir := filepath.Join(paths.Home(), logger.DefaultLogDirectory)


First I tried to use:

elastic-agent/internal/pkg/agent/application/paths/common.go

Lines 163 to 166 in cda5b7e

// Logs returns the log directory for Agent

func Logs() string {

return logsPath

}

But (at least on Mac) it lead to /Library/Elastic/Agent which had some logs but they were not actively updated while the agent was running.

After some digging, I found this line and copied from here:

elastic-agent/pkg/core/logger/logger.go

Lines 139 to 145 in cda5b7e

func MakeInternalFileOutput(cfg *Config) (zapcore.Core, error) {

// defaultCfg is used to set the defaults for the file rotation of the internal logging

// these settings cannot be changed by a user configuration

defaultCfg := logp.DefaultConfig(logp.DefaultEnvironment)

filename := filepath.Join(paths.Home(), DefaultLogDirectory, cfg.Beat)

al := zap.NewAtomicLevelAt(cfg.Level.ZapLevel())

internalLevelEnabler = &al // directly persisting struct will panic on accessing unitialized backing pointer

This seems to be the correct path for logs.

Can anyone explain why the logs are written in 2 different locations and confirm that this is the correct way to get the path?

According to @blakerouse the current location is the right location to watch for logs.

pchila

Just an optional small suggestion, otherwise LGTM

pchila · 2023-05-31T12:21:06Z

internal/pkg/agent/cmd/logs.go

+				if !filter(line) {
+					continue
+				}
+				_, _ = w.Write(line)
+				_, _ = w.Write([]byte{'\n'})


Maybe we can flip the test here ?

Suggested change

if !filter(line) {

continue

}

_, _ = w.Write(line)

_, _ = w.Write([]byte{'\n'})

if filter(line) {

_, _ = w.Write(line)

_, _ = w.Write([]byte{'\n'})

}

I think I'm going to keep it as it is. I always prefer less indentation with a terminal if.

pchila · 2023-05-31T14:19:22Z

Suggestion for future dev: could we add a --color option to have colored output (using something like https://github.com/fatih/color) for the different log levels? It could be useful to spot at a glance error/warning logs mixed with other log levels ;)

michalpristas · 2023-06-05T07:54:11Z

tested on windows it works, all flags tested

Related issues: elastic/elastic-agent#2745 elastic/elastic-agent#3345

Add the logs subcommand to the CLI

389b3c6

Following is supported: * Print the last `-n` lines of the logs (10 by default) * Follow the logs after printing * Filter logs by a component ID

rdner added enhancement New feature or request Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team Team:Elastic-Agent Label for the Agent team backport-skip labels May 30, 2023

rdner self-assigned this May 30, 2023

Add changelog fragment

59fab55

rdner added 2 commits May 30, 2023 13:33

Add support for multi-day log rotation

d9536e8

Use the correct logs directory

7913d52

rdner commented May 30, 2023

View reviewed changes

rdner added 2 commits May 30, 2023 15:02

Add filter support while following the logs

e3cc5e6

Refactor and doc

ebead62

rdner marked this pull request as ready for review May 30, 2023 13:45

rdner requested a review from a team as a code owner May 30, 2023 13:45

rdner requested review from AndersonQ, pchila, joshdover, cmacknz and blakerouse May 30, 2023 13:45

pchila approved these changes May 31, 2023

View reviewed changes

belimawr self-requested a review May 31, 2023 15:07

rdner merged commit a35c498 into elastic:main May 31, 2023
22 checks passed

rdner added a commit to elastic/ingest-docs that referenced this pull request Sep 4, 2023

Add documentation for the logs command

1e4868e

Related issues: elastic/elastic-agent#2745 elastic/elastic-agent#3345

rdner mentioned this pull request Sep 4, 2023

Add documentation for the logs command elastic/ingest-docs#450

Merged

rdner added a commit to elastic/ingest-docs that referenced this pull request Sep 11, 2023

Add documentation for the logs command (#450)

105ef9b

Related issues: elastic/elastic-agent#2745 elastic/elastic-agent#3345

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add the `logs` subcommand to the CLI #2745

Add the `logs` subcommand to the CLI #2745

rdner commented May 30, 2023 •

edited by jlind23

elasticmachine commented May 30, 2023 •

edited

Build stats

Test stats 🧪

elasticmachine commented May 30, 2023 •

edited

rdner May 30, 2023

rdner May 31, 2023

pchila left a comment

pchila May 31, 2023

rdner May 31, 2023

pchila commented May 31, 2023

michalpristas commented Jun 5, 2023

	// Logs returns the log directory for Agent
	func Logs() string {
	return logsPath
	}

	func MakeInternalFileOutput(cfg *Config) (zapcore.Core, error) {
	// defaultCfg is used to set the defaults for the file rotation of the internal logging
	// these settings cannot be changed by a user configuration
	defaultCfg := logp.DefaultConfig(logp.DefaultEnvironment)
	filename := filepath.Join(paths.Home(), DefaultLogDirectory, cfg.Beat)
	al := zap.NewAtomicLevelAt(cfg.Level.ZapLevel())
	internalLevelEnabler = &al // directly persisting struct will panic on accessing unitialized backing pointer

Add the logs subcommand to the CLI #2745

Add the logs subcommand to the CLI #2745

Conversation

rdner commented May 30, 2023 • edited by jlind23

What does this PR do?

Why is it important?

Checklist

How to test this PR locally

Related issues

Screenshots

elasticmachine commented May 30, 2023 • edited

💚 Build Succeeded

Build stats

Test stats 🧪

💚 Flaky test report

🤖 GitHub comments

elasticmachine commented May 30, 2023 • edited

🌐 Coverage report

rdner May 30, 2023

Choose a reason for hiding this comment

rdner May 31, 2023

Choose a reason for hiding this comment

pchila left a comment

Choose a reason for hiding this comment

pchila May 31, 2023

Choose a reason for hiding this comment

rdner May 31, 2023

Choose a reason for hiding this comment

pchila commented May 31, 2023

michalpristas commented Jun 5, 2023

Add the `logs` subcommand to the CLI #2745

Add the `logs` subcommand to the CLI #2745

rdner commented May 30, 2023 •

edited by jlind23

elasticmachine commented May 30, 2023 •

edited

elasticmachine commented May 30, 2023 •

edited