Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add performance profile flags for logcli. #2248

Merged
merged 4 commits into from
Jun 25, 2020
Merged

Add performance profile flags for logcli. #2248

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
de970d8
Add performance profile flags for logcli.
cyriltovena Jun 22, 2020
b70d249
Update the logcli documentation.
cyriltovena Jun 22, 2020
c1d17d9
Remove comment.
cyriltovena Jun 22, 2020
79ea039
remove unused import.
cyriltovena Jun 23, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 32 additions & 3 deletions cmd/logcli/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"log"
"net/url"
"os"
"runtime/pprof"
"time"

"github.com/prometheus/common/config"
Expand All @@ -20,11 +21,14 @@ import (

var (
app = kingpin.New("logcli", "A command-line for loki.").Version(version.Print("logcli"))
quiet = app.Flag("quiet", "suppress query metadata").Default("false").Short('q').Bool()
statistics = app.Flag("stats", "show query statistics").Default("false").Bool()
outputMode = app.Flag("output", "specify output mode [default, raw, jsonl]. raw suppresses log labels and timestamp.").Default("default").Short('o').Enum("default", "raw", "jsonl")
quiet = app.Flag("quiet", "Suppress query metadata").Default("false").Short('q').Bool()
statistics = app.Flag("stats", "Show query statistics").Default("false").Bool()
outputMode = app.Flag("output", "Specify output mode [default, raw, jsonl]. raw suppresses log labels and timestamp.").Default("default").Short('o').Enum("default", "raw", "jsonl")
timezone = app.Flag("timezone", "Specify the timezone to use when formatting output timestamps [Local, UTC]").Default("Local").Short('z').Enum("Local", "UTC")

cpuProfile = app.Flag("cpuprofile", "Specify the location for writing a CPU profile.").Default("").String()
memProfile = app.Flag("memprofile", "Specify the location for writing a memory profile.").Default("").String()

queryClient = newQueryClient(app)

queryCmd = app.Command("query", `Run a LogQL query.
Expand Down Expand Up @@ -82,6 +86,31 @@ func main() {

cmd := kingpin.MustParse(app.Parse(os.Args[1:]))

if cpuProfile != nil && *cpuProfile != "" {
cpuFile, err := os.Create(*cpuProfile)
if err != nil {
log.Fatal("could not create CPU profile: ", err)
}
defer cpuFile.Close()
if err := pprof.StartCPUProfile(cpuFile); err != nil {
log.Fatal("could not start CPU profile: ", err)
}
defer pprof.StopCPUProfile()
}

if memProfile != nil && *memProfile != "" {
memFile, err := os.Create(*memProfile)
if err != nil {
log.Fatal("could not create memory profile: ", err)
}
defer memFile.Close()
defer func() {
if err := pprof.WriteHeapProfile(memFile); err != nil {
log.Fatal("could not write memory profile: ", err)
}
}()
}

switch cmd {
case queryCmd.FullCommand():
location, err := time.LoadLocation(*timezone)
Expand Down
142 changes: 89 additions & 53 deletions docs/getting-started/logcli.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,19 +77,22 @@ A command-line for loki.
Flags:
--help Show context-sensitive help (also try --help-long and --help-man).
--version Show application version.
-q, --quiet suppress query metadata
--stats show query statistics
-o, --output=default specify output mode [default, raw, jsonl]. raw suppresses log labels and timestamp.
-q, --quiet Suppress query metadata
--stats Show query statistics
-o, --output=default Specify output mode [default, raw, jsonl]. raw suppresses log labels and timestamp.
-z, --timezone=Local Specify the timezone to use when formatting output timestamps [Local, UTC]
--addr="http://localhost:3100"
--cpuprofile="" Specify the location for writing a CPU profile.
--memprofile="" Specify the location for writing a memory profile.
--addr="http://localhost:3100"
Server address. Can also be set using LOKI_ADDR env var.
--username="" Username for HTTP basic auth. Can also be set using LOKI_USERNAME env var.
--password="" Password for HTTP basic auth. Can also be set using LOKI_PASSWORD env var.
--ca-cert="" Path to the server Certificate Authority. Can also be set using LOKI_CA_CERT_PATH env var.
--tls-skip-verify Server certificate TLS skip verify. Can also be set using LOKI_TLS_SKIP_VERIFY env var.
--tls-skip-verify Server certificate TLS skip verify.
--cert="" Path to the client certificate. Can also be set using LOKI_CLIENT_CERT_PATH env var.
--key="" Path to the client certificate key. Can also be set using LOKI_CLIENT_KEY_PATH env var.
--org-id=ORG-ID adds X-Scope-OrgID to API requests for representing tenant ID. Useful for requesting tenant data when bypassing an auth gateway. Can also be set using LOKI_ORG_ID env var.
--org-id="" adds X-Scope-OrgID to API requests for representing tenant ID. Useful for requesting tenant data when
bypassing an auth gateway.

Commands:
help [<command>...]
Expand All @@ -106,26 +109,29 @@ Commands:

The output of the log can be specified with the "-o" flag, for example, "-o raw" for the raw output format.

The "query" command will output extra information about the query and its results, such as the API URL, set of common labels, and set of excluded labels. This
extra information can be suppressed with the --quiet flag.
The "query" command will output extra information about the query and its results, such as the API URL, set of common labels,
and set of excluded labels. This extra information can be suppressed with the --quiet flag.

While "query" does support metrics queries, its output contains multiple data points between the start and end query time. This output is used to build graphs,
like what is seen in the Grafana Explore graph view. If you are querying metrics and just want the most recent data point (like what is seen in the Grafana Explore
table view), then you should use the "instant-query" command instead.
While "query" does support metrics queries, its output contains multiple data points between the start and end query time.
This output is used to build graphs, like what is seen in the Grafana Explore graph view. If you are querying metrics and just
want the most recent data point (like what is seen in the Grafana Explore table view), then you should use the "instant-query"
command instead.

instant-query [<flags>] <query>
Run an instant LogQL query.

The "instant-query" command is useful for evaluating a metric query for a single point in time. This is equivalent to the Grafana Explore table view; if you want a
metrics query that is used to build a Grafana graph, you should use the "query" command instead.
The "instant-query" command is useful for evaluating a metric query for a single point in time. This is equivalent to the
Grafana Explore table view; if you want a metrics query that is used to build a Grafana graph, you should use the "query"
command instead.

This command does not produce useful output when querying for log lines; you should always use the "query" command when you are running log queries.
This command does not produce useful output when querying for log lines; you should always use the "query" command when you
are running log queries.

For more information about log queries and metric queries, refer to the LogQL documentation:

https://github.com/grafana/loki/blob/master/docs/logql.md

labels [<label>]
labels [<flags>] [<label>]
Find values for a given label.

series --match=MATCH [<flags>]
Expand All @@ -136,34 +142,59 @@ usage: logcli query [<flags>] <query>

Run a LogQL query.

The "query" command is useful for querying for logs. Logs can be returned in a few output modes:

raw: log line
default: log timestamp + log labels + log line
jsonl: JSON response from Loki API of log line

The output of the log can be specified with the "-o" flag, for example, "-o raw" for the raw output format.

The "query" command will output extra information about the query and its results, such as the API URL, set of common labels, and
set of excluded labels. This extra information can be suppressed with the --quiet flag.

While "query" does support metrics queries, its output contains multiple data points between the start and end query time. This
output is used to build graphs, like what is seen in the Grafana Explore graph view. If you are querying metrics and just want the
most recent data point (like what is seen in the Grafana Explore table view), then you should use the "instant-query" command
instead.

Flags:
--help Show context-sensitive help (also try --help-long and --help-man).
--version Show application version.
-q, --quiet suppress everything but log lines
-o, --output=default specify output mode [default, raw, jsonl]
-z, --timezone=Local Specify the timezone to use when formatting output timestamps [Local, UTC]
--addr="http://localhost:3100"
Server address. Can also be set using LOKI_ADDR env var.
--username="" Username for HTTP basic auth. Can also be set using LOKI_USERNAME env var.
--password="" Password for HTTP basic auth. Can also be set using LOKI_PASSWORD env var.
--ca-cert="" Path to the server Certificate Authority. Can also be set using LOKI_CA_CERT_PATH env var.
--tls-skip-verify Server certificate TLS skip verify.
--cert="" Path to the client certificate. Can also be set using LOKI_CLIENT_CERT_PATH env var.
--key="" Path to the client certificate key. Can also be set using LOKI_CLIENT_KEY_PATH env var.
--limit=30 Limit on number of entries to print.
--since=1h Lookback window.
--from=FROM Start looking for logs at this absolute time (inclusive)
--to=TO Stop looking for logs at this absolute time (exclusive)
--step=STEP Query resolution step width
--forward Scan forwards through logs.
--no-labels Do not print any labels
--exclude-label=EXCLUDE-LABEL ...
Exclude labels given the provided key during output.
--include-label=INCLUDE-LABEL ...
Include labels given the provided key during output.
--labels-length=0 Set a fixed padding to labels
-t, --tail Tail the logs
--delay-for=0 Delay in tailing by number of seconds to accumulate logs for re-ordering
--help Show context-sensitive help (also try --help-long and --help-man).
--version Show application version.
-q, --quiet Suppress query metadata
--stats Show query statistics
-o, --output=default Specify output mode [default, raw, jsonl]. raw suppresses log labels and timestamp.
-z, --timezone=Local Specify the timezone to use when formatting output timestamps [Local, UTC]
--cpuprofile="" Specify the location for writing a CPU profile.
--memprofile="" Specify the location for writing a memory profile.
--addr="http://localhost:3100"
Server address. Can also be set using LOKI_ADDR env var.
--username="" Username for HTTP basic auth. Can also be set using LOKI_USERNAME env var.
--password="" Password for HTTP basic auth. Can also be set using LOKI_PASSWORD env var.
--ca-cert="" Path to the server Certificate Authority. Can also be set using LOKI_CA_CERT_PATH env var.
--tls-skip-verify Server certificate TLS skip verify.
--cert="" Path to the client certificate. Can also be set using LOKI_CLIENT_CERT_PATH env var.
--key="" Path to the client certificate key. Can also be set using LOKI_CLIENT_KEY_PATH env var.
--org-id="" adds X-Scope-OrgID to API requests for representing tenant ID. Useful for requesting tenant data when
bypassing an auth gateway.
--limit=30 Limit on number of entries to print.
--since=1h Lookback window.
--from=FROM Start looking for logs at this absolute time (inclusive)
--to=TO Stop looking for logs at this absolute time (exclusive)
--step=STEP Query resolution step width, for metric queries. Evaluate the query at the specified step over the time
range.
--interval=INTERVAL Query interval, for log queries. Return entries at the specified interval, ignoring those between.
**This parameter is experimental, please see Issue 1779**
--forward Scan forwards through logs.
--no-labels Do not print any labels
--exclude-label=EXCLUDE-LABEL ...
Exclude labels given the provided key during output.
--include-label=INCLUDE-LABEL ...
Include labels given the provided key during output.
--labels-length=0 Set a fixed padding to labels
--store-config="" Execute the current query using a configured storage from a given Loki configuration file.
-t, --tail Tail the logs
--delay-for=0 Delay in tailing by number of seconds to accumulate logs for re-ordering

Args:
<query> eg '{foo="bar",baz=~".*blip"} |~ ".*error.*"'
Expand All @@ -176,20 +207,22 @@ Find values for a given label.
Flags:
--help Show context-sensitive help (also try --help-long and --help-man).
--version Show application version.
-q, --quiet suppress query metadata
--stats show query statistics
-o, --output=default specify output mode [default, raw, jsonl]. raw suppresses log labels and timestamp.
-q, --quiet Suppress query metadata
--stats Show query statistics
-o, --output=default Specify output mode [default, raw, jsonl]. raw suppresses log labels and timestamp.
-z, --timezone=Local Specify the timezone to use when formatting output timestamps [Local, UTC]
--addr="http://localhost:3100"
--cpuprofile="" Specify the location for writing a CPU profile.
--memprofile="" Specify the location for writing a memory profile.
--addr="http://localhost:3100"
Server address. Can also be set using LOKI_ADDR env var.
--username="" Username for HTTP basic auth. Can also be set using LOKI_USERNAME env var.
--password="" Password for HTTP basic auth. Can also be set using LOKI_PASSWORD env var.
--ca-cert="" Path to the server Certificate Authority. Can also be set using LOKI_CA_CERT_PATH env var.
--tls-skip-verify Server certificate TLS skip verify.
--cert="" Path to the client certificate. Can also be set using LOKI_CLIENT_CERT_PATH env var.
--key="" Path to the client certificate key. Can also be set using LOKI_CLIENT_KEY_PATH env var.
--org-id=ORG-ID adds X-Scope-OrgID to API requests for representing tenant ID. Useful for requesting
tenant data when bypassing an auth gateway.
--org-id="" adds X-Scope-OrgID to API requests for representing tenant ID. Useful for requesting tenant data when
bypassing an auth gateway.
--since=1h Lookback window.
--from=FROM Start looking for labels at this absolute time (inclusive)
--to=TO Stop looking for labels at this absolute time (exclusive)
Expand All @@ -205,19 +238,22 @@ Run series query.
Flags:
--help Show context-sensitive help (also try --help-long and --help-man).
--version Show application version.
-q, --quiet suppress query metadata
--stats show query statistics
-o, --output=default specify output mode [default, raw, jsonl]. raw suppresses log labels and timestamp.
-q, --quiet Suppress query metadata
--stats Show query statistics
-o, --output=default Specify output mode [default, raw, jsonl]. raw suppresses log labels and timestamp.
-z, --timezone=Local Specify the timezone to use when formatting output timestamps [Local, UTC]
--addr="http://localhost:3100"
--cpuprofile="" Specify the location for writing a CPU profile.
--memprofile="" Specify the location for writing a memory profile.
--addr="http://localhost:3100"
Server address. Can also be set using LOKI_ADDR env var.
--username="" Username for HTTP basic auth. Can also be set using LOKI_USERNAME env var.
--password="" Password for HTTP basic auth. Can also be set using LOKI_PASSWORD env var.
--ca-cert="" Path to the server Certificate Authority. Can also be set using LOKI_CA_CERT_PATH env var.
--tls-skip-verify Server certificate TLS skip verify.
--cert="" Path to the client certificate. Can also be set using LOKI_CLIENT_CERT_PATH env var.
--key="" Path to the client certificate key. Can also be set using LOKI_CLIENT_KEY_PATH env var.
--org-id=ORG-ID adds X-Scope-OrgID to API requests for representing tenant ID. Useful for requesting tenant data when bypassing an auth gateway.
--org-id="" adds X-Scope-OrgID to API requests for representing tenant ID. Useful for requesting tenant data when
bypassing an auth gateway.
--since=1h Lookback window.
--from=FROM Start looking for logs at this absolute time (inclusive)
--to=TO Stop looking for logs at this absolute time (exclusive)
Expand Down