Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Website doc link fixes #2566

Merged
merged 5 commits into from
Aug 27, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 18 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
<p align="center"><img src="docs/sources/logo_and_name.png" alt="Loki Logo"></p>

<a href="https://cloud.drone.io/grafana/loki"><img src="https://cloud.drone.io/api/badges/grafana/loki/status.svg" alt="Drone CI" /></a>
<a href="https://drone.grafana.net/grafana/loki"><img src="https://drone.grafana.net/api/badges/grafana/loki/status.svg" alt="Drone CI" /></a>
<a href="https://circleci.com/gh/grafana/loki/tree/master"><img src="https://circleci.com/gh/grafana/loki.svg?style=shield&circle-token=618193e5787b2951c1ea3352ad5f254f4f52313d" alt="CircleCI" /></a>
<a href="https://goreportcard.com/report/github.com/grafana/loki"><img src="https://goreportcard.com/badge/github.com/grafana/loki" alt="Go Report Card" /></a>
<a href="https://slack.grafana.com/"><img src="https://img.shields.io/badge/join%20slack-%23loki-brightgreen.svg" alt="Slack" /></a>
Expand Down Expand Up @@ -29,18 +29,17 @@ Loki differs from Prometheus by focusing on logs instead of metrics, and deliver

## Getting started

* [Installing Loki](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/installation/_index.md)
* [Installing Promtail](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/clients/promtail/installation.md)
* [Getting Started Guide](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/getting-started/_index.md)
* [Installing Loki](https://grafana.com/docs/loki/latest/installation/)
* [Installing Promtail](https://grafana.com/docs/loki/latest/clients/promtail/installation/)
* [Getting Started Guide](https://grafana.com/docs/loki/latest/getting-started/)

## Upgrading

* [Upgrading Loki](https://github.com/grafana/loki/blob/master/docs/sources/operations/upgrade.md)
* [Upgrading Loki](https://grafana.com/docs/loki/latest/operations/upgrade/)

### Documentation

* [master](./docs/README.md)
* [v1.6.0](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/_index.md)
* [master](https://grafana.com/docs/loki/latest/)
* [v1.5.0](https://github.com/grafana/loki/tree/v1.5.0/docs/README.md)
* [v1.4.1](https://github.com/grafana/loki/tree/v1.4.1/docs/README.md)
* [v1.4.0](https://github.com/grafana/loki/tree/v1.4.0/docs/README.md)
Expand All @@ -49,18 +48,18 @@ Loki differs from Prometheus by focusing on logs instead of metrics, and deliver
* [v1.1.0](https://github.com/grafana/loki/tree/v1.1.0/docs/README.md)
* [v1.0.0](https://github.com/grafana/loki/tree/v1.0.0/docs/README.md)

Commonly used sections (from the latest release v1.6.0):

- [API documentation](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/api/_index.md) for alternative ways of getting logs into Loki.
- [Labels](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/getting-started/labels.md)
- [Operations](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/operations/_index.md) for important aspects of running Loki.
- [Promtail](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/clients/promtail/_index.md) is an agent which can tail your log files and push them to Loki.
- [Pipelines](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/clients/promtail/pipelines.md) for detailed log processing pipeline documentation
- [Docker Logging Driver](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/clients/docker-driver/_index.md) is a docker plugin to send logs directly to Loki from Docker containers.
- [LogCLI](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/getting-started/logcli.md) on how to query your logs without Grafana.
- [Loki Canary](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/operations/loki-canary.md) for monitoring your Loki installation for missing logs.
- [Troubleshooting](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/getting-started/troubleshooting.md) for help around frequent error messages.
- [Loki in Grafana](https://github.com/grafana/loki/tree/v1.6.0/docs/sources/getting-started/grafana.md) for how to set up a Loki datasource in Grafana and query your logs.
Commonly used sections:

- [API documentation](https://grafana.com/docs/loki/latest/api/) for alternative ways of getting logs into Loki.
- [Labels](https://grafana.com/docs/loki/latest/getting-started/labels/)
- [Operations](https://grafana.com/docs/loki/latest/operations/) for important aspects of running Loki.
- [Promtail](https://grafana.com/docs/loki/latest/clients/promtail/) is an agent which can tail your log files and push them to Loki.
- [Pipelines](https://grafana.com/docs/loki/latest/clients/promtail/pipelines/) for detailed log processing pipeline documentation
- [Docker Logging Driver](https://grafana.com/docs/loki/latest/clients/docker-driver/) is a docker plugin to send logs directly to Loki from Docker containers.
- [LogCLI](https://grafana.com/docs/loki/latest/getting-started/logcli/) on how to query your logs without Grafana.
- [Loki Canary](https://grafana.com/docs/loki/latest/operations/loki-canary/) for monitoring your Loki installation for missing logs.
- [Troubleshooting](https://grafana.com/docs/loki/latest/getting-started/troubleshooting/) for help around frequent error messages.
- [Loki in Grafana](https://grafana.com/docs/loki/latest/getting-started/grafana/) for how to set up a Loki datasource in Grafana and query your logs.

## Getting Help

Expand Down
97 changes: 2 additions & 95 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,98 +2,5 @@

<p align="center"> <img src="sources/logo_and_name.png" alt="Loki Logo"> <br>
<small>Like Prometheus, but for logs!</small> </p>

Grafana Loki is a set of components that can be composed into a fully featured
logging stack.

Unlike other logging systems, Loki is built around the idea of only indexing
metadata about your logs: labels (just like Prometheus labels). Log data itself
is then compressed and stored in chunks in object stores such as S3 or GCS, or
even locally on the filesystem. A small index and highly compressed chunks
simplifies the operation and significantly lowers the cost of Loki.

## Table of Contents

1. [Overview](sources/overview/_index.md)
1. [Comparison to other Log Systems](sources/overview/comparisons.md)
2. [Installation](sources/installation/_index.md)
1. [Installing with Tanka](sources/installation/tanka.md)
2. [Installing with Helm](sources/installation/helm.md)
3. [Installing with Docker](sources/installation/docker.md)
4. [Installing locally](sources/installation/local.md)
3. [Getting Started](sources/getting-started/_index.md)
1. [Grafana](sources/getting-started/grafana.md)
2. [LogCLI](sources/getting-started/logcli.md)
3. [Labels](sources/getting-started/labels.md)
4. [Troubleshooting](sources/getting-started/troubleshooting.md)
4. [Best Practices](sources/best-practices/_index.md)
1. [Current Best Practices](sources/best-practices/current-best-practices.md)
5. [Configuration](sources/configuration/_index.md)
1. [Examples](sources/configuration/examples.md)
6. [Clients](sources/clients/_index.md)
1. [Promtail](sources/clients/promtail/_index.md)
1. [Installation](sources/clients/promtail/installation.md)
2. [Configuration](sources/clients/promtail/configuration.md)
3. [Scraping](sources/clients/promtail/scraping.md)
4. [Pipelines](sources/clients/promtail/pipelines.md)
5. [Troubleshooting](sources/clients/promtail/troubleshooting.md)
2. [Docker Driver](sources/clients/docker-driver/_index.md)
1. [Configuration](sources/clients/docker-driver/configuration.md)
4. [Fluent Bit](sources/clients/fluentbit/_index.md)
3. [Fluentd](sources/clients/fluentd/_index.md)
7. [LogQL](sources/logql/_index.md)
8. [Operations](sources/operations/_index.md)
1. [Authentication](sources/operations/authentication.md)
2. [Observability](sources/operations/observability.md)
3. [Scalability](sources/operations/scalability.md)
4. [Storage](sources/operations/storage/_index.md)
1. [Table Manager](sources/operations/storage/table-manager.md)
2. [Retention](sources/operations/storage/retention.md)
3. [BoltDB Shipper](sources/operations/storage/boltdb-shipper.md)
5. [Multi-tenancy](sources/operations/multi-tenancy.md)
6. [Loki Canary](sources/operations/loki-canary.md)
9. [HTTP API](sources/api/_index.md)
10. [Architecture](sources/architecture/_index.md)
11. [Community](sources/community/_index.md)
1. [Governance](sources/community/governance.md)
2. [Getting in Touch](sources/community/getting-in-touch.md)
3. [Contributing to Loki](sources/community/contributing.md)
12. [Loki Maintainers Guide](sources/./maintaining/_index.md)
1. [Releasing Loki](sources/./maintaining/release.md)

## Updating docs

### Building the docs locally

When you contribute to documentation, it is a good practice to build the docs on your local machine to make sure your changes appear as you expect. This README explains the process for doing that.

#### Requirements

Docker >= 2.1.0.3

### Build the doc site

1. In the command line, make sure you are in the docs folder: `cd docs`.
2. Run `make docs`. This launches a preview of the docs website at `http://localhost:3002/docs/loki/latest/` which will refresh automatically when changes to content in the `sources` directory are made.

---

### Content guidelines

Edit content in the `sources` directory.

### Edit the side menu

The side menu is built automatically from the folder structure. Ordering is done with the `weight` front matter param.

### Add images

Images are sourced in this repo alonside content. They will sync to the website repo just like markdown files.

---

## Deploy changes to grafana.com

When a PR is merged to master with changes in the `docs/sources` directory, those changes are automatically synched to the grafana/website repo and published to the staging site.

Generally, someone from marketing will publish to production each day, so as long as the sync is successful your docs edits will be published.

Please see the [Documentation Site](https://grafana.com/docs/loki/latest/) The files in this directory are used to generate it but consequently the links don't work in Github.
2 changes: 1 addition & 1 deletion docs/sources/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,4 @@ Unlike other logging systems, Loki is built around the idea of only indexing
metadata about your logs: labels (just like Prometheus labels). Log data itself
is then compressed and stored in chunks in object stores such as S3 or GCS, or
even locally on the filesystem. A small index and highly compressed chunks
simplifies the operation and significantly lowers the cost of Loki.
simplifies the operation and significantly lowers the cost of Loki.
2 changes: 1 addition & 1 deletion docs/sources/best-practices/current-best-practices.md
Original file line number Diff line number Diff line change
Expand Up @@ -121,4 +121,4 @@ Starting in version 1.6.0 Loki and Promtail have flags which will dump the entir

`-print-config-stderr` is nice when running loki directly e.g. `./loki ` as you can get a quick output of the entire Loki config.

`-log-config-reverse-order` is the flag we run Loki with in all our environments, the config entries are reversed so that the order of configs reads correctly top to bottom when viewed in Grafana's Explore.
`-log-config-reverse-order` is the flag we run Loki with in all our environments, the config entries are reversed so that the order of configs reads correctly top to bottom when viewed in Grafana's Explore.
2 changes: 1 addition & 1 deletion docs/sources/clients/aws/_index.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
---
title: AWS
---
---
8 changes: 4 additions & 4 deletions docs/sources/clients/aws/ec2/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,7 @@ That's it, save the config and you can `reboot` the machine (or simply restart t

Let's head back to Grafana and verify that your Promtail logs are available in Grafana by using the [LogQL](../../../logql/) query `{unit="promtail.service"}` in Explore. Finally make sure to checkout [live tailing][live tailing] to see logs appearing as they are ingested in Loki.

[promtail]: ../../promtail/README.md
[promtail]: ../../promtail/README
[aws cli]: https://aws.amazon.com/cli/
[create an vpc]: https://docs.aws.amazon.com/vpc/latest/userguide/vpc-subnets-commands-example.html
[create an ssh key]: https://confluence.atlassian.com/bitbucketserver/creating-ssh-keys-776639788.html
Expand All @@ -276,18 +276,18 @@ Let's head back to Grafana and verify that your Promtail logs are available in G
[chef]: https://www.chef.io/
[security group]: https://docs.aws.amazon.com/vpc/latest/userguide/VPC_SecurityGroups.html
[Amazon Linux 2]: https://aws.amazon.com/amazon-linux-2/
[promtail configuration]: ../../promtail/configuration.md
[promtail configuration]: ../../promtail/configuration
[prometheus scrape config]: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#scrape_config
[ec2_sd_config]: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#ec2_sd_config
[role]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html
[discovery page]: ./promtail-ec2-discovery.png "Service discovery"
[relabel]: https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config
[systemd]: https://www.freedesktop.org/software/systemd/man/systemd.service.html
[logql]: ../../../logql.md
[logql]: ../../../logql
[ec2 logs]: ./promtail-ec2-logs.png "Grafana Loki logs"
[config gist]: https://gist.github.com/cyriltovena/d0881cc717757db951b642be48c01445
[labels]: https://grafana.com/blog/2020/04/21/how-labels-in-loki-can-make-log-queries-faster-and-easier/
[troubleshooting loki]: ../../../getting-started/troubleshooting.md#troubleshooting-targets
[troubleshooting loki]: ../../../getting-started/troubleshooting#troubleshooting-targets
[live tailing]: https://grafana.com/docs/grafana/latest/features/datasources/loki/#live-tailing
[systemd]: https://github.com/grafana/loki/tree/master/production/helm/promtail#run-promtail-with-systemd-journal-support
[journald]: https://www.freedesktop.org/software/systemd/man/systemd-journald.service.html
Expand Down
2 changes: 1 addition & 1 deletion docs/sources/clients/aws/ecs/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -239,4 +239,4 @@ That's it ! Make sure to checkout the [LogQL][logql] to learn more about Loki po
[routing]: https://fluentbit.io/documentation/0.13/getting_started/routing.html
[grafanacloud account]: https://grafana.com/login
[grafana logs firelens]: ./ecs-grafana.png
[logql]: ../../../logql.md
[logql]: ../../../logql
2 changes: 1 addition & 1 deletion docs/sources/clients/aws/eks/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -261,7 +261,7 @@ If you want to push this further you can check out [Joe's blog post][blog annota
[blog annotations]: https://grafana.com/blog/2019/12/09/how-to-do-automatic-annotations-with-grafana-and-loki/
[kubectl]: https://kubernetes.io/docs/tasks/tools/install-kubectl/
[eksctl]: https://docs.aws.amazon.com/eks/latest/userguide/getting-started-eksctl.html
[Promtail]: ./../../promtail/README.md
[Promtail]: ./../../promtail/README
[prometheus conf]: https://prometheus.io/docs/prometheus/latest/configuration/configuration/
[eventrouter]: https://github.com/heptiolabs/eventrouter
[explore]: https://grafana.com/docs/grafana/latest/features/explore/
2 changes: 1 addition & 1 deletion docs/sources/clients/docker-driver/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ to a private Loki instance or [Grafana Cloud](https://grafana.com/oss/loki).
> [Docker docs](https://docs.docker.com/engine/extend) for more information.

Documentation on configuring the Loki Docker Driver can be found on the
[configuration page](./configuration.md).
[configuration page](./configuration).

If you have any questions or issues using the Docker plugin feel free to open an issue in this [repository](https://github.com/grafana/loki/issues).

Expand Down
2 changes: 1 addition & 1 deletion docs/sources/clients/docker-driver/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ Once deployed, the Grafana service will send its logs to Loki.

## Labels

Loki can received a set of labels along with log line. These labels are used to index log entries and query back logs using [LogQL stream selector](../../logql/_index.md#log-stream-selector).
Loki can received a set of labels along with log line. These labels are used to index log entries and query back logs using [LogQL stream selector](../../logql#log-stream-selector).

By default, the Docker driver will add the following labels to each log line:

Expand Down
10 changes: 5 additions & 5 deletions docs/sources/clients/fluentbit/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ docker run -v /var/log:/var/log \

You can run Fluent Bit as a [Daemonset](https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/) to collect all your Kubernetes workload logs.

To do so you can use our [Fluent Bit helm chart](../../production/helm/fluent-bit/README.md):
To do so you can use our [Fluent Bit helm chart](../../production/helm/fluent-bit/README):

> Make sure [tiller](https://helm.sh/docs/install/) is installed correctly in your cluster

Expand All @@ -44,11 +44,11 @@ helm upgrade --install loki-stack loki/loki-stack \
### AWS Elastic Container Service (ECS)

You can use fluent-bit Loki Docker image as a Firelens log router in AWS ECS.
For more information about this see our [AWS documentation](../aws/ecs/_index.md)
For more information about this see our [AWS documentation](../aws/ecs)

### Local

First you need to follow those [instructions](https://github.com/grafana/loki/blob/master/cmd/fluent-bit/README.md) to build the plugin dynamic library.
First you need to follow those [instructions](https://github.com/grafana/loki/blob/master/cmd/fluent-bit/README) to build the plugin dynamic library.

The assuming you have Fluent Bit installed in your `$PATH` you can run the plugin using:

Expand Down Expand Up @@ -92,7 +92,7 @@ You can also adapt your plugins.conf, removing the need to change the command li

### Labels

Labels are used to [query logs](../../logql/_index.md) `{container_name="nginx", cluster="us-west1"}`, they are usually metadata about the workload producing the log stream (`instance`, `container_name`, `region`, `cluster`, `level`). In Loki labels are indexed consequently you should be cautious when choosing them (high cardinality label values can have performance drastic impact).
Labels are used to [query logs](../../logql) `{container_name="nginx", cluster="us-west1"}`, they are usually metadata about the workload producing the log stream (`instance`, `container_name`, `region`, `cluster`, `level`). In Loki labels are indexed consequently you should be cautious when choosing them (high cardinality label values can have performance drastic impact).

You can use `Labels`, `RemoveKeys` , `LabelKeys` and `LabelMapPath` to how the output plugin will perform labels extraction.

Expand All @@ -103,7 +103,7 @@ If set to true, it will add all Kubernetes labels to Loki labels automatically a
### LabelMapPath

When using the `Parser` and `Filter` plugins Fluent Bit can extract and add data to the current record/log data. While Loki labels are key value pair, record data can be nested structures.
You can pass a json file that defines how to extract [labels](../../docs/sources/overview/_index.md#overview-of-loki) from each record. Each json key from the file will be matched with the log record to find label values. Values from the configuration are used as label names.
You can pass a json file that defines how to extract [labels](../../docs/sources/overview#overview-of-loki) from each record. Each json key from the file will be matched with the log record to find label values. Values from the configuration are used as label names.

Considering the record below :

Expand Down
2 changes: 1 addition & 1 deletion docs/sources/clients/lambda-promtail/_index.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Lambda Promtail

Loki includes an [AWS SAM](https://aws.amazon.com/serverless/sam/) package template for shipping Cloudwatch logs to Loki via a set of promtails [here](../../../tools/lambda-promtail/). This is done via an intermediary [lambda function](https://aws.amazon.com/lambda/) which processes cloudwatch events and propagates them to a promtail instance (or set of instances behind a load balancer) via the push-api [scrape config](docs/clients/promtail/configuration#loki_push_api_config).
Loki includes an [AWS SAM](https://aws.amazon.com/serverless/sam/) package template for shipping Cloudwatch logs to Loki via a set of promtails [here](../../../tools/lambda-promtail/). This is done via an intermediary [lambda function](https://aws.amazon.com/lambda/) which processes cloudwatch events and propagates them to a promtail instance (or set of instances behind a load balancer) via the push-api [scrape config](../promtail/configuration#loki_push_api_config).

## Uses

Expand Down
2 changes: 1 addition & 1 deletion docs/sources/clients/promtail/stages/drop.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,4 +190,4 @@ Given the pipeline:
regex: ".*trace.*"
```

Would drop all logs older than 24h OR longer than 8kb bytes OR have a json `msg` field containing the word _trace_
Would drop all logs older than 24h OR longer than 8kb bytes OR have a json `msg` field containing the word _trace_
Loading