Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore(promtail): Use minimal init config in rpm/deb packaging. #11511

Merged
merged 12 commits into from
Jan 14, 2024
Merged

chore(promtail): Use minimal init config in rpm/deb packaging. #11511

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
bd0dd7d
chore(promtail): Use minimal init config in npm/deb packaging.
kavirajk Dec 18, 2023
a3ab311
add changelog
kavirajk Dec 18, 2023
cb4a913
move it to `tools/packaging`
kavirajk Dec 18, 2023
f8cee7a
Update CHANGELOG.md
kavirajk Dec 18, 2023
6c000c6
Update tools/packaging/promtail-minimal-config.yaml
kavirajk Dec 18, 2023
47c551a
Merge branch 'main' into kavirajk/sane-init-config-packaging
kavirajk Jan 9, 2024
c4f52a3
Update CHANGELOG.md
kavirajk Jan 10, 2024
b452796
indendation fix
kavirajk Jan 10, 2024
c6dfb85
change rpm and deb packages testing log file name
kavirajk Jan 11, 2024
14443d9
Merge branch 'main' into kavirajk/sane-init-config-packaging
kavirajk Jan 11, 2024
2d898a3
Some dummy commit to force re-run CLA check
kavirajk Jan 11, 2024
fee7a9e
Merge branch 'main' into kavirajk/sane-init-config-packaging
kavirajk Jan 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@
#### Promtail

* [10752](https://github.com/grafana/loki/pull/10752) **gonzalesraul**: structured_metadata: enable structured_metadata convert labels
* [11511](https://github.com/grafana/loki/pull/11511) **kavirajk**: chore(promtail): Use minimal init config in rpm/deb packaging.
kavirajk marked this conversation as resolved.
Show resolved Hide resolved

##### Enhancements

Expand Down
8 changes: 4 additions & 4 deletions tools/packaging/nfpm.jsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ local overrides = {

loki: {
description: |||
Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus.
It is designed to be very cost effective and easy to operate.
Loki is a horizontally-scalable, highly-available, multi-tenant log aggregation system inspired by Prometheus.
It is designed to be very cost effective and easy to operate.
It does not index the contents of the logs, but rather a set of labels for each log stream.
|||,
contents+: [
Expand All @@ -35,7 +35,7 @@ local overrides = {

promtail: {
description: |||
Promtail is an agent which ships the contents of local logs to a private Grafana Loki instance or Grafana Cloud.
Promtail is an agent which ships the contents of local logs to a private Grafana Loki instance or Grafana Cloud.
It is usually deployed to every machine that has applications needed to be monitored.
|||,
license: 'Apache-2.0',
Expand All @@ -45,7 +45,7 @@ local overrides = {
dst: '/etc/systemd/system/promtail.service',
},
{
src: './clients/cmd/promtail/promtail-local-config.yaml',
src: './tools/packaging/promtail-minimal-config.yaml',
dst: '/etc/promtail/config.yml',
type: 'config|noreplace',
},
Expand Down
23 changes: 23 additions & 0 deletions tools/packaging/promtail-minimal-config.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# This minimal config scrape only single log file.
# Primarily used in rpm/deb packaging where promtail service can be started during system init process.
# And too much scraping during init process can overload the complete system.
# https://github.com/grafana/loki/issues/11398

server:
http_listen_port: 9080
grpc_listen_port: 0

positions:
filename: /tmp/positions.yaml

clients:
- url: http://localhost:3100/loki/api/v1/push
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use consistent list indentation (compare with scrape_configs).


scrape_configs:
- job_name: system
static_configs:
- targets:
- localhost
labels:
job: varlogs
__path__: /var/log/messages
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a comment that this section should be changed by the user after installation?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a hint :-) - there is now more failing in the build-pipeline - https://drone.grafana.net/grafana/loki/32891/25/8

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Release stage" on drone pipeline seems flaky.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think found the issue. The packaging test pipeline generate test log and writes to /var/log/test.log and checks if its scraped fine, By using minimal config we scrape only /var/log/messages and doesn't include /var/log/test.log.

I changed the filename from /var/log/test.log -> /var/log/messages instead in the scripts so test logs are properly scraped.

I think that should fix it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It did.