-
Notifications
You must be signed in to change notification settings - Fork 7
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[#361] Integrated Loki and Promtail into Deployment Stack
This commit introduces Loki and Promtail to the deployment stack, enhancing the project's logging infrastructure by enabling advanced log aggregation, visualization, and analysis capabilities. The integration involves modifications across multiple configuration files and the introduction of new templates for Loki and Promtail. Here's a breakdown of the changes: - config.mk Adjustments: Updated the prepare-config target to include steps for preparing Loki and Promtail configuration files, ensuring these services are configured as part of the deployment setup. - Docker Compose Template Update: Extended the docker-compose.yml.tpl to define loki and promtail services, including their respective Docker images, volumes, and dependencies. This update ensures that both Loki for log aggregation and Promtail for log shipping are part of the containerized environment. - Grafana Data Sources Configuration: Modified the grafana-provisioning/datasources/datasource.yml to include Loki as a data source, enabling Grafana to query log data collected by Loki. This allows for comprehensive log visualization alongside metrics within Grafana dashboards. - Loki Configuration Template (loki.yml): Introduced a new configuration template for Loki, specifying its operational parameters such as server ports, ingester configuration, schema, storage, and retention policies. This configuration is designed to optimize Loki's log aggregation capabilities within the deployment environment. - Promtail Configuration Template (promtail.yml): Added a new configuration template for Promtail, detailing how log files are located, scraped, and forwarded to Loki. The configuration includes the definition of job names, static configs, and labels for log categorization. The integration of Loki and Promtail represents a significant enhancement to the project's observability infrastructure. It enables efficient log collection, storage, and querying, providing developers and operators with powerful tools for troubleshooting, performance monitoring, and operational analysis.
- Loading branch information
Showing
5 changed files
with
136 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
auth_enabled: false | ||
|
||
server: | ||
http_listen_port: 3100 | ||
grpc_listen_port: 9095 | ||
|
||
ingester: | ||
wal: | ||
enabled: false | ||
lifecycler: | ||
address: 127.0.0.1 | ||
ring: | ||
kvstore: | ||
store: inmemory | ||
replication_factor: 1 | ||
chunk_idle_period: 15m | ||
chunk_block_size: 262144 | ||
chunk_retain_period: 1m | ||
max_transfer_retries: 0 | ||
|
||
compactor: | ||
working_directory: /loki/compactor | ||
shared_store: filesystem | ||
|
||
schema_config: | ||
configs: | ||
- from: 2024-01-01 | ||
store: boltdb-shipper | ||
object_store: filesystem | ||
schema: v11 | ||
index: | ||
prefix: index_ | ||
period: 24h | ||
|
||
storage_config: | ||
boltdb_shipper: | ||
active_index_directory: /loki/index | ||
cache_location: /loki/index_cache | ||
cache_ttl: 24h | ||
shared_store: filesystem | ||
filesystem: | ||
directory: /loki/chunks | ||
|
||
limits_config: | ||
enforce_metric_name: false | ||
reject_old_samples: true | ||
reject_old_samples_max_age: 168h | ||
max_cache_freshness_per_query: 10m | ||
retention_period: 672h | ||
|
||
chunk_store_config: | ||
max_look_back_period: 672h | ||
|
||
table_manager: | ||
retention_deletes_enabled: true | ||
retention_period: 672h |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
server: | ||
http_listen_port: 9080 | ||
grpc_listen_port: 0 | ||
|
||
positions: | ||
filename: /tmp/positions.yaml | ||
|
||
clients: | ||
- url: http://loki:3100/loki/api/v1/push | ||
|
||
scrape_configs: | ||
- job_name: system | ||
static_configs: | ||
- targets: | ||
- localhost | ||
labels: | ||
job: varlogs | ||
__path__: /var/log/deployment.log |