Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deploy Loki #2722

Merged
merged 14 commits into from
Aug 12, 2020
Merged

Deploy Loki #2722

merged 14 commits into from
Aug 12, 2020

Conversation

alexandre-allard
Copy link
Contributor

@alexandre-allard alexandre-allard commented Aug 6, 2020 •
edited by TeddyAndrieux
Loading

Component: salt, build, log

Context:

#2682

Summary:

  • Add support to remove manifest when rendering charts from render.py script
  • Create a new storage class for Loki metalk8s-loki
  • Take Loki charts from https://grafana.github.io/loki/charts and adapt value file to metalk8s
  • Render the Loki chart to salt states
  • Add CSC handling for Loki configuration
  • Add simple test to check that Loki work
  • Add new example file for Loki volume and use it in CI and vagrant deployment

Acceptance criteria:

Closes: #2682
Refs: #2712

@bert-e
Copy link
Contributor

bert-e commented Aug 6, 2020

Hello alexandre-allard-scality,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Status report is not available.

@bert-e
Copy link
Contributor

bert-e commented Aug 6, 2020

Waiting for approval

The following approvals are needed before I can proceed with the merge:

  • the author

  • one peer

Peer approvals must include at least 1 approval from the following list:

@alexandre-allard alexandre-allard mentioned this pull request Aug 6, 2020
Merged
@TeddyAndrieux TeddyAndrieux force-pushed the improvement/2682-deploy-loki branch 4 times, most recently from 8ffd6e1 to 7d03763 Compare August 7, 2020 15:42
@TeddyAndrieux TeddyAndrieux requested a review from a team August 7, 2020 17:07
@TeddyAndrieux TeddyAndrieux marked this pull request as ready for review August 7, 2020 17:07
This was referenced Aug 10, 2020
Merged
Closed
alexandre-allard and others added 14 commits August 11, 2020 10:40
@alexandre-allard @TeddyAndrieux
charts/log: initial import of loki helm charts
41acd19 
  helm repo add loki https://grafana.github.io/loki/charts
  helm repo update
  helm fetch -d charts --untar loki/loki

Refs: #2682
@alexandre-allard @TeddyAndrieux
charts: add option to remove manifests
65cf7bf 
Add `--remove-manifest KIND NAME` option allowing
to remove some manifests from the resulting chart.
Useful when we want to deploy and manage ourself
some object of an addon (e.g. Loki secret).

Refs: #2682
@alexandre-allard @TeddyAndrieux
charts/log: add loki helm configuration values
2da2ddd 
this file is used to overwrite default values.

Refs: #2682
@alexandre-allard @TeddyAndrieux
build: fetch loki image during build
79e58f8 
Refs: #2682
@alexandre-allard @TeddyAndrieux
salt: add Loki storageclass
6628336 
Refs: #2682
@TeddyAndrieux
charts: add persistence storage to Loki
97cd2ff 
Enable persistence from Loki chart
NOTE: That Loki chart does not allow to provide selector to claim
Volume, so any volume using `metalk8s-loki` storageClass can be bound to
the Loki container

Refs: #2682
@alexandre-allard @TeddyAndrieux
salt: render loki chart into Salt formula
4e5d4d7 
  ./charts/render.py loki --namespace metalk8s-logging \
    --remove-manifest Secret loki \
    charts/loki.yaml charts/loki/ \
    > salt/metalk8s/addons/logging/loki/deployed/chart.sls

Refs: #2682
@alexandre-allard @TeddyAndrieux
salt: deploy metalk8s-logging namespace
fce1062 
Refs: #2682
@alexandre-allard @TeddyAndrieux
build: add loki formulas to salt tree
e98d6ad 
Refs: #2682
@alexandre-allard @TeddyAndrieux
salt: deploy logging stack during MetalK8s install
a66ef4d 
For now it only deploys loki, but it'll
also include Fluent Bit installation.

Refs: #2682
@alexandre-allard @TeddyAndrieux
charts: add option to change path of CSC defaults
daea4b2 
This is needed as we put Loki inside logging directory,
so the path is not metalk8s/addons/loki/config/loki.yaml
but metalk8s/addons/logging/loki/config/loki.yaml, so
we need to be able to set the path.

Refs: #2682
@alexandre-allard @TeddyAndrieux
salt: add CSC mechanism to Loki
97b15c9 
Re-render the chart with CSC:

  ./charts/render.py loki \
    --remove-manifest Secret loki \
    --namespace metalk8s-logging \
    --service-config loki metalk8s-loki-config \
    metalk8s/addons/logging/loki/config/loki.yaml \
    charts/loki.yaml charts/loki/ \
    > salt/metalk8s/addons/logging/loki/deployed/chart.sls

Refs: #2682
@TeddyAndrieux
tests: Add tests for Loki alone
381b70f 
Add test about Loki only, to check that Loki work properly, this test
basically just use Loki API
(https://grafana.com/docs/loki/latest/api/#get-lokiapiv1query) to push a
Simple test log and then query this pushed log from Loki

Refs: #2712
@TeddyAndrieux
build,example,eve: Add example YAML for Loki volume
784ae86 
Add an example file to create a sparseLoopDevice volume for Loki, add
this file in the ISO and also use it for CI and vagrant deployment

Refs: #2682
TeddyAndrieux
TeddyAndrieux approved these changes Aug 11, 2020
View reviewed changes
Copy link
Collaborator

@TeddyAndrieux TeddyAndrieux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@alexandre-allard
Copy link
Contributor Author

/approve

@bert-e
Copy link
Contributor

bert-e commented Aug 12, 2020

In the queue

The changeset has received all authorizations and has been added to the
relevant queue(s). The queue(s) will be merged in the target development
branch(es) as soon as builds have passed.

The changeset will be merged in:

  • ✔️ development/2.6

The following branches will NOT be impacted:

  • development/1.0
  • development/1.1
  • development/1.2
  • development/1.3
  • development/2.0
  • development/2.1
  • development/2.2
  • development/2.3
  • development/2.4
  • development/2.5

There is no action required on your side. You will be notified here once
the changeset has been merged. In the unlikely event that the changeset
fails permanently on the queue, a member of the admin team will
contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

  • Any commit you add on the source branch will trigger a new cycle after the
    current queue is merged.
  • Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a
member of the admin team now.

The following options are set: approve

@bert-e
Copy link
Contributor

bert-e commented Aug 12, 2020

I have successfully merged the changeset of this pull request
into targetted development branches:

  • ✔️ development/2.6

The following branches have NOT changed:

  • development/1.0
  • development/1.1
  • development/1.2
  • development/1.3
  • development/2.0
  • development/2.1
  • development/2.2
  • development/2.3
  • development/2.4
  • development/2.5

Please check the status of the associated issue None.

Goodbye alexandre-allard-scality.

@bert-e bert-e merged commit ca7177d into development/2.6 Aug 12, 2020
@bert-e bert-e deleted the improvement/2682-deploy-loki branch August 12, 2020 14:26
@alexandre-allard alexandre-allard mentioned this pull request Aug 13, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@TeddyAndrieux TeddyAndrieux TeddyAndrieux approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Deployment of Loki
3 participants
@alexandre-allard @bert-e @TeddyAndrieux