feat(administration): Helm chart for deploying Vector with an aggregator role#4404
Merged
feat(administration): Helm chart for deploying Vector with an aggregator role#4404
Conversation
This was referenced Oct 6, 2020
Merged
3 tasks
MOZGIII
force-pushed
the
helm-aggregator-role
branch
from
0f50142 to
76ca48f
Compare
MOZGIII
force-pushed
the
helm-aggregator-role
branch
from
76ca48f to
7ee5ac4
Compare
MOZGIII
force-pushed
the
helm-aggregator-role
branch
3 times, most recently
from
9aa3cb3 to
b7a72a6
Compare
Hoverbear
reviewed
Oct 16, 2020
MOZGIII
force-pushed
the
helm-aggregator-role
branch
3 times, most recently
from
df31eef to
ff6b3db
Compare
MOZGIII
force-pushed
the
helm-aggregator-role
branch
2 times, most recently
from
675e269 to
9d89c98
Compare
eeyun
reviewed
Nov 2, 2020
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
…aemonset.yaml Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
…etes-yaml/values.yaml Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
…es.yaml Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
MOZGIII
force-pushed
the
helm-aggregator-role
branch
from
638a6cb to
8067139
Compare
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
MOZGIII
force-pushed
the
helm-aggregator-role
branch
from
f61d343 to
274a443
Compare
Signed-off-by: MOZGIII <mike-n@narod.ru>
Signed-off-by: MOZGIII <mike-n@narod.ru>
mengesb
pushed a commit
to jacobbraaten/vector
that referenced
this pull request
Dec 9, 2020
…tor role (vectordotdev#4404) * rough draft of service style vector deployment Signed-off-by: Spencer Gilbert <spencer.gilbert@gmail.com> Signed-off-by: MOZGIII <mike-n@narod.ru> * Update the values at vector-service helm chart Signed-off-by: MOZGIII <mike-n@narod.ru> * Rename vector-service to vector-aggregator and update Chart.yaml Signed-off-by: MOZGIII <mike-n@narod.ru> * Add vector-aggregator to scripting config Signed-off-by: MOZGIII <mike-n@narod.ru> * Update the helm charts dependencies Signed-off-by: MOZGIII <mike-n@narod.ru> * Switch vector-aggregator to libvector helpers Signed-off-by: MOZGIII <mike-n@narod.ru> * Update PodSecurityPolicy for vector-aggregator Signed-off-by: MOZGIII <mike-n@narod.ru> * Limit the ClusterRole permissions for vector-aggregator Signed-off-by: MOZGIII <mike-n@narod.ru> * Do not exclude aggregator from vector log collection Signed-off-by: MOZGIII <mike-n@narod.ru> * Order StatefulSet params in the different order Signed-off-by: MOZGIII <mike-n@narod.ru> * Enable dynamic configuration files at StatefulSet Signed-off-by: MOZGIII <mike-n@narod.ru> * Only pass env if defined Signed-off-by: MOZGIII <mike-n@narod.ru> * Fix the PodSecurityPolicy at vector-agent Signed-off-by: MOZGIII <mike-n@narod.ru> * Fix the PodSecurityPolicy at vector-aggregator Signed-off-by: MOZGIII <mike-n@narod.ru> * Update the volumeClaimTemplates configuration at StatefulSet Signed-off-by: MOZGIII <mike-n@narod.ru> * More advanced storage configuration Signed-off-by: MOZGIII <mike-n@narod.ru> * Correct the config resolution for StatefulSet Signed-off-by: MOZGIII <mike-n@narod.ru> * Add headless Service and switch the StatefulSet to it Signed-off-by: MOZGIII <mike-n@narod.ru> * Add vector-aggregator chart to vector chart as a dependency Signed-off-by: MOZGIII <mike-n@narod.ru> * Update dependencies Signed-off-by: MOZGIII <mike-n@narod.ru> * Allow overriding vector repository via global var at vector-aggregator Signed-off-by: MOZGIII <mike-n@narod.ru> * Add the ability to configure service ports Signed-off-by: MOZGIII <mike-n@narod.ru> * Add built-in vector source Signed-off-by: MOZGIII <mike-n@narod.ru> * Add values.yaml to vector chart Signed-off-by: MOZGIII <mike-n@narod.ru> * Correct the comment at kubernetesLogsSource at the vector-agent chart Signed-off-by: MOZGIII <mike-n@narod.ru> * Add vector chart E2E test Signed-off-by: MOZGIII <mike-n@narod.ru> * Set the proper header and file name at the vector-aggregator ConfigMap Signed-off-by: MOZGIII <mike-n@narod.ru> * Add a comment about the data dir location at the vector-agent config Signed-off-by: MOZGIII <mike-n@narod.ru> * Add the build-in vector sink to the vector-agent Signed-off-by: MOZGIII <mike-n@narod.ru> * Preconfigure agent and aggregator to work together out of the box at the vector chart Signed-off-by: MOZGIII <mike-n@narod.ru> * Update generated distribution/kubernetes/vector.yaml Signed-off-by: MOZGIII <mike-n@narod.ru> * Fix typo at distribution/helm/vector-agent/values.yaml Co-authored-by: Ana Hobden <operator@hoverbear.org> Signed-off-by: MOZGIII <mike-n@narod.ru> * Fix the PSP Signed-off-by: MOZGIII <mike-n@narod.ru> * Enable global env vars passing Signed-off-by: MOZGIII <mike-n@narod.ru> * Add persistentVolumeClaim and emptyDir to allowed volume types at PSP Signed-off-by: MOZGIII <mike-n@narod.ru> * Add a configmap checksum annotation to the StatefulSet Signed-off-by: MOZGIII <mike-n@narod.ru> * Correct service ports handling Signed-off-by: MOZGIII <mike-n@narod.ru> * Update the generated distribution/kubernetes/vector.yaml Signed-off-by: MOZGIII <mike-n@narod.ru> * Add the ability to pass extraContainersPorts for StatefulSet Pod Just ported over from the master implementation so far. Proper container ports for the built-in components is to be implemented. Signed-off-by: MOZGIII <mike-n@narod.ru> * Update the generated distribution/kubernetes/vector.yaml Signed-off-by: MOZGIII <mike-n@narod.ru> * Correct the managedPersistentVolumeClaim storage mode Signed-off-by: MOZGIII <mike-n@narod.ru> * Fix labels and annotations at volumeClaimTemplates Signed-off-by: MOZGIII <mike-n@narod.ru> * Update the generated distribution/kubernetes/vector.yaml Signed-off-by: MOZGIII <mike-n@narod.ru> * Allow choosing the test to run Signed-off-by: MOZGIII <mike-n@narod.ru> * Add a simple vector-aggregator chart test Signed-off-by: MOZGIII <mike-n@narod.ru> * Remove the features from k8s-e2e-tests Signed-off-by: MOZGIII <mike-n@narod.ru> * Add header to distribution/helm/vector/values.yaml Signed-off-by: MOZGIII <mike-n@narod.ru> * Move global values docs to the vector chart Signed-off-by: MOZGIII <mike-n@narod.ru> * Adjust the documentation of the global values slightly Signed-off-by: MOZGIII <mike-n@narod.ru> * Remove an extra space from distribution/helm/vector-agent/templates/daemonset.yaml Signed-off-by: MOZGIII <mike-n@narod.ru> * Do not add the checksum annotation if the external config is used Signed-off-by: MOZGIII <mike-n@narod.ru> * Backport the removal of the hardcoded configs Signed-off-by: MOZGIII <mike-n@narod.ru> * Hide the extra volumes comment if no extra volumes are defined Signed-off-by: MOZGIII <mike-n@narod.ru> * Set the extraConfigDirSources for vector-aggregator at scripts/kubernetes-yaml/values.yaml Signed-off-by: MOZGIII <mike-n@narod.ru> * Switch the scripts/kubernetes-yaml/values.yaml to use global key Signed-off-by: MOZGIII <mike-n@narod.ru> * Eliminate the warning about skipping globals Signed-off-by: MOZGIII <mike-n@narod.ru> * Correct the commonEnvKV docs Signed-off-by: MOZGIII <mike-n@narod.ru> * More adjustments to the globals docs at distribution/helm/vector/values.yaml Signed-off-by: MOZGIII <mike-n@narod.ru> * Add a comment about sliding tags Signed-off-by: MOZGIII <mike-n@narod.ru> * Update the generated distribution/kubernetes/vector.yaml Signed-off-by: MOZGIII <mike-n@narod.ru> * Add Kubernetes YAML generation for the vector-aggregator Signed-off-by: MOZGIII <mike-n@narod.ru> * Update generated Kubernetes YAMLs Signed-off-by: MOZGIII <mike-n@narod.ru> * Fix clippy offenses Signed-off-by: MOZGIII <mike-n@narod.ru> * Re-add features to the E2E tests to avoid running them accidentally Signed-off-by: MOZGIII <mike-n@narod.ru> Co-authored-by: Spencer Gilbert <sgilbert@optoro.com> Co-authored-by: Ana Hobden <operator@hoverbear.org> Signed-off-by: Brian Menges <brian.menges@anaplan.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3227.
This PR implements the design outlined at #3227 (comment). The chart is based on #3840.
To do:
global- see https://helm.sh/docs/chart_template_guide/subcharts_and_globals/Serviceportsconfigurablecontainerportsconfigurableability to disablerejected: people should use individual charts for now if they don't want the full packagevector-agentorvector-aggregatorwhen usingvectorchartvector-all.yaml,vector-agent.yamlandvector-aggregator.yaml, ensure they're correct (Restructure generated Kubernetes YAMLs to match Helm charts #5042 tracks this, so doesn't have to happen at this PR - however it makes sense to do the second part at this PR)