Provide mechanism for config based kube dns service #2016
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Removed the hardcoded kube dns service Signed-off-by: JerrinAndrei <105700145+JerrinAndrei@users.noreply.github.com>
Update nginx.conf file with the kube_dns_service name configured in configmap Signed-off-by: JerrinAndrei <105700145+JerrinAndrei@users.noreply.github.com>
Added a new config "PL_KUBE_DNS_SERVICE" which can store the name of the kubernetes dns service in the cluster Signed-off-by: JerrinAndrei <105700145+JerrinAndrei@users.noreply.github.com>
|
@JerrinAndrei thanks for opening this PR and the GitHub issue! After thinking about this and the IPV6 fix (#2013) more, I think the existing The ideal the solution would make it possible to facilitate additional nginx configuration tweaks should another use case arise. After discussing this with the team today, we believe #2018 will serve both of these use cases and future ones well. Would appreciate any feedback you have on that approach and we can proceed with that instead of my earlier IPv6 PR and this one. |
ddelnano
added a commit
that referenced
this pull request
Sep 16, 2024
…igmaps for easier runtime overrides (#2018) Summary: Remove nginx config files from cloud proxy container in favor of Configmaps for easier runtime overrides This is an alternative approach to #2014 and #2016. While this doesn't provide an environment variable for configuring the intended behavior, this approach is more flexible since many Nginx directives don't work with variables (`server_name`, `resolver`, among others ). Because nginx prohibits variables in these directives, it makes it very difficult to provide environment variable based settings without our previous `sed` approach. The `sed` approach also has its problems since it requires [hacks](https://github.com/pixie-io/pixie/pull/2014/files#diff-5ec7ca8d0f624fe1f4eb3778cc96dcee2f999bf39bad422807b67b15ce2f8e7bR27) to support configuration removals. Rather than trying to solve all potential use cases, this PR opts to make the configuration easy to swap out via the `pl-proxy-nginx-config` Configmap. I plan to update the self hosted cloud docs to call out that this Configmap exists and should be used if custom nginx configuration is needed outside of the upstream defaults. Relevant Issues: #2017 Type of change: /kind feature Test Plan: Deployed to a cloud environment and verified that the upstream defaults and `PL_DOMAIN_NAME` apply as expected Changelog Message: Removed nginx configuration from the container image into `pl-proxy-nginx-config` Configmap for easier runtime overrides --------- Signed-off-by: Dom Del Nano <ddelnano@gmail.com>
|
Thanks for raising this issue @JerrinAndrei. Since I didn't hear back on my earlier comment, I decided to proceed with #2018. Please let me know if you have any trouble using that and I'm happy to help. |
|
This looks good to me @ddelnano . Most of the components in nginx conf would vary from cluster to cluster, so moving this to a configmap would give the entire configuration control to the user. |
ddelnano
added a commit
to ddelnano/pixie
that referenced
this pull request
Sep 23, 2024
…igmaps for easier runtime overrides (pixie-io#2018) Summary: Remove nginx config files from cloud proxy container in favor of Configmaps for easier runtime overrides This is an alternative approach to pixie-io#2014 and pixie-io#2016. While this doesn't provide an environment variable for configuring the intended behavior, this approach is more flexible since many Nginx directives don't work with variables (`server_name`, `resolver`, among others ). Because nginx prohibits variables in these directives, it makes it very difficult to provide environment variable based settings without our previous `sed` approach. The `sed` approach also has its problems since it requires [hacks](https://github.com/pixie-io/pixie/pull/2014/files#diff-5ec7ca8d0f624fe1f4eb3778cc96dcee2f999bf39bad422807b67b15ce2f8e7bR27) to support configuration removals. Rather than trying to solve all potential use cases, this PR opts to make the configuration easy to swap out via the `pl-proxy-nginx-config` Configmap. I plan to update the self hosted cloud docs to call out that this Configmap exists and should be used if custom nginx configuration is needed outside of the upstream defaults. Relevant Issues: pixie-io#2017 Type of change: /kind feature Test Plan: Deployed to a cloud environment and verified that the upstream defaults and `PL_DOMAIN_NAME` apply as expected Changelog Message: Removed nginx configuration from the container image into `pl-proxy-nginx-config` Configmap for easier runtime overrides --------- Signed-off-by: Dom Del Nano <ddelnano@gmail.com> GitOrigin-RevId: 9b5f295
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently the nginx.conf expects the a kubernetes dns service named kube-dns.kube-system.svc.cluster.local to be existing in the cluster. If this dns service does not exist the cloud-proxy pod gets terminated on startup. This issue has been addressed in this PR. Here, we are moving the kubeernetes dns service name to a config and updating the nginx conf with this name at startup. This will prevent the container failing to be up on clusters that do not have a standard naming convention for the dns service.