New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support updating config map and secret with --from-file #30558

Open
pwittrock opened this Issue Aug 13, 2016 · 45 comments

Comments

@pwittrock
Member

pwittrock commented Aug 13, 2016

It is possible to create a config map using kubectl create configmap foo --from-file bar.txt. We support similarly updating the config map using kubectl update configmap foo --from-file bar.txt.

@bgrant0607

This comment has been minimized.

Member

bgrant0607 commented Aug 30, 2016

See also #25832

@bgrant0607

This comment has been minimized.

Member

bgrant0607 commented Aug 30, 2016

and #24049

mengqiy added a commit to mengqiy/kubernetes that referenced this issue Sep 7, 2016

mengqiy added a commit to mengqiy/kubernetes that referenced this issue Sep 7, 2016

mengqiy added a commit to mengqiy/kubernetes that referenced this issue Sep 7, 2016

mengqiy added a commit to mengqiy/kubernetes that referenced this issue Sep 7, 2016

mengqiy added a commit to mengqiy/kubernetes that referenced this issue Sep 9, 2016

mengqiy added a commit to mengqiy/kubernetes that referenced this issue Sep 9, 2016

@mengqiy

This comment has been minimized.

Contributor

mengqiy commented Jun 5, 2017

/sig cli

@luxas

This comment has been minimized.

Member

luxas commented Jun 5, 2017

@mengqiy Are you working on this?

@mengqiy

This comment has been minimized.

Contributor

mengqiy commented Jun 5, 2017

@luxas Not any more.
Feel free to take this if you want. I can help review.

@luxas

This comment has been minimized.

Member

luxas commented Jun 6, 2017

@mengqiy I'm not planning to work on this, have a bunch of other stuff going on, but given the amount of +1 on this and the inactivity, I wanted to make this free for new/other contributors.

@jamiehannaford

This comment has been minimized.

Member

jamiehannaford commented Jun 6, 2017

I might have some spare cycles to work on this

@arthur0

This comment has been minimized.

Contributor

arthur0 commented Jun 6, 2017

Can I work on it?

@luxas

This comment has been minimized.

Member

luxas commented Jun 6, 2017

@arthur0 Sounds great!
Talked to @jamiehannaford and he will help out on some sig-cluster-lifecycle-related issues instead, so this one seems free to pick up!

Can't technically assign this one to you in the Github UI, but feel assigned and work on it 👍

@mengqiy

This comment has been minimized.

Contributor

mengqiy commented Sep 22, 2017

update is not a command.

@bgrant0607 Back in 2016 Q3, kubectl update is the alias of kubectl replace.
At that time, this feature request looks like it would improve the UX.

But after reading your comment #30558 (comment), I found this approach may turn out to encourge the users to do something dangerous. So should we leave a comment and close this issue?

We should always encourage users to create a new configmap and switch to it.

@bgrant0607

This comment has been minimized.

Member

bgrant0607 commented Sep 22, 2017

@mengqiy update was an alias, but has been deprecated since 1.0.
9b3d42c#diff-1569293db00490c928c40f53fc14800c

Yes, it is potentially dangerous, but convenient in some cases.

In any case, more thought needs to be put into the UX, because lots of kubectl replace configmap ..., kubectl apply configmap ..., etc. commands doesn't feel right, especially if some resource types will be supported but others would not.

@liggitt commented pretty much the same thing a year ago:
#33335 (comment)

#14040 requested create-or-replace ages ago, and we always intended for PUT to have that behavior in the API (#2114).

What about a --replace-existing flag to create?

@husseincoder

This comment has been minimized.

husseincoder commented Oct 12, 2017

@bgrant0607 @pwittrock @liggitt Guys its been so long since last interaction here .. should I close my PR and start implementing --replace-existing flag to create instead? please let me know .. I'm really passionate to get my first PR merged and start working on a new one :)

@itmard

This comment has been minimized.

itmard commented Oct 26, 2017

We have this issue and I want a tool to easily update configmaps create from directory and files.

@bgrant0607

This comment has been minimized.

Member

bgrant0607 commented Oct 26, 2017

@husseincoder Sorry, yes, please close your PR. If you are eager to contribute, please start with bug fixes, of which we have an overwhelming number. Contact sig-cli to ask for starter issues.

@itmard We are currently working on #24744 and #22368 in this area.

@yonran

This comment has been minimized.

yonran commented Nov 1, 2017

I hope kubectl replace configmap will support --from-file. In the meantime, I’ve been using some bash scripts to convert --from-file to the json structure required by kubectl replace: https://gist.github.com/yonran/c5590a52179aa4841d1067991338b920

@dinvlad

This comment has been minimized.

dinvlad commented Dec 5, 2017

Are there plans to also support --from-env-file for updates? Thanks

@tjma2001

This comment has been minimized.

tjma2001 commented Dec 11, 2017

@dinvlad this is the easiest workaround for now

kubectl create configmap --dry-run -o yaml --from-file=_env env | kubectl replace -f -

@dinvlad

This comment has been minimized.

dinvlad commented Dec 11, 2017

@tjma2001 nice, haven't thought of it this way. Dry-running with --from-env-file and apply in our case should work just as well.

@fejta-bot

This comment has been minimized.

fejta-bot commented Mar 16, 2018

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

@irvifa

This comment has been minimized.

irvifa commented Mar 26, 2018

We can use

kubectl create configmap some-config --from-file=some-key=some-config.yaml -n some-namaespace -o yaml --dry-run | kubectl apply -f -  

This will create as well update ConfigMap if it's not exist yet

@cakraww

This comment has been minimized.

cakraww commented Mar 27, 2018

@irvifa doesn't work in cloud builder :(

@irvifa

This comment has been minimized.

irvifa commented Mar 30, 2018

@cakraww yes, unfortunately this won't work in container builder, if we look into this link, we will know that the init script is supposed to receive only kubectl arguments.

FROM gcr.io/cloud-builders/gcloud

COPY kubectl.bash /builder/kubectl.bash

ENTRYPOINT ["/builder/kubectl.bash"]

kubectl.bash

#!/bin/bash

# If there is no current context, get one.
if [[ $(kubectl config current-context 2> /dev/null) == "" ]]; then
    cluster=$(gcloud config get-value container/cluster 2> /dev/null)
    zone=$(gcloud config get-value compute/zone 2> /dev/null)
    project=$(gcloud config get-value core/project 2> /dev/null)

    function var_usage() {
        cat <<EOF
No cluster is set. To set the cluster (and the zone where it is found), set the environment variables
  CLOUDSDK_COMPUTE_ZONE=<cluster zone>
  CLOUDSDK_CONTAINER_CLUSTER=<cluster name>
EOF
        exit 1
    }

    [[ -z "$cluster" ]] && var_usage
    [[ -z "$zone" ]] && var_usage

    echo "Running: gcloud container clusters get-credentials --project=\"$project\" --zone=\"$zone\" \"$cluster\""
    gcloud container clusters get-credentials --project="$project" --zone="$zone" "$cluster" || exit
fi

echo "Running: kubectl $@"
kubectl "$@"

Actually in bash if we can replace kubectl in the init script with exec "$@" instead, however I'm still finding another difficulty since any special character just won't work in container builder yaml syntax. I wonder if anyone in there can explained it in details.

@irvifa

This comment has been minimized.

irvifa commented Apr 9, 2018

Based on #22 we can use:

- name: 'gcr.io/cloud-builders/gcloud'
  id: 'test entrypoint bash'
  entrypoint: 'bash'
  args:
  - '-c'
  - |
    gcloud container clusters get-credentials --project=$PROJECT_ID --zone="<some-region>" "<cluster-name>"
    kubectl create configmap <configmap-name> --from-file=<key>=<file-name> -n <namespace-name> -o yaml --dry-run -o yaml | kubectl apply -f -
@jobernolte

This comment has been minimized.

jobernolte commented Apr 17, 2018

When I try to execute this command:

kubectl create configmap logstash-config --from-file=logstash.yml --from-file=logstash.conf --dry-run --save-config -o yaml | kubectl apply -f -
It gives me the following error:
error: error validating "STDIN": error validating data: [apiVersion not set, kind not set]; if you choose to ignore these errors, turn validation off with --validate=false

The output is somehow missing the apiVersion and kind attribute. Any clues on how to solve this?

@liggitt

This comment has been minimized.

Member

liggitt commented Apr 17, 2018

The output is somehow missing the apiVersion and kind attribute. Any clues on how to solve this?

This was a bug in 1.10.0 fixed in 1.10.1

@inthecloud247

This comment has been minimized.

inthecloud247 commented Apr 19, 2018

I'm seeing the same error in version 1.9.3, so it seems like this bug was introduced earlier than 1.10.0 ?

@liggitt

This comment has been minimized.

Member

liggitt commented Apr 20, 2018

I'm seeing the same error in version 1.9.3, so it seems like this bug was introduced earlier than 1.10.0 ?

No, it came in with 1.10.0

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.3", GitCommit:"d2835416544f298c919e2ead3be3d0864b52323b", GitTreeState:"clean", BuildDate:"2018-04-20T14:01:18Z", GoVersion:"go1.9.4", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.3", GitCommit:"d2835416544f298c919e2ead3be3d0864b52323b", GitTreeState:"clean", BuildDate:"2018-04-20T14:01:18Z", GoVersion:"go1.9.4", Compiler:"gc", Platform:"darwin/amd64"}

$ kubectl create configmap foo --from-literal key=value --dry-run -o yaml
apiVersion: v1
data:
  key: value
kind: ConfigMap
metadata:
  creationTimestamp: null
  name: foo

$ kubectl create configmap foo --from-literal key=value --dry-run -o yaml | kubectl apply -f -
configmap "foo" created

$ kubectl create configmap foo --from-literal key=value2 --dry-run -o yaml | kubectl apply -f -
configmap "foo" configured
@fejta-bot

This comment has been minimized.

fejta-bot commented May 20, 2018

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten
/remove-lifecycle stale

@neurostream

This comment has been minimized.

neurostream commented Jun 13, 2018

  • @bgrant0607 Skimming through each of the nuanced variations of the issue cited here, I'm not seeing any that Kustomize doesn't seem to address - and then some.
  • For others who will probably have Kustomize to look forward to; but are still working it into your tooling adoption in the days or weeks ahead and:
    • if you are stuck until then with:
      • an initial kubectl create -f app-config.yml
    • but later needed to add new ConfigMap K/V pairs --from-env-file=more-app-config.env
    • or if, in my case, just wanted to keep the original app-config.yml definition clean and separated from dirtier bits coming in --from-env-file=...
    • then - inspired by Brian Grant's original 2017-08-23 thoughts here in this thread - was able to get by at the time with ( brace yourself ) :
      • kubectl create -f app-config.yml --output=json | kubectl patch -f - -p "$(kubectl create configmap app-config --from-env-file=more-app-config.env --output=json --dry-run)"
@bgrant0607

This comment has been minimized.

Member

bgrant0607 commented Jun 15, 2018

@irvifa

This comment has been minimized.

irvifa commented Sep 19, 2018

@neurostream @bgrant0607 is it possible to do this without using additional feature such as kustomize? I wonder if we can include this inside helm chart as well. Is this even possible?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment