Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to run skaffold in linux system with Azure Kubernetes service #4443

Closed
anikm1987 opened this issue Jul 3, 2020 · 6 comments
Closed

Unable to run skaffold in linux system with Azure Kubernetes service #4443

anikm1987 opened this issue Jul 3, 2020 · 6 comments
Labels
area/deploy kind/bug Something isn't working platform/azure Issues related to Azure (including AKS) triage/duplicate This issue or pull request already exists

Comments

@anikm1987
Copy link

anikm1987 commented Jul 3, 2020 •
edited by briandealwis

Executed command skaffold dev --port-forward -vdebug not working. Getting below error -

INFO[0000] starting gRPC server on port 50051           
INFO[0000] starting gRPC HTTP server on port 50052      
INFO[0000] Skaffold &{Version:v1.12.0 ConfigVersion:skaffold/v2beta5 GitVersion: GitCommit:e680a831292e1c7efc54e0c6d40544ae141e6354 GitTreeState:clean BuildDate:2020-06-25T13:14:09Z GoVersion:go1.14.2 Compiler:gc Platform:linux/amd64} 
DEBU[0000] Defaulting build type to local build         
DEBU[0000] validating yamltags of struct SkaffoldConfig 
DEBU[0000] validating yamltags of struct Metadata       
DEBU[0000] validating yamltags of struct Pipeline       
DEBU[0000] validating yamltags of struct BuildConfig    
DEBU[0000] validating yamltags of struct Artifact       
DEBU[0000] validating yamltags of struct Sync           
DEBU[0000] validating yamltags of struct ArtifactType   
DEBU[0000] validating yamltags of struct DockerArtifact 
DEBU[0000] validating yamltags of struct TagPolicy      
DEBU[0000] validating yamltags of struct GitTagger      
DEBU[0000] validating yamltags of struct BuildType      
DEBU[0000] validating yamltags of struct LocalBuild     
DEBU[0000] validating yamltags of struct DeployConfig   
DEBU[0000] validating yamltags of struct DeployType     
DEBU[0000] validating yamltags of struct KubectlDeploy  
DEBU[0000] validating yamltags of struct KubectlFlags   
INFO[0000] Using kubectl context: akscluster00001       
DEBU[0000] found config for context "akscluster00001"   
DEBU[0000] Using builder: local                         
DEBU[0000] setting Docker user agent to skaffold-v1.12.0 
DEBU[0000] push value not present, defaulting to true because localCluster is false 
Listing files to watch...
 - ccoehp.azurecr.io/flaskapp
DEBU[0000] Found dependencies for dockerfile: [{hello.py /app/hello.py false} {requirements.txt / true}] 
INFO[0000] List generated in 2.2495ms                   
Generating tags...
 - ccoehp.azurecr.io/flaskapp -> DEBU[0000] Running command: [git describe --tags --always] 
DEBU[0000] unable to find git commit: running [git describe --tags --always]
 - stdout: ""
 - stderr: "fatal: not a git repository (or any of the parent directories): .git\n"
 - cause: exit status 128 
DEBU[0000] Using a fall-back tagger                     
ccoehp.azurecr.io/flaskapp:latest
Some taggers failed. Rerun with -vdebug for errors.
INFO[0000] Tags generated in 2.5711ms                   
Checking cache...
DEBU[0000] Found dependencies for dockerfile: [{hello.py /app/hello.py false} {requirements.txt / true}] 
 - ccoehp.azurecr.io/flaskapp: Found Remotely
INFO[0000] Cache check complete in 146.6993ms           
Tags used in deployment:
 - ccoehp.azurecr.io/flaskapp -> ccoehp.azurecr.io/flaskapp:latest@sha256:dc5ffea5a9be61dc5a057be8a1f10c634668ef2f87edd68e95a693f336f6265c
DEBU[0000] getting client config for kubeContext: ``    
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code EWUWECC7U to authenticate.
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x1683f90]

goroutine 1 [running]:
k8s.io/client-go/plugin/pkg/client/auth/azure.(*azureTokenSource).storeTokenInCfg(0xc00004ce40, 0xc0007dc6e0, 0xd, 0xc0007dc6e0)
        /skaffold/vendor/k8s.io/client-go/plugin/pkg/client/auth/azure/azure.go:276 +0x390
k8s.io/client-go/plugin/pkg/client/auth/azure.(*azureTokenSource).Token(0xc00004ce40, 0x0, 0x0, 0x0)
        /skaffold/vendor/k8s.io/client-go/plugin/pkg/client/auth/azure/azure.go:194 +0x453
k8s.io/client-go/plugin/pkg/client/auth/azure.(*azureRoundTripper).RoundTrip(0xc00011bdc0, 0xc0005fea00, 0x1d7c85a, 0xa, 0xc000693d80)
        /skaffold/vendor/k8s.io/client-go/plugin/pkg/client/auth/azure/azure.go:125 +0x81
k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip(0xc00011bde0, 0xc0005fe900, 0xc00011bde0, 0xbfb80315af4f4780, 0x77e053e3d)
        /skaffold/vendor/k8s.io/client-go/transport/round_trippers.go:159 +0x1c0
net/http.send(0xc0005fe800, 0x20e7340, 0xc00011bde0, 0xbfb80315af4f4780, 0x77e053e3d, 0x30a9320, 0xc00011e210, 0xbfb80315af4f4780, 0x1, 0x0)
        /usr/local/go/src/net/http/client.go:252 +0x43e
net/http.(*Client).send(0xc000698300, 0xc0005fe800, 0xbfb80315af4f4780, 0x77e053e3d, 0x30a9320, 0xc00011e210, 0x0, 0x1, 0x0)
        /usr/local/go/src/net/http/client.go:176 +0xfa
net/http.(*Client).do(0xc000698300, 0xc0005fe800, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/http/client.go:699 +0x44a
net/http.(*Client).Do(...)
        /usr/local/go/src/net/http/client.go:567
k8s.io/client-go/rest.(*Request).request(0xc0000c6a50, 0xc0008593a8, 0x0, 0x0)
        /skaffold/vendor/k8s.io/client-go/rest/request.go:801 +0x3e9
k8s.io/client-go/rest.(*Request).Do(0xc0000c6a50, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /skaffold/vendor/k8s.io/client-go/rest/request.go:873 +0xd8
k8s.io/client-go/discovery.(*DiscoveryClient).ServerVersion(0xc00011be60, 0x214b9e0, 0xc00011be60, 0x0)
        /skaffold/vendor/k8s.io/client-go/discovery/discovery_client.go:408 +0xa9
github.com/GoogleContainerTools/skaffold/pkg/skaffold/runner.failIfClusterIsNotReachable(0x20e7920, 0xc00000e018)
        /skaffold/pkg/skaffold/runner/deploy.go:124 +0x7a
github.com/GoogleContainerTools/skaffold/pkg/skaffold/runner.(*SkaffoldRunner).Deploy(0xc0004037c0, 0x212f060, 0xc00014cec0, 0x20e7920, 0xc00000e018, 0xc0006369c0, 0x1, 0x1, 0xc0006369a0, 0x1)
        /skaffold/pkg/skaffold/runner/deploy.go:56 +0x25c
github.com/GoogleContainerTools/skaffold/pkg/skaffold/runner.(*SkaffoldRunner).Dev(0xc0004037c0, 0x212f060, 0xc00014cec0, 0x20e7920, 0xc00000e018, 0xc00011e6b8, 0x1, 0x1, 0x0, 0x0)
        /skaffold/pkg/skaffold/runner/dev.go:226 +0x100c
github.com/GoogleContainerTools/skaffold/cmd/skaffold/app/cmd.runDev.func5(0x2150ae0, 0xc0004037c0, 0xc0004a7320, 0x0, 0x1010100010001)
        /skaffold/cmd/skaffold/app/cmd/dev.go:63 +0xad
github.com/GoogleContainerTools/skaffold/cmd/skaffold/app/cmd.withRunner(0x212f060, 0xc00014cec0, 0xc0006bfc10, 0x1dce3e5, 0x32)
        /skaffold/cmd/skaffold/app/cmd/runner.go:49 +0x12b
github.com/GoogleContainerTools/skaffold/cmd/skaffold/app/cmd.runDev(0x212f060, 0xc00014cec0, 0x20e7920, 0xc00000e018, 0x0, 0x0)
        /skaffold/cmd/skaffold/app/cmd/dev.go:62 +0x165
github.com/GoogleContainerTools/skaffold/cmd/skaffold/app/cmd.(*builder).NoArgs.func1(0xc000367080, 0xc00000d940, 0x0, 0x2, 0x0, 0x0)
        /skaffold/cmd/skaffold/app/cmd/commands.go:105 +0x98
github.com/spf13/cobra.(*Command).execute(0xc000367080, 0xc00000d920, 0x2, 0x2, 0xc000367080, 0xc00000d920)
        /skaffold/vendor/github.com/spf13/cobra/command.go:842 +0x453
github.com/spf13/cobra.(*Command).ExecuteC(0xc0003662c0, 0xc00000e018, 0x20e7920, 0xc00000e020)
        /skaffold/vendor/github.com/spf13/cobra/command.go:950 +0x349
github.com/spf13/cobra.(*Command).Execute(...)
        /skaffold/vendor/github.com/spf13/cobra/command.go:887
github.com/spf13/cobra.(*Command).ExecuteContext(...)
        /skaffold/vendor/github.com/spf13/cobra/command.go:880
github.com/GoogleContainerTools/skaffold/cmd/skaffold/app.Run(0x20e7920, 0xc00000e018, 0x20e7920, 0xc00000e020, 0x0, 0x0)
        /skaffold/cmd/skaffold/app/skaffold.go:33 +0xde
main.main()
        /skaffold/cmd/skaffold/skaffold.go:31 +0x4e

Information

  • Skaffold version: version...
    v1.12.0
  • kubectl version
kubectl version client
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.5", GitCommit:"e6503f8d8f769ace2f338794c914a96fc335df0f", GitTreeState:"clean", BuildDate:"2020-06-26T03:47:41Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.10", GitCommit:"ba1ce4122d67b0f7399a41afbbd8a7258d22ecb8", GitTreeState:"clean", BuildDate:"2020-04-21T00:13:25Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"linux/amd64"}

apiVersion: skaffold/v2beta5
kind: Config
metadata:
  name: iac-micorservice
build:
  tagPolicy:
    gitCommit: {}
  artifacts:
    - image: ccoehp.azurecr.io/flaskapp
      sync:
        infer:
          - '**/*.py'

deploy:
  kubectl:
    manifests:
      - flask_app_deployment.yaml
      - flask-app-service.yaml
  #kubeContext: akscluster00001

Steps to reproduce the behavior

  1. a clonable repository with the sample skaffold project
  2. skaffold <command>
    skaffold dev --port-forward -vdebug
  3. ...
    Below command gets executed successfully-
kubectl get pods -n ccoe-hp-devtest
NAME                                             READY   STATUS    RESTARTS   AGE
nginx-ingress-controller-596cfc569f-bjzl8        1/1     Running   0          8d
nginx-ingress-default-backend-576b86996d-rfxzz   1/1     Running   0          8d
@anikm1987
Copy link
Author

anikm1987 commented Jul 3, 2020 •
edited by briandealwis

Hi,
I ran the sample code as well and got the similar error

#Downloading the sample app
#Clone the Skaffold repository:
git clone https://github.com/GoogleContainerTools/skaffold

Change to the examples/getting-started in skaffold directory.

cd skaffold/examples/getting-started

Change k8s-pod.yaml as below to add the secret

apiVersion: v1
kind: Pod
metadata:
  name: getting-started
spec:
  containers:
  - name: getting-started
    image: skaffold-example
  imagePullSecrets:
        - name: ccoehpacrsecret

execute below command

skaffold dev --default-repo ccoehp.azurecr.io -vdebug -n ccoe-hp-devtest
INFO[0000] starting gRPC server on port 50051           
INFO[0000] starting gRPC HTTP server on port 50052      
INFO[0000] Skaffold &{Version:v1.12.0 ConfigVersion:skaffold/v2beta5 GitVersion: GitCommit:e680a831292e1c7efc54e0c6d40544ae141e6354 GitTreeState:clean BuildDate:2020-06-25T13:14:09Z GoVersion:go1.14.2 Compiler:gc Platform:linux/amd64} 
DEBU[0000] Defaulting build type to local build         
DEBU[0000] validating yamltags of struct SkaffoldConfig 
DEBU[0000] validating yamltags of struct Metadata       
DEBU[0000] validating yamltags of struct Pipeline       
DEBU[0000] validating yamltags of struct BuildConfig    
DEBU[0000] validating yamltags of struct Artifact       
DEBU[0000] validating yamltags of struct ArtifactType   
DEBU[0000] validating yamltags of struct DockerArtifact 
DEBU[0000] validating yamltags of struct TagPolicy      
DEBU[0000] validating yamltags of struct GitTagger      
DEBU[0000] validating yamltags of struct BuildType      
DEBU[0000] validating yamltags of struct LocalBuild     
DEBU[0000] validating yamltags of struct DeployConfig   
DEBU[0000] validating yamltags of struct DeployType     
DEBU[0000] validating yamltags of struct KubectlDeploy  
DEBU[0000] validating yamltags of struct KubectlFlags   
INFO[0000] Using kubectl context: akscluster00001       
DEBU[0000] found config for context "akscluster00001"   
DEBU[0000] Using builder: local                         
DEBU[0000] setting Docker user agent to skaffold-v1.12.0 
DEBU[0000] push value not present, defaulting to true because localCluster is false 
Listing files to watch...
 - skaffold-example
DEBU[0000] Found dependencies for dockerfile: [{main.go /go true}] 
INFO[0000] List generated in 3.7664ms                   
Generating tags...
 - skaffold-example -> DEBU[0000] Running command: [git describe --tags --always] 
DEBU[0000] Command output: [v1.12.0-32-gfc51e7f83
]     
DEBU[0000] Running command: [git status . --porcelain]  
DEBU[0000] Command output: [ M examples/getting-started/k8s-pod.yaml
] 
ccoehp.azurecr.io/skaffold-example:v1.12.0-32-gfc51e7f83-dirty
INFO[0000] Tags generated in 22.6592ms                  
Checking cache...
DEBU[0000] Found dependencies for dockerfile: [{main.go /go true}] 
 - skaffold-example: Found. Tagging
DEBU[0002] attempting to add tag ccoehp.azurecr.io/skaffold-example:v1.12.0-32-gfc51e7f83-dirty to src ccoehp.azurecr.io/skaffold-example:v1.12.0-32-gfc51e7f83-dirty@sha256:b1973146aadc607ddab9205182ccf0818c812475e3c5d90302ceffa1a1504606 
INFO[0037] Cache check complete in 37.1647429s          
Tags used in deployment:
 - skaffold-example -> ccoehp.azurecr.io/skaffold-example:v1.12.0-32-gfc51e7f83-dirty@sha256:b1973146aadc607ddab9205182ccf0818c812475e3c5d90302ceffa1a1504606
DEBU[0037] getting client config for kubeContext: ``    
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x1683f90]

goroutine 1 [running]:
k8s.io/client-go/plugin/pkg/client/auth/azure.(*azureTokenSource).storeTokenInCfg(0xc000847d40, 0xc000c56d10, 0xd, 0xc000c56d10)
        /skaffold/vendor/k8s.io/client-go/plugin/pkg/client/auth/azure/azure.go:276 +0x390
k8s.io/client-go/plugin/pkg/client/auth/azure.(*azureTokenSource).Token(0xc000847d40, 0x0, 0x0, 0x0)
        /skaffold/vendor/k8s.io/client-go/plugin/pkg/client/auth/azure/azure.go:194 +0x453
k8s.io/client-go/plugin/pkg/client/auth/azure.(*azureRoundTripper).RoundTrip(0xc00067fe80, 0xc00013fc00, 0x1d7c85a, 0xa, 0xc0007396a0)
        /skaffold/vendor/k8s.io/client-go/plugin/pkg/client/auth/azure/azure.go:125 +0x81
k8s.io/client-go/transport.(*userAgentRoundTripper).RoundTrip(0xc00067fea0, 0xc00013fb00, 0xc00067fea0, 0xbfb8055e67788634, 0x101e326ee1)
        /skaffold/vendor/k8s.io/client-go/transport/round_trippers.go:159 +0x1c0
net/http.send(0xc00013fa00, 0x20e7340, 0xc00067fea0, 0xbfb8055e67788634, 0x101e326ee1, 0x30a9320, 0xc00011cd98, 0xbfb8055e67788634, 0x1, 0x0)
        /usr/local/go/src/net/http/client.go:252 +0x43e
net/http.(*Client).send(0xc0006793b0, 0xc00013fa00, 0xbfb8055e67788634, 0x101e326ee1, 0x30a9320, 0xc00011cd98, 0x0, 0x1, 0x0)
        /usr/local/go/src/net/http/client.go:176 +0xfa
net/http.(*Client).do(0xc0006793b0, 0xc00013fa00, 0x0, 0x0, 0x0)
        /usr/local/go/src/net/http/client.go:699 +0x44a
net/http.(*Client).Do(...)
        /usr/local/go/src/net/http/client.go:567
k8s.io/client-go/rest.(*Request).request(0xc000026c30, 0xc0007f13a8, 0x0, 0x0)
        /skaffold/vendor/k8s.io/client-go/rest/request.go:801 +0x3e9
k8s.io/client-go/rest.(*Request).Do(0xc000026c30, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
        /skaffold/vendor/k8s.io/client-go/rest/request.go:873 +0xd8
k8s.io/client-go/discovery.(*DiscoveryClient).ServerVersion(0xc00067ff00, 0x214b9e0, 0xc00067ff00, 0x0)
        /skaffold/vendor/k8s.io/client-go/discovery/discovery_client.go:408 +0xa9
github.com/GoogleContainerTools/skaffold/pkg/skaffold/runner.failIfClusterIsNotReachable(0x20e7920, 0xc00011c008)
        /skaffold/pkg/skaffold/runner/deploy.go:124 +0x7a
github.com/GoogleContainerTools/skaffold/pkg/skaffold/runner.(*SkaffoldRunner).Deploy(0xc000167400, 0x212f060, 0xc000122000, 0x20e7920, 0xc00011c008, 0xc000412780, 0x1, 0x1, 0xc000412760, 0x1)
        /skaffold/pkg/skaffold/runner/deploy.go:56 +0x25c
github.com/GoogleContainerTools/skaffold/pkg/skaffold/runner.(*SkaffoldRunner).Dev(0xc000167400, 0x212f060, 0xc000122000, 0x20e7920, 0xc00011c008, 0xc00000e9d0, 0x1, 0x1, 0x0, 0x0)
        /skaffold/pkg/skaffold/runner/dev.go:226 +0x100c
github.com/GoogleContainerTools/skaffold/cmd/skaffold/app/cmd.runDev.func5(0x2150ae0, 0xc000167400, 0xc0000199e0, 0x0, 0x1010100010001)
        /skaffold/cmd/skaffold/app/cmd/dev.go:63 +0xad
github.com/GoogleContainerTools/skaffold/cmd/skaffold/app/cmd.withRunner(0x212f060, 0xc000122000, 0xc0006cbc10, 0x1dce3e5, 0x32)
        /skaffold/cmd/skaffold/app/cmd/runner.go:49 +0x12b
github.com/GoogleContainerTools/skaffold/cmd/skaffold/app/cmd.runDev(0x212f060, 0xc000122000, 0x20e7920, 0xc00011c008, 0x0, 0x0)
        /skaffold/cmd/skaffold/app/cmd/dev.go:62 +0x165
github.com/GoogleContainerTools/skaffold/cmd/skaffold/app/cmd.(*builder).NoArgs.func1(0xc0001e9600, 0xc000101220, 0x0, 0x5, 0x0, 0x0)
        /skaffold/cmd/skaffold/app/cmd/commands.go:105 +0x98
github.com/spf13/cobra.(*Command).execute(0xc0001e9600, 0xc0001011d0, 0x5, 0x5, 0xc0001e9600, 0xc0001011d0)
        /skaffold/vendor/github.com/spf13/cobra/command.go:842 +0x453
github.com/spf13/cobra.(*Command).ExecuteC(0xc0001e8000, 0xc00011c008, 0x20e7920, 0xc00011c010)
        /skaffold/vendor/github.com/spf13/cobra/command.go:950 +0x349
github.com/spf13/cobra.(*Command).Execute(...)
        /skaffold/vendor/github.com/spf13/cobra/command.go:887
github.com/spf13/cobra.(*Command).ExecuteContext(...)
        /skaffold/vendor/github.com/spf13/cobra/command.go:880
github.com/GoogleContainerTools/skaffold/cmd/skaffold/app.Run(0x20e7920, 0xc00011c008, 0x20e7920, 0xc00011c010, 0x0, 0x0)
        /skaffold/cmd/skaffold/app/skaffold.go:33 +0xde
main.main()
        /skaffold/cmd/skaffold/skaffold.go:31 +0x4e

@briandealwis briandealwis added area/deploy kind/bug Something isn't working platform/azure Issues related to Azure (including AKS) triage/duplicate This issue or pull request already exists labels Jul 5, 2020
@briandealwis
Copy link
Member

Duplicate of #2807 which we're hoping is fixed by #4436. This was just committed a few days ago, so you'll need to build from HEAD to give it a try.

@anikm1987
Copy link
Author

Hi,
Thanks for the information. How can I build from HEAD to give it a try? Can you please help me with some steps?

@anikm1987
Copy link
Author

anikm1987 commented Jul 5, 2020 •
edited

Hi,
I am able to do using
sudo apt install linuxbrew-wrapper
brew install skaffold --HEAD

But if there any other step without installing brew then please let me know.

Also now I can see it is getting deployed properly. The issue is fixed now.
Is there any plan for next release with this fix soon?

@gsquared94
Copy link
Collaborator

@anikm1987 you can find that in our documentation on how to install the latest bleeding edge binary

@gsquared94
Copy link
Collaborator

closing since duplicate of #2807

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/deploy kind/bug Something isn't working platform/azure Issues related to Azure (including AKS) triage/duplicate This issue or pull request already exists
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@briandealwis @anikm1987 @gsquared94