Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ERRO[0000] VirtulMachine.infra.sealyun.com "virtulmachine-test" not foundunable #947

Closed
fanux opened this issue Aug 16, 2019 · 21 comments

Comments

@fanux
Copy link

commented Aug 16, 2019

package main

import (
	"context"
	"fmt"
	"os"

	"k8s.io/apimachinery/pkg/types"

	v1 "github.com/fanux/sealvm/api/v1"
	"github.com/prometheus/common/log"
	"k8s.io/apimachinery/pkg/runtime"
	ctrl "sigs.k8s.io/controller-runtime"

	clientgoscheme "k8s.io/client-go/kubernetes/scheme"
)

var scheme = runtime.NewScheme()

func init() {
	v1.AddToScheme(scheme)
	clientgoscheme.AddToScheme(scheme)
}

func main() {
	mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
		Scheme: scheme,
	})
	if err != nil {
		os.Exit(1)
	}

	client := mgr.GetClient()
	ctx := context.Background()
	name := types.NamespacedName{Namespace: "default", Name: "virtulmachine-test"}

	vm := &v1.VirtulMachine{}
	if err := client.Get(ctx, name, vm); err != nil {
		log.Error(err, "unable to fetch vm")
	} else {
		fmt.Println(vm.Spec.CPU, vm.Spec.Memory, vm)
	}
}

But my CRD name is: virtulmachines.infra.sealyun.com not VirtulMachine.infra.sealyun.com , How can I specifie CRD name?

@fanux fanux added the kind/bug label Aug 16, 2019
@fanux

This comment has been minimized.

Copy link
Author

commented Aug 16, 2019

kubectl get virtulmachines.infra.sealyun.com 
NAME                   AGE
virtulmachine-test     38m

my yaml:

apiVersion: infra.sealyun.com/v1
kind: VirtulMachine
metadata:
  name: virtulmachine-test
spec:
  cpu: "1"
  memory: "2G"
  image: "centos.qcow"
status:
    status: "Createing"

source file:


// EDIT THIS FILE!  THIS IS SCAFFOLDING FOR YOU TO OWN!
// NOTE: json tags are required.  Any new fields you add must have json tags for the fields to be serialized.

// VirtulMachineSpec defines the desired state of VirtulMachine
type VirtulMachineSpec struct {
	// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
	// Important: Run "make" to regenerate code after modifying this file
	CPU    string `json:"cpu"`
	Memory string `json:"memory"`
	Image  string `json:"image"`
}

// VirtulMachineStatus defines the observed state of VirtulMachine
type VirtulMachineStatus struct {
	// INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
	// Important: Run "make" to regenerate code after modifying this file
	Status string `json:"status"`
}

// +kubebuilder:object:root=true
// +kubebuilder:subresource:status
// +kubebuilder:webhook:path=/vmhook,mutating=true,failurePolicy=fail,groups=infra.sealyun.com,resources=virtulmachine,verbs=create;update,versions=v1,name=virtulmachine.infra.sealyum.com

// VirtulMachine is the Schema for the virtulmachines API
type VirtulMachine struct {
	metav1.TypeMeta   `json:",inline"`
	metav1.ObjectMeta `json:"metadata,omitempty"`

	Spec   VirtulMachineSpec   `json:"spec,omitempty"`
	Status VirtulMachineStatus `json:"status,omitempty"`
}

// +kubebuilder:object:root=true

// VirtulMachineList contains a list of VirtulMachine
type VirtulMachineList struct {
	metav1.TypeMeta `json:",inline"`
	metav1.ListMeta `json:"metadata,omitempty"`
	Items           []VirtulMachine `json:"items"`
}

func init() {
	SchemeBuilder.Register(&VirtulMachine{}, &VirtulMachineList{})
}
@droot

This comment has been minimized.

Copy link
Contributor

commented Aug 16, 2019

name := types.NamespacedName{Namespace: "default", Name: "virtulmachine-test"}

	vm := &v1.VirtulMachine{}
	if err := client.Get(ctx, name, vm); err != nil {
		log.Error(err, "unable to fetch vm")
	} else {
		fmt.Println(vm.Spec.CPU, vm.Spec.Memory, vm)
	}

Code looks fine to me. Can you ensure you ran the Go program after creating the virtulmachines-test ?

Also, Can you share the output for kubectl get crds virtulmachines.infra.sealyun.com -o yaml ?

@fanux

This comment has been minimized.

Copy link
Author

commented Aug 18, 2019

@droot

kubectl get virtulmachines.infra.sealyun.com 
NAME                   AGE
virtulmachine-test     38m
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"apiextensions.k8s.io/v1beta1","kind":"CustomResourceDefinition","metadata":{"annotations":{},"creationTimestamp":null,"name":"virtulmachines.infra.sealyun.com","namespace":""},"spec":{"group":"infra.sealyun.com","names":{"kind":"VirtulMachine","plural":"virtulmachines"},"scope":"","subresources":{"status":{}},"validation":{"openAPIV3Schema":{"description":"VirtulMachine is the Schema for the virtulmachines API","properties":{"apiVersion":{"description":"APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources","type":"string"},"kind":{"description":"Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds","type":"string"},"metadata":{"type":"object"},"spec":{"description":"VirtulMachineSpec defines the desired state of VirtulMachine","properties":{"cpu":{"description":"INSERT ADDITIONAL SPEC FIELDS - desired state of cluster Important: Run \"make\" to regenerate code after modifying this file","type":"string"},"image":{"type":"string"},"memory":{"type":"string"}},"required":["cpu","image","memory"],"type":"object"},"status":{"description":"VirtulMachineStatus defines the observed state of VirtulMachine","properties":{"status":{"description":"INSERT ADDITIONAL STATUS FIELD - define observed state of cluster Important: Run \"make\" to regenerate code after modifying this file","type":"string"}},"required":["status"],"type":"object"}},"type":"object"}},"versions":[{"name":"v1","served":true,"storage":true}]},"status":{"acceptedNames":{"kind":"","plural":""},"conditions":[],"storedVersions":[]}}
  creationTimestamp: 2019-07-26T08:36:28Z
  generation: 5
  name: virtulmachines.infra.sealyun.com
  resourceVersion: "15127274"
  selfLink: /apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions/virtulmachines.infra.sealyun.com
  uid: 76294a52-af80-11e9-b63e-5254bc16e436
spec:
  conversion:
    strategy: None
  group: infra.sealyun.com
  names:
    kind: VirtulMachine
    listKind: VirtulMachineList
    plural: virtulmachines
    singular: virtulmachine
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      description: VirtulMachine is the Schema for the virtulmachines API
      properties:
        apiVersion:
          description: 'APIVersion defines the versioned schema of this representation
            of an object. Servers should convert recognized schemas to the latest
            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources'
          type: string
        kind:
          description: 'Kind is a string value representing the REST resource this
            object represents. Servers may infer this from the endpoint the client
            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds'
          type: string
        metadata:
          type: object
        spec:
          description: VirtulMachineSpec defines the desired state of VirtulMachine
          properties:
            cpu:
              description: 'INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
                Important: Run "make" to regenerate code after modifying this file'
              type: string
            image:
              type: string
            memory:
              type: string
          required:
          - cpu
          - image
          - memory
          type: object
        status:
          description: VirtulMachineStatus defines the observed state of VirtulMachine
          properties:
            status:
              description: 'INSERT ADDITIONAL STATUS FIELD - define observed state
                of cluster Important: Run "make" to regenerate code after modifying
                this file'
              type: string
          required:
          - status
          type: object
      type: object
  version: v1
  versions:
  - name: v1
    served: true
    storage: true
status:
  acceptedNames:
    kind: VirtulMachine
    listKind: VirtulMachineList
    plural: virtulmachines
    singular: virtulmachine
  conditions:
  - lastTransitionTime: 2019-07-26T08:36:28Z
    message: no conflicts found
    reason: NoConflicts
    status: "True"
    type: NamesAccepted
  - lastTransitionTime: null
    message: the initial names have been accepted
    reason: InitialNamesAccepted
    status: "True"
    type: Established
  storedVersions:
  - v1

I don't know why controller Reconcile work fine but I new a client in an other project main failed. CRD name changed: VirtulMachine.infra.sealyun.com not virtulmachines.infra.sealyun.com

@fanux

This comment has been minimized.

Copy link
Author

commented Aug 23, 2019

How to specified the CRD name in client?

@DirectXMan12

This comment has been minimized.

Copy link
Contributor

commented Sep 3, 2019

The resource name is pulled from discovery. Can you post the exact error message that you're seeing? Seeing the exact error might tell us where it's coming from (there are a couple from client-go that look similar, but come from different places).

@fanux

This comment has been minimized.

Copy link
Author

commented Sep 16, 2019

ERRO[0003] VirtulMachine.infra.sealyun.com "virtulmachine-sample" not found

➜ client git:(master) ✗ kubectl get crd
NAME AGE
virtulmachines.infra.sealyun.com 51d
➜ client git:(master) ✗ kubectl get virtulmachines.infra.sealyun.com
NAME AGE
virtulmachine-sample 49d

@fanux

This comment has been minimized.

Copy link
Author

commented Sep 16, 2019

Everything is ok when I useing client-go:

package main

import (
	"fmt"
	v1 "github.com/fanux/sealvm/api/v1"
	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
	"k8s.io/apimachinery/pkg/runtime"
	"k8s.io/apimachinery/pkg/runtime/schema"
	"k8s.io/apimachinery/pkg/runtime/serializer"
	"k8s.io/client-go/kubernetes/scheme"
	"k8s.io/client-go/rest"
	"k8s.io/client-go/tools/clientcmd"
	"log"
)

const GroupName = "infra.sealyun.com"
const GroupVersion = "v1"

var SchemeGroupVersion = schema.GroupVersion{Group: GroupName, Version: GroupVersion}

var (
	SchemeBuilder = runtime.NewSchemeBuilder(addKnownTypes)
	AddToScheme   = SchemeBuilder.AddToScheme
)

func addKnownTypes(scheme *runtime.Scheme) error {
	scheme.AddKnownTypes(SchemeGroupVersion,
		&v1.VirtulMachine{},
		&v1.VirtulMachineList{},
	)

	metav1.AddToGroupVersion(scheme, SchemeGroupVersion)
	return nil
}

func main() {
	var config *rest.Config
	var err error
	kubeconfig := "/Users/fanux/.kube/config"

	if kubeconfig == "" {
		log.Printf("using in-cluster configuration")
		config, err = rest.InClusterConfig()
	} else {
		log.Printf("using configuration from '%s'", kubeconfig)
		config, err = clientcmd.BuildConfigFromFlags("", kubeconfig)
	}

	if err != nil {
		panic(err)
	}

	AddToScheme(scheme.Scheme)

	crdConfig := *config
	crdConfig.ContentConfig.GroupVersion = &schema.GroupVersion{Group: GroupName, Version: GroupVersion}
	crdConfig.APIPath = "/apis"
	crdConfig.NegotiatedSerializer = serializer.DirectCodecFactory{CodecFactory: scheme.Codecs}
	crdConfig.UserAgent = rest.DefaultKubernetesUserAgent()

	exampleRestClient, err := rest.UnversionedRESTClientFor(&crdConfig)
	if err != nil {
		panic(err)
	}
	result := v1.VirtulMachineList{}

	err = exampleRestClient.
		Get().
		Resource("virtulmachines").
		Do().
		Into(&result)

	if err != nil {
		panic(err)
	}

	fmt.Println(result)
}
@DirectXMan12

This comment has been minimized.

Copy link
Contributor

commented Sep 16, 2019

Is ERRO[0003] VirtulMachine.infra.sealyun.com "virtulmachine-sample" not found the entirety of the error message? You don't have line information, or file information, or logger name, or anything like that?

@fanux

This comment has been minimized.

Copy link
Author

commented Sep 17, 2019

Full client error message is:

ERRO[0000] VirtulMachine.infra.sealyun.com "virtulmachine-test" not foundunable to fetch vm  source="main.go:39"

line 39:
image

@DirectXMan12

This comment has been minimized.

Copy link
Contributor

commented Sep 17, 2019

aaah, sorry, I completely missed something before, and seeing the full line number info drove it home. In the provided example, you're using the manger's client, which operates off the caches, but you haven't started the manager, so there's nothing in the cache. I think we have a footgun where we don't wait for cache sync if the manager hasn't even been started yet, that's probably your issue.

See kubernetes-sigs/controller-runtime#595 (comment), and let me know if that doesn't help you .

@DirectXMan12

This comment has been minimized.

Copy link
Contributor

commented Sep 17, 2019

(closing for bookkeeping, but we can re-open if the above doesn't help)

@fanux

This comment has been minimized.

Copy link
Author

commented Sep 17, 2019

I got it, but some times we don't want to run a long live process. For example query CRD in CNI command or create a command line to create CRD(like: kubectl plugin create vm -c2 -m1)
so

    if err := mgr.Start(ctrl.SetupSignalHandler()); err != nil {
        panic(err)
    }

does not meet our needs...

@DirectXMan12

This comment has been minimized.

Copy link
Contributor

commented Sep 17, 2019

If you don't want to read from the cache, use a "live" client by calling client.New instead of mgr.GetClient. Then, you don't have to have anything running in the background maintaining the cache. If you do want to read from a cache, you need something maintaining the cache. You probably want a live client in CLIs anyways.

@fanux

This comment has been minimized.

Copy link
Author

commented Sep 18, 2019

I got it, tks a lot.

@fanux

This comment has been minimized.

Copy link
Author

commented Sep 18, 2019

I am using client := mgr.GetAPIReader() problem solved.
I suggest that add a readwrite client in controller manager.
apiReader, err := client.New(config, client.Options{Scheme: options.Scheme, Mapper: mapper})
Some times we need that (without cache, not long live process).

GetReadWriteClient()client.Client

If you agree with me, I will PR those codes.

It is useful for command line or CNI and so on..

@DirectXMan12

This comment has been minimized.

Copy link
Contributor

commented Sep 18, 2019

You generally don't need to be using a manager (and shouldn't be) unless you're managing long-running processes, so it doesn't make much sense, IMO, to have that.

@fanux

This comment has been minimized.

Copy link
Author

commented Sep 19, 2019

Yes, I agree with you. But I think add a apiClient is reasonable. For if we need write status to apiserver (not through cache)...

@fanux

This comment has been minimized.

Copy link
Author

commented Sep 19, 2019

func getClient() (client.Client, error){
	config := ctrl.GetConfigOrDie()
	if config == nil {
		return nil, fmt.Errorf("config is nil")
	}
	options := ctrl.Options{Scheme:scheme}
	// Set default values for options fields
	options = setOptionsDefaults(options) 
	mapper, err := options.MapperProvider(config) // will panic
	if err != nil {
		log.Error(err, "Failed to get API Group-Resources")
		return nil, err
	}

	client, err := client.New(config, client.Options{Scheme: options.Scheme, Mapper: mapper})
	if err !=nil {
		return nil, err
	}
	return client,nil
}

as you can see, options = setOptionsDefaults(options) this is private function, so client.New is complex.

./client 
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1c69199]

goroutine 1 [running]:
main.getClient(0x150fe64, 0x2ab1b60, 0xc000372e00, 0x0)
	/Users/fanux/work/src/github.com/fanux/sealvm/client/main.go:31 +0x99
main.main()
	/Users/fanux/work/src/github.com/fanux/sealvm/client/main.go:45 +0x34
@DirectXMan12

This comment has been minimized.

Copy link
Contributor

commented Sep 19, 2019

write status to apiserver (not through cache)

writes never go through the cache anyway.

What you're looking for is

cl, err := client.New(config.GetConfigOrDie(), client.Options{Scheme: scheme})

There's an example in the godocs for the client: https://godoc.org/sigs.k8s.io/controller-runtime/pkg/client#New

@DirectXMan12

This comment has been minimized.

Copy link
Contributor

commented Sep 19, 2019

things like mappers are automatically defaulted

@fanux

This comment has been minimized.

Copy link
Author

commented Sep 20, 2019

Greate, I will close the PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.