We can't add a new api using operator-sdk 0.0.6 - must run command in project root dir: stat ./build/Dockerfile: no such file or directory

[cmoulliard]

Bug Report

Command issued

operator-sdk add api --api-version=component.k8s.io/v1alpha1 --kind=Export
2018/10/23 17:42:55 must run command in project root dir: stat ./build/Dockerfile: no such file or directory

Tree of the local project

 tree -I vendor
.
├── Gopkg.lock
├── Gopkg.toml
├── Makefile
├── README.md
├── cmd
│   ├── component-operator
│   │   └── main.go
│   └── sd
│       └── sd.go
├── config
│   └── config.yaml
├── deploy
│   ├── cr.yaml
│   ├── crd.yaml
│   ├── operator.yaml
│   ├── rbac.yaml
│   └── sa.yaml
├── examples
│   ├── component-crud.yml
│   ├── component-link.yml
│   ├── component-route.yml
│   ├── component-service.yml
│   ├── component-vertx.yml
│   └── component.yml
├── hack
│   ├── k8s-api.sh
│   └── steps.md
├── pkg
│   ├── apis
│   │   └── component
│   │       └── v1alpha1
│   │           ├── doc.go
│   │           ├── register.go
│   │           ├── types.go
│   │           └── zz_generated.deepcopy.go
│   ├── client
│   │   └── cmd
│   │       ├── create.go
│   │       ├── root.go
│   │       └── version.go
│   ├── integration
│   │   ├── enum_test.go
│   │   ├── enum_weekday.go
│   │   └── json_test.go
│   ├── stub
│   │   ├── handler.go
│   │   └── pipeline
│   │       ├── generic
│   │       │   ├── pvc.go
│   │       │   ├── route.go
│   │       │   └── service.go
│   │       ├── innerloop
│   │       │   ├── deploymentconfig.go
│   │       │   ├── images.go
│   │       │   ├── install.go
│   │       │   └── resources.go
│   │       ├── link
│   │       │   └── link.go
│   │       ├── servicecatalog
│   │       │   └── install.go
│   │       └── step.go
│   └── util
│       ├── kubernetes
│       │   ├── config.go
│       │   ├── convert.go
│       │   └── namespace.go
│       ├── oc
│       │   └── oc.go
│       ├── openshift
│       │   └── register.go
│       └── template
│           ├── assets.go
│           ├── assets_vfsdata.go
│           ├── doc.go
│           ├── template.go
│           └── tmpl
│               ├── innerloop
│               │   ├── deploymentconfig
│               │   ├── imagestream
│               │   ├── pvc
│               │   ├── route
│               │   └── service
│               └── servicecatalog
│                   ├── servicebinding
│                   └── serviceinstance
├── tmp
│   ├── _output
│   │   └── bin
│   │       ├── component-operator
│   │       ├── sb
│   │       ├── sd
│   │       └── spring-boot-operator
│   ├── build
│   │   ├── Dockerfile
│   │   ├── build.sh
│   │   ├── go-test.sh
│   │   └── test-framework
│   │       └── Dockerfile
│   └── codegen
│       ├── boilerplate.go.txt
│       └── update-generated.sh
└── version
    └── version.go

Environment

• operator-sdk version: 0.0.6+git

• Kubernetes version information:

  Insert output of kubectl version here

• Kubernetes cluster kind:

oc version
oc v3.11.0+0cbc58b
kubernetes v1.11.0+d4cacc0
features: Basic-Auth

Server https://192.168.99.50:8443
kubernetes v1.11.0+d4cacc0

• Are you writing your operator in ansible or go? go

[cmoulliard]

A temporary workaround is to create a symbolic link

ln -s ./tmp/build build           

But, now, I get such problems

operator-sdk add api --api-version=component.k8s.io/v1alpha1 --kind=Export
Create pkg/apis/component/v1alpha1/export_types.go
Create pkg/apis/addtoscheme_component_v1alpha1.go
Create deploy/crds/component_v1alpha1_export_cr.yaml
Create deploy/crds/component_v1alpha1_export_crd.yaml
2018/10/23 18:02:55 failed to update the RBAC manifest for the resource (component.k8s.io/v1alpha1, Export): failed to read role manifest /Users/dabou/Code/go-workspace/src/github.com/snowdrop/component-operator/deploy/role.yaml: open /Users/dabou/Code/go-workspace/src/github.com/snowdrop/component-operator/deploy/role.yaml: no such file or directory

[mhrivnak]

It looks like you're running from master, which broke compatibility after 0.0.7. I think the reported version of "0.0.6+git" is misleading, and was just an oversight that it didn't get updated in-code.

I suggest using the 0.0.7 release for now. When master gets released as 0.1.0, there will be a guide to dealing with the broken compatibility.

[cmoulliard]

Can we add an api or operator with version 0.0.7 or the only way is to create a new project ? @mhrivnak

[hasbro17]

If you build the operator-sdk from the master you should be able to to use the commands add api and add controller but running them in your existing project might cause some issues.

We're working on a migration guide right now in which you essentially have to create a new project and copy your reconcile code and other files over because there are some differences in the project layout post v0.0.7. We'll have the guide finished and v0.1.0 release out soon.
#653

It is possible to first fix up your project layout first and then run add api and add controller to migrate your project in place but we haven't documented the steps for that yet, so that's not currently supported. Plus only running add api would still cause issues since that type wouldn't be registered with the sdk scheme.

We're adding a doc to highlight the layout differences but you can compare the sample memcached-operator in the meanwhile to see the differences:
v0.0.7-memcached-operator
v0.0.7+git-memcached-operator
https://github.com/operator-framework/operator-sdk/blob/master/doc/project_layout.md

[cmoulliard]

We still have the issue reported within the title of this ticket using the operator-sdk version 0.1.0

operator-sdk add api --api-version=component.k8s.io/v1alpha1 --kind=Component
2018/11/16 09:25:03 must run command in project root dir: stat ./build/Dockerfile: no such file or directory

The workaround is to copy the Dockerfile under the ./build directory

Question : Why the command adding a new API is looking about the Dockerfile which is only needed when we build the docker image if the operator ?

[lilic]

@cmoulliard This is because the new commands creates the build/Dockerfile and there is a check to make sure that while running the add command the user is in the root directory, that check looks for the Dockerfile. This is because it is also used during other commands such as build.

This check is used in ansible as well and that is the only common file hence why we look for the Dockerfile. I think the error msg can be cleared up a bit, but we can't check for any other file right now.

@cmoulliard I am curious why do you not have a Dockerfile there in the first place? And what would you like the outcome to be?

[cmoulliard]

Dockerfile there in the first place

Just because I have created a circleci job doing the work to compile the go binary and next to create our operator's image - See here

[cmoulliard]

And what would you like the outcome to be?

A warning message explaining that the Dockerfile is not defined under the following ./build/ directory and then the operator-sdk build could fail. TBH, I would prefer that no message is displayed when we generate the code of a new API as that will confuse the developers

[joelanford]

One option would be that we change our logic for how we decide we're in a project root directory. Right now, we check ./build/Dockerfile for all operator types, but we could instead just check that we have a valid operator type.

Our GetOperatorType() function currently assumes we're in a project root, but there's no reason it has to. Its logic could be something like:

func GetOperatorType() OperatorType {
    if IsGoOperator() {
        return OperatorTypeGo
    }
    if IsAnsibleOperator() {
        return OperatorTypeAnsible
    }
    return OperatorTypeUnknown
}

Then MustInProjectRoot would look something like:

func MustInProjectRoot() {
    if GetOperatorType() == OperatorTypeUnknown {
        log.Fatal("must run command in project root dir: could not identify operator type")
    }
}

With this type of check, we would no longer need to rely on a single file that's present in every type of operator for an operation that doesn't depend on that file.

[lilic]

@joelanford Overall that sounds like a good option. For the IsGoOperator we could just keep whatever we have currently in the GetOperatorType. What would the IsAnsibleOperator look like then? cc @shawn-hurley for any further ideas.

[shawn-hurley]

So, I think that requiring a dockerfile location to be somewhere is reasonable. We are an opinionated CLI that asks that the structure of the project looks a certain way. While I think @joelanford is right, and we could do something like what he is saying. I don't see why the check for ./Build/Dockerfile is wrong, if you are not in the project root, (and that is defined by a certain structure, including build/Dockerfile) then we should error out.

Obviously, this could change as we progress with the docker file management stuff, but I think it is fine to say "We expect the project to look like X, if it does not then we don't know what to do." Thoughts?

[lilic]

@shawn-hurley Agreed, but that check should not be called MustInProjectRoot but maybe MustHaveOperatorSDKStructure or something, if you get what I am saying? And we should adjust the error msg to reflect that.

[shawn-hurley]

Fair, I am ok with changing the error message and function name, but what would it be? wouldn't it be just as nebulous as asking to be in the project root? I run this command from say /pkg and we say "Must have the correct operator sdk structure" but they do they are just in the wrong directory?

maybe a combo message but that gets wordy, I do think that we probably want to remove the error message from that message. What about "Expected correct structure and root directory to have ./build/Dockerfile"? This is confusing though IMO. I don't know what the right thing is to do with the error message.

[lilic]

@shawn-hurley Yes I tried to come up with something as well, but failed to think of something that makes sense. :D But that sgtm, rephrased a bit. :)

@cmoulliard since you opened the original issue about this, would this make more sense to you:

"Expected correct project structure and root directory to have ./build/Dockerfile. Double check that the ./build/Dockerfile is present and the command is run from root directory."