odo project create fails for devfile feature on kubernetes cluster

[prietyc123]

/kind bug
/area testing

What versions of software are you using?

Operating System:
Linux

Output of odo version:
master

How did you run odo exactly?

Running devfile test script on travis through pr #2809

Actual behavior

Test Script fails while creating odo project odo project create

Expected behavior

Test Script should create the project successfully.

Any logs, error output, etc?

https://travis-ci.com/github/openshift/odo/jobs/316819721#L319

[prietyc123]

I am blocked as the test scripts are failing on travis ci configured with Kubernetes cluster.

Creating a new project: rvurnsriks
Running odo with args [odo project create rvurnsriks -w -v4]
[odo] I0408 11:13:45.189748   14644 preference.go:165] The path for preference file is /tmp/209434294/config.yaml
[odo]  •  Waiting for project to come up  ...
[odo]  ✗  Waiting for project to come up [26ms]
[odo]  ✗  the server could not find the requested resource (get projects.project.openshift.io)
Deleting project: 
Running odo with args [odo project delete  -f]
[odo]  ✗  The project  does not exist. Please check the list of projects using `odo project list`

Even after moving the experimental flag in BeforeEach for ex :

var _ = BeforeEach(func() {
		SetDefaultEventuallyTimeout(10 * time.Minute)
		context = helper.CreateNewContext()
		os.Setenv("GLOBALODOCONFIG", filepath.Join(context, "config.yaml"))
		helper.CmdShouldPass("odo", "preference", "set", "Experimental", "true")
		project = helper.CreateRandProject()
		currentWorkingDirectory = helper.Getwd()
		helper.Chdir(context)
	})

gives the same failure. Logs are :

Created dir: /tmp/209434294
Running odo with args [odo preference set Experimental true]
[odo] Global preference was successfully updated
Creating a new project: rvurnsriks
Running odo with args [odo project create rvurnsriks -w -v4]
[odo] I0408 11:13:45.189748   14644 preference.go:165] The path for preference file is /tmp/209434294/config.yaml
[odo]  •  Waiting for project to come up  ...
[odo]  ✗  Waiting for project to come up [26ms]
[odo]  ✗  the server could not find the requested resource (get projects.project.openshift.io)
Deleting project: 
Running odo with args [odo project delete  -f]
[odo]  ✗  The project  does not exist. Please check the list of projects using `odo project list`

For detailed info: https://travis-ci.com/github/openshift/odo/jobs/316819721

Ping @girishramnani @johnmcollier @GeekArthur

[GeekArthur]

@prietyc123 AFAIK, the experimental flag shouldn't affect the namespace/project level command like odo project, it only affects component level commands (eg. odo component, odo push). Also I think moving the experimental flag from It statement to BeforeEach only changes the order of executing the experimental flag setup, it won't affect the actual project creation. The running flow should be like this:

Case 1: experimental flag in It statement

1. Create project
2. Set experimental flag

Case 2: experimental flag in BeforeEach statement

1. Set experimental flag
2. Create project

If the test case fails on the two cases above, that means it fails even without experimental flag (without devfile feature), that sounds like a flaky testing environment issue, you can either remove the experimental flag setup code or retest to see if you still can see the issue.

FWIW, I manually try to create project with experimental flag, it works for me

odo preference view                                                                                               ✔  7s   11:22:29 
PARAMETER             CURRENT_VALUE
UpdateNotification
NamePrefix
Timeout
PushTimeout
Experimental          true
PushTarget
odo project create odo-test                                                                                 
 ✓  Project 'odo-test' is ready for use
 ✓  New project created and now using project: odo-test

And I run the failing test case on my environment it works as well

ginkgo -focus="odo devfile catalog command tests" tests/integration/devfile                                    ✔  11:22:50 
Running Suite: Devfile Suite
============================
Random Seed: 1586359652
Will run 2 of 23 specs

SSSSSSSSSSSSSSSSSSSSS
------------------------------
• [SLOW TEST:10.120 seconds]
odo devfile catalog command tests
/Users/jingfuwang/go/src/github.com/openshift/odo/tests/integration/devfile/cmd_devfile_catalog_test.go:13
  When executing catalog list components
  /Users/jingfuwang/go/src/github.com/openshift/odo/tests/integration/devfile/cmd_devfile_catalog_test.go:35
    should list all supported devfile components
    /Users/jingfuwang/go/src/github.com/openshift/odo/tests/integration/devfile/cmd_devfile_catalog_test.go:36
------------------------------
• [SLOW TEST:10.125 seconds]
odo devfile catalog command tests
/Users/jingfuwang/go/src/github.com/openshift/odo/tests/integration/devfile/cmd_devfile_catalog_test.go:13
  When executing catalog list components with -a flag
  /Users/jingfuwang/go/src/github.com/openshift/odo/tests/integration/devfile/cmd_devfile_catalog_test.go:43
    should list all supported and unsupported devfile components
    /Users/jingfuwang/go/src/github.com/openshift/odo/tests/integration/devfile/cmd_devfile_catalog_test.go:44
------------------------------

Ran 2 of 23 Specs in 20.246 seconds
SUCCESS! -- 2 Passed | 0 Failed | 0 Pending | 21 Skipped
PASS

Ginkgo ran 1 suite in 23.20108303s
Test Suite Passed

[kadel]

From the odo perspective, we need to rethink how we are going to handle projects vs namespace.

The original idea was to use terminology in odo that is independent of
OpenShift or Kubernetes. We could still use the project everywhere in
the odo. The original expectation was that the odo user doesn't know
anything about Kubernetes or OpenShift so the project might actually
make more sense then namespace. When working with the Kubernetes
cluster we would just create namespace instead of the project, but from odo point of view it will be still project.

Similarly, how for odo url create we are going to create Route by default when the user is pushing to OpenShift cluster (#2727) we could do the same here.
Command will always be odo project create but odo would check if it is Kubernetes or OpenShift and it will create Namespace or Project accordingly.

What do you think @elsony ?

[prietyc123]

@prietyc123 AFAIK, the experimental flag shouldn't affect the namespace/project level command like odo project, it only affects component level commands (eg. odo component, odo push). Also I think moving the experimental flag from It statement to BeforeEach only changes the order of executing the experimental flag setup, it won't affect the actual project creation. The running flow should be like this:

Case 1: experimental flag in It statement

1. Create project
2. Set experimental flag

Case 2: experimental flag in BeforeEach statement

1. Set experimental flag
2. Create project

If the test case fails on the two cases above, that means it fails even without experimental flag (without devfile feature), that sounds like a flaky testing environment issue, you can either remove the experimental flag setup code or retest to see if you still can see the issue.

FWIW, I manually try to create project with experimental flag, it works for me

odo preference view                                                                                               ✔  7s   11:22:29 
PARAMETER             CURRENT_VALUE
UpdateNotification
NamePrefix
Timeout
PushTimeout
Experimental          true
PushTarget
odo project create odo-test                                                                                 
 ✓  Project 'odo-test' is ready for use
 ✓  New project created and now using project: odo-test

And I run the failing test case on my environment it works as well

ginkgo -focus="odo devfile catalog command tests" tests/integration/devfile                                    ✔  11:22:50 
Running Suite: Devfile Suite
============================
Random Seed: 1586359652
Will run 2 of 23 specs

SSSSSSSSSSSSSSSSSSSSS
------------------------------
• [SLOW TEST:10.120 seconds]
odo devfile catalog command tests
/Users/jingfuwang/go/src/github.com/openshift/odo/tests/integration/devfile/cmd_devfile_catalog_test.go:13
  When executing catalog list components
  /Users/jingfuwang/go/src/github.com/openshift/odo/tests/integration/devfile/cmd_devfile_catalog_test.go:35
    should list all supported devfile components
    /Users/jingfuwang/go/src/github.com/openshift/odo/tests/integration/devfile/cmd_devfile_catalog_test.go:36
------------------------------
• [SLOW TEST:10.125 seconds]
odo devfile catalog command tests
/Users/jingfuwang/go/src/github.com/openshift/odo/tests/integration/devfile/cmd_devfile_catalog_test.go:13
  When executing catalog list components with -a flag
  /Users/jingfuwang/go/src/github.com/openshift/odo/tests/integration/devfile/cmd_devfile_catalog_test.go:43
    should list all supported and unsupported devfile components
    /Users/jingfuwang/go/src/github.com/openshift/odo/tests/integration/devfile/cmd_devfile_catalog_test.go:44
------------------------------

Ran 2 of 23 Specs in 20.246 seconds
SUCCESS! -- 2 Passed | 0 Failed | 0 Pending | 21 Skipped
PASS

Ginkgo ran 1 suite in 23.20108303s
Test Suite Passed

Thanks a lot @GeekArthur
May be in travis I am getting the flake due minikube version. In travis I am setting up minikube 0.25 for running devfile test against kubernetes cluster. Would you please let me know which minikube version you are using for your local testing ?
In the mean time I will test the same scenario locally against minikuke 0.25 locally just to confirm if its flake in CI.

[GeekArthur]

@prietyc123 I didn't use minikube, but yeah that could be the reason, and manually test the scenario locally should be very helpful 👍

[prietyc123]

@prietyc123 I didn't use minikube, but yeah that could be the reason, and manually test the scenario locally should be very helpful 👍

IMO it is the known issue that odo project create won't work on minikube as stated by tomas in #2846 (comment) .

@GeekArthur Can you please elaborate how you are testing manually ? Are you running the scripts on kubernetes cluster? Because I think first we need to handle projects vs namespace from odo prospective then only we can proceed further with using kubernetes cluster on minikube.

[GeekArthur]

@prietyc123 You are right, please ignore my previous testing as I think the fresh k8s cluster shouldn't have projectrequests.project.openshift.io API group for project creation. I agree this is known issue and we should handle the project vs namespace properly as Tomas suggests.