Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 1896101: Added negative tests for migration from VMWare and RHV #6249

Merged
merged 1 commit into from Nov 10, 2020
Merged

Conversation

ibragins
Copy link
Contributor

@ibragins ibragins commented Aug 6, 2020

…HV and VMWare.

Refactored framework to simplify error messaging when wrong input occur.

@openshift-ci-robot openshift-ci-robot added the component/kubevirt Related to kubevirt-plugin label Aug 6, 2020
await wizard.next();
}

it('RHV - Import Wizard shows warning when using incorrect VM name', async () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm I wonder if we should group this by feature (like naming) or by a step?. Like ...general-tab-scenario...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can leave it for now and refactor later once we feel a need for it

@ibragins
Copy link
Contributor Author

ibragins commented Sep 8, 2020

@suomiy @rhrazdil Can you please review once again? Most of your comments were applied, everything works and I need to complete this PR asap

Copy link
Member

@atiratree atiratree left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also, there are still some unresolved points from last reviews.

@ibragins
Copy link
Contributor Author

ibragins commented Sep 10, 2020 via email

Copy link
Member

@atiratree atiratree left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ibragins can you please take a look at these as well?

#6249 (comment)
#6249 (comment)
#6249 (comment)

@openshift-ci-robot openshift-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Sep 13, 2020
@ibragins
Copy link
Contributor Author

/retest

@ibragins
Copy link
Contributor Author

/test e2e-gcp-console

1 similar comment
@ibragins
Copy link
Contributor Author

/test e2e-gcp-console

@@ -101,7 +89,11 @@ export class RhvImportWizard extends ImportWizard {
// Clicking `edit` button to reach network and storage settings
await click(rhvView.editButton);
await click(view.nextButton);
await this.next();
await click(view.nextButton);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems very prone to breaking. Why are we doing this? Can we wait on some condition. Is Please wait for a VM to load. error getting triggered with the first next?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No that error, cause I'm waiting until spinner will end. If I leave only one click on next nothing will happen, button will not be pressed. I have no explanation why.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so it seems this is working purely by coincidence or timing on your env. Please debug it and introduce some consistent solution.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed by adding timeout to make sure this button will be clickable

@@ -101,7 +89,11 @@ export class RhvImportWizard extends ImportWizard {
// Clicking `edit` button to reach network and storage settings
await click(rhvView.editButton);
await click(view.nextButton);
await this.next();
await click(view.nextButton);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so it seems this is working purely by coincidence or timing on your env. Please debug it and introduce some consistent solution.

@dgur
Copy link

dgur commented Sep 24, 2020

@suomiy I see we have this PR in review for a long time already, can we be a bit more acceptable so we could progress to additional tests?

@atiratree
Copy link
Member

@dgur This PR is not waiting for a review ATM. There are still some unresolved issues.
Let me know if you do not have enough capacity to tackle them, so I can give it a go and fix them.

Copy link
Member

@atiratree atiratree left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are mistakes regarding timeouts and conditions - this test does not pass on my machine

// rhvVMConfigSecond,
rhvVMConfigStartOnCreate,
rhvVMMultiNicConfig,
} from './v2v.rhv.configs';
import { RhvImportWizard } from '../../tests/models/rhvImportWizard';
import { V2V_VM_IMPORT_TIMEOUT } from '../../tests/utils/constants/common';
import { VM_STATUS } from '../../tests/utils/constants/vm';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please fix the describe here

@@ -99,9 +91,13 @@ export class RhvImportWizard extends ImportWizard {
await this.selectSourceVirtualMachine(sourceVMName);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this await fails. Please make sure the elements are in focus and clickable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This await should look like this inside:

  async selectSourceVirtualMachine(sourceVirtualMachine: string) {
    await selectOptionByText(rhvView.ovirtVmSelect, sourceVirtualMachine);
  }

Does it look the same in your code? If not - it will fail

@@ -99,9 +91,13 @@ export class RhvImportWizard extends ImportWizard {
await this.selectSourceVirtualMachine(sourceVMName);
await this.waitForSpinner();
// Clicking `edit` button to reach network and storage settings
await browser.sleep(2000);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's not use hard timeout here, but wait for the element to be clickable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How I can check that element is clickable?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can use a combination of these https://www.protractortest.org/#/api?view=ProtractorExpectedConditions . Sometimes it is necessary to scroll the elem into view as well

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can also get inspired in ./frontend/packages/dev-console/integration-tests/utilities/elementInteractions.ts . eg selectCheckBox


// Review
await this.validateReviewTab(config);

// Import
await this.confirmAndCreate();
await browser.sleep(15000);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should rather pass higher timeout to waitForCreation then here

@@ -19,8 +24,19 @@ describe('Kubevirt create VM using wizard', () => {
V2V_VM_IMPORT_TIMEOUT,
);

// it(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this commented out?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I commented this test cause it was always failing, saying (suddenly) that usage of existing RHV instance is not implemented. I will get back to this test later, cause on CNV 2.4 everything worked fine.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

expect(err).toBeDefined();
});
await browser.sleep(10000);
await click(view.modalCancelButton);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please don't use hard sleep - let's rather wait for some condition

const err = await nicDialog.fillName(curValue);
expect(err).toBeDefined();
});
await browser.sleep(10000);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

@openshift-ci-robot openshift-ci-robot added the component/ceph Related to ceph-storage-plugin label Oct 7, 2020
@ibragins
Copy link
Contributor Author

ibragins commented Oct 7, 2020

/test kubevirt-plugin

@atiratree
Copy link
Member

@ibragins There is a new update of #6850.

Can you also please cleanup the git and show only the relevant commits here? That is your commits and commits from #6850.

@openshift-ci-robot openshift-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. component/core Related to console core functionality component/dashboard Related to dashboard component/dev-console Related to dev-console component/knative Related to knative-plugin component/metal3 Related to metal3-plugin component/noobaa Related to noobaa-storage-plugin labels Oct 9, 2020
@openshift-ci-robot openshift-ci-robot added component/olm Related to OLM component/shared Related to console-shared and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Oct 9, 2020
@ibragins
Copy link
Contributor Author

/test kubevirt-plugin

@openshift-ci-robot openshift-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Oct 20, 2020
.setTemplate(vmt.name)
.setName('from-template')
.build();
const vm = new VMBuilder(getBasicVMBuilder()).setName('from-template').build();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please don't remove the templates. Anyway this whole commit should not be here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't touch this file, I have no idea how it got changed somehow. I will re-base it from the master

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, file recovered

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the commit is still there, can you please remove it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just copied content of the file to my local file before commit, so it should be equal to what is placed on the server

@@ -25,11 +25,30 @@ const exInstanceConfig: InstanceConfig = {

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we have distinct credentials env variables for vmware and rhv import, so there is no overlap between them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used names that should be obvious for the one who should run those tests, also there are different parameters required for RHV and VMWare. How do you suggest to change them?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe with RHV VMWARE prefix?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please give me example of exact change, not sure I've got you correctly

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for example V2V_VMWARE_INSTANCE_API_URL and V2V_RHV_INSTANCE_API_URL

// importedNICs = importedNICs.filter((_, i) => i % 11 === 0);
await asyncForEach(importedNICs, async (nic) => {
// eslint-disable-next-line no-console
console.log('Current NIC: ', nic.name);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we remove all the unnecesary logs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

'Imports VM from VMware Instance, basic test',
async () => {
const vm = await wizard.import(vmwareVMConfig);
await withResource(leakedResources, vm.asResource(), async () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose we should register the leaked vm before we start the import. + in other cases too

await wizard.importVmConnectProviderStep(vmwareVMConfig);
await asyncForEach(wrongValues, async (curValue) => {
const err = await wizard.fillName(curValue);
expect(err).toBeDefined();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the same, do we have the err?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found the reason, some my code was removed when rebasing from master, now everything is working fine.

import { DiskDialog } from '../../tests/dialogs/diskDialog';
import { KEBAP_ACTION } from '../../tests/utils/constants/common';

describe('VMWare Wizard validation, negative tests', () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am getting this error in all of these:


3) VMWare Wizard validation, negative tests : VMWare - Import Wizard shows warning when using incorrect VM disk name
   TypeError: each key must be a number of string; got undefined
       at /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/webdriver.js:2162:21
       at Array.forEach <anonymous>
       at /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/webdriver.js:2157:16
       at runMicrotasks <anonymous>
       at processTicksAndRejections internal/process/task_queues.js:93:5
   From: Task: WebElement.sendKeys()
       at Driver.schedule /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/webdriver.js:807:17
       at WebElement.schedule_ /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/webdriver.js:2010:25
       at WebElement.sendKeys /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/webdriver.js:2174:19
       at actionFn /home/ansy/projects/openshift/console/frontend/node_modules/protractor/built/element.js:89:44
       at Array.map <anonymous>
       at /home/ansy/projects/openshift/console/frontend/node_modules/protractor/built/element.js:461:65
       at ManagedPromise.invokeCallback_ /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:1376:14
       at TaskQueue.execute_ /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:3084:14
       at TaskQueue.executeNext_ /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:3067:27
       at /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:2927:27Error
       at ElementArrayFinder.applyAction_ /home/ansy/projects/openshift/console/frontend/node_modules/protractor/built/element.js:459:27
       at ElementArrayFinder.<computed> [as sendKeys] /home/ansy/projects/openshift/console/frontend/node_modules/protractor/built/element.js:91:29
       at ElementFinder.<computed> [as sendKeys] /home/ansy/projects/openshift/console/frontend/node_modules/protractor/built/element.js:831:22
       at Object.<anonymous> /home/ansy/projects/openshift/console/frontend/packages/console-shared/src/test-utils/utils.ts:106:16
       at Generator.next <anonymous>
       at fulfilled /home/ansy/projects/openshift/console/frontend/packages/console-shared/src/test-utils/utils.ts:5:58
       at runMicrotasks <anonymous>
       at processTicksAndRejections internal/process/task_queues.js:93:5
   From: Task: Run it"VMWare - Import Wizard shows warning when using incorrect VM disk name" in control flow
       at UserContext.<anonymous> /home/ansy/projects/openshift/console/frontend/node_modules/jasminewd2/index.js:94:19
       at /home/ansy/projects/openshift/console/frontend/node_modules/jasminewd2/index.js:64:48
       at ControlFlow.emit /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/events.js:62:21
       at ControlFlow.shutdown_ /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:2674:10
       at /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:2599:53
   From asynchronous test: 
   Error
       at Suite.<anonymous> /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/v2v/vmware/v2v.vmware.import.negative.scenario.incorrect.names.ts:54:3
       at Object.<anonymous> /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/v2v/vmware/v2v.vmware.import.negative.scenario.incorrect.names.ts:13:1
       at Module._compile internal/modules/cjs/loader.js:1063:30
       at Module.m._compile /home/ansy/projects/openshift/console/frontend/node_modules/ts-node/src/index.ts:400:23
       at Module._extensions..js internal/modules/cjs/loader.js:1092:10
       at Object.require.extensions.<computed> [as .ts] /home/ansy/projects/openshift/console/frontend/node_modules/ts-node/src/index.ts:403:12

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found the reason, some my code was removed when rebasing from master, now everything is working fine.

@@ -0,0 +1,76 @@
import { asyncForEach, click } from '@console/shared/src/test-utils/utils';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we adhere to the naming convention in all of these? Meaning that they should end with scenario.ts name

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

import { VM_STATUS } from '../../tests/utils/constants/vm';
import { V2V_VM_IMPORT_TIMEOUT } from '../../tests/utils/constants/common';

describe('Kubevirt create VM using wizard', () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it is not possible to run these test for me as I am getting this error:

will debug more


>> An error was thrown in an afterAll
   UnexpectedAlertOpenError: unexpected alert open: {Alert text : Are you sure you want to navigate away from this form? Any data you've added will be lost.}
     (Session info: chrome=84.0.4147.125)
     (Driver info: chromedriver=84.0.4147.30 (48b3e868b4cc0aa7e8149519690b6f6949e110a8-refs/branch-heads/4147@{#310}),platform=Linux 5.8.14-arch1-1 x86_64)
       at Object.checkLegacyResponse /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/error.js:553:13
       at parseHttpResponse /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/http.js:509:13
       at /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/http.js:441:30
       at processTicksAndRejections internal/process/task_queues.js:93:5
   From: Task: WebDriver.takeScreenshot()
       at Driver.schedule /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/webdriver.js:807:17
       at Driver.takeScreenshot /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/webdriver.js:1085:17
       at run /home/ansy/projects/openshift/console/frontend/node_modules/protractor/built/browser.js:59:33
       at ProtractorBrowser.to.<computed> [as takeScreenshot] /home/ansy/projects/openshift/console/frontend/node_modules/protractor/built/browser.js:67:16
       at /home/ansy/projects/openshift/console/frontend/node_modules/protractor-jasmine2-screenshot-reporter/index.js:600:23
       at /home/ansy/projects/openshift/console/frontend/node_modules/protractor-jasmine2-screenshot-reporter/node_modules/lodash/lodash.js:4911:15
       at baseForOwn /home/ansy/projects/openshift/console/frontend/node_modules/protractor-jasmine2-screenshot-reporter/node_modules/lodash/lodash.js:2996:24
       at /home/ansy/projects/openshift/console/frontend/node_modules/protractor-jasmine2-screenshot-reporter/node_modules/lodash/lodash.js:4880:18
       at Function.forEach /home/ansy/projects/openshift/console/frontend/node_modules/protractor-jasmine2-screenshot-reporter/node_modules/lodash/lodash.js:9344:14
       at Jasmine2ScreenShotReporter.specDone /home/ansy/projects/openshift/console/frontend/node_modules/protractor-jasmine2-screenshot-reporter/index.js:596:7

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found the reason, some my code was removed when rebasing from master, now everything is working fine.

import { browser, ExpectedConditions as until, $ } from 'protractor';
// import * as wizardView from "../../views/importWizard.view";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra comment

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

it(
'Importing VM from RHV Instance with multi NIC and multi disk config',
async () => {
createResources([multusNAD]);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should use withResource syntax to prevent leaking the NAD resource

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as we spoke, resources are leaked only if test fails with critical exception

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so, that is still an issue then

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the only thing I can imaging here is to catch such exceptions and through error messages to prevent those failures

describe('Kubevirt imports VM from RHV using wizard', () => {
const leakedResources = new Set<string>();
const wizard = new RhvImportWizard();

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please go all over the resources and check they are not leaked, as I mentioned in the last PR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as we spoke, resources are leaked only if test fails with critical exception

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so, that is still an issue then

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the only thing I can imaging here is to catch such exceptions and through error messages to prevent those failures

await diskDialog.selectStorageClass(disk.storageClass);
await click(saveButton);
await waitForNoLoaders();
if (disk.name === 'cloudinitdisk') {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed this, not relevant any more

);

it(
'Importing VM from RHV Instance with reuse of existing one',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this test fails. You have to skip the network/storage/advanced step when advanced edit is not activated

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

);

it(
'Importing VM from RHV Instance with multi NIC and multi disk config',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this one is failing on not skipping the checking of the wizard errors

Failed Specs:

1) Kubevirt imports VM from RHV using wizard : Importing VM from RHV Instance with multi NIC and multi disk config
   Error: The following fields must be completed before importing this virtual machine
   Networking
   nic1
   Network
   Networking
   nic2
   Network
       at RhvImportWizard.<anonymous> /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/tests/models/wizard.ts:84:13
       at Generator.next <anonymous>
       at fulfilled /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/tests/models/wizard.ts:5:58
       at runMicrotasks <anonymous>
       at processTicksAndRejections internal/process/task_queues.js:93:5
   From: Task: Run it"Importing VM from RHV Instance with multi NIC and multi disk config" in control flow
       at UserContext.<anonymous> /home/ansy/projects/openshift/console/frontend/node_modules/jasminewd2/index.js:94:19
   From asynchronous test: 
   Error
       at Suite.<anonymous> /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/v2v/rhv/v2v.rhv.basic.import.scenario.ts:42:3
       at Object.<anonymous> /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/v2v/rhv/v2v.rhv.basic.import.scenario.ts:16:1
       at Module._compile internal/modules/cjs/loader.js:1063:30
       at Module.m._compile /home/ansy/projects/openshift/console/frontend/node_modules/ts-node/src/index.ts:400:23
       at Module._extensions..js internal/modules/cjs/loader.js:1092:10
       at Object.require.extensions.<computed> [as .ts] /home/ansy/projects/openshift/console/frontend/node_modules/ts-node/src/index.ts:403:12

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is fixed, problem was caused by using this.next to click next button. Inside this method there is validation of UI error messages and test fails if any. Clicked next directly and test passed normally.

await wizard.closeWizard();
});

it('VMWare - Import Wizard shows warning when using incorrect VM name', async () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am getting this for the vmware imports:


1) VMWare Wizard validation, negative tests : VMWare - Import Wizard shows warning when using incorrect VM name
   Error: Exceeded number of attempts
       at VmwareImportWizard.<anonymous> /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/tests/models/importWizard.ts:203:17
       at Generator.next <anonymous>
       at fulfilled /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/tests/models/importWizard.ts:5:58
   From: Task: Run it"VMWare - Import Wizard shows warning when using incorrect VM name" in control flow
       at UserContext.<anonymous> /home/ansy/projects/openshift/console/frontend/node_modules/jasminewd2/index.js:94:19
       at /home/ansy/projects/openshift/console/frontend/node_modules/jasminewd2/index.js:64:48
       at ControlFlow.emit /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/events.js:62:21
       at ControlFlow.shutdown_ /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:2674:10
       at /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:2599:53
   From asynchronous test: 
   Error
       at Suite.<anonymous> /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/v2v/vmware/v2v.vmware.import.incorrect.names.negative.scenario.ts:24:3
       at Object.<anonymous> /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/v2v/vmware/v2v.vmware.import.incorrect.names.negative.scenario.ts:13:1
       at Module._compile internal/modules/cjs/loader.js:1063:30
       at Module.m._compile /home/ansy/projects/openshift/console/frontend/node_modules/ts-node/src/index.ts:400:23
       at Module._extensions..js internal/modules/cjs/loader.js:1092:10
       at Object.require.extensions.<computed> [as .ts] /home/ansy/projects/openshift/console/frontend/node_modules/ts-node/src/index.ts:403:12

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is floating problem, it is reproducing on different tests from time to time, I have no idea how to reproduce it and what causes it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Found the reason, when CNV tried to connect to VMWare, it tries some amount of times and then fail with this exact issue. This error is not a problem of my tests, this problem is caused by connectivity issue between CNV and VMWare

const err = await nicDialog.fillName(curValue);
expect(err).toBeDefined();
});
await browser.sleep(10000);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are all of these sleeps necessary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did that for debug purpose, will remove them

Copy link
Member

@atiratree atiratree left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The RHV import works great for me.
But the vmware import still does not pass...

  • also please remove the extra commit from the git


async edit(config) {
const { advancedEdit } = config;
if (advancedEdit) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this fails for me here - because there is one another condition here - a presence and evaluation of vmware-to-kubevirt-os configMap

IMO we should check if there are errors on top of the page are present if we wish to use advancedEdit: otherwise I get this exception:

1) VMWare Wizard validation, negative tests : VMWare - Import Wizard shows warning when using incorrect VM name
   Error: Timeout - Async callback was not invoked within timeout specified by jasmine.DEFAULT_TIMEOUT_INTERVAL.
       at listOnTimeout internal/timers.js:554:17
       at processTimers internal/timers.js:497:7
   
   UnexpectedAlertOpenError: unexpected alert open: {Alert text : Are you sure you want to navigate away from this form? Any data you've added will be lost.}
     (Session info: chrome=84.0.4147.125)
     (Driver info: chromedriver=84.0.4147.30 (48b3e868b4cc0aa7e8149519690b6f6949e110a8-refs/branch-heads/4147@{#310}),platform=Linux 5.8.14-arch1-1 x86_64)
       at Object.checkLegacyResponse /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/error.js:553:13
       at parseHttpResponse /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/http.js:509:13
       at /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/http.js:441:30
       at runMicrotasks <anonymous>:null:null
       at processTicksAndRejections internal/process/task_queues.js:93:5
   
   From: Task: WebDriver.findElements(Bycss selector, #vm-name)
       at Driver.schedule /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/webdriver.js:807:17
       at Driver.findElements /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/webdriver.js:1048:19
       at /home/ansy/projects/openshift/console/frontend/node_modules/protractor/built/element.js:159:44
       at ManagedPromise.invokeCallback_ /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:1376:14
       at TaskQueue.execute_ /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:3084:14
       at TaskQueue.executeNext_ /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:3067:27
       at /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:2927:27
       at /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:668:7
   
   From: Task: <anonymous>
       at Timeout.pollCondition [as _onTimeout] /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:2195:19
       at listOnTimeout internal/timers.js:554:17
       at processTimers internal/timers.js:497:7
   
   From: Task: <anonymous wait>
       at scheduleWait /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:2188:20
       at ControlFlow.wait /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:2517:12
       at Driver.wait /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/webdriver.js:934:29
       at run /home/ansy/projects/openshift/console/frontend/node_modules/protractor/built/browser.js:59:33
       at ProtractorBrowser.to.<computed> [as wait] /home/ansy/projects/openshift/console/frontend/node_modules/protractor/built/browser.js:67:16
       at Object.<anonymous> /home/ansy/projects/openshift/console/frontend/packages/console-shared/src/test-utils/utils.ts:106:40
       at Generator.next <anonymous>:null:null
       at /home/ansy/projects/openshift/console/frontend/packages/console-shared/src/test-utils/utils.ts:8:71
       at new Promise <anonymous>:null:null
       at __awaiter /home/ansy/projects/openshift/console/frontend/packages/console-shared/src/test-utils/utils.ts:4:12
       at Object.fillInput /home/ansy/projects/openshift/console/frontend/packages/console-shared/src/test-utils/utils.ts:98:12
       at VmwareImportWizard.<anonymous> /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/tests/models/wizard.ts:81:27
       at Generator.next <anonymous>:null:null
       at /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/tests/models/wizard.ts:8:71
       at new Promise <anonymous>:null:null
       at __awaiter /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/tests/models/wizard.ts:4:12
       at VmwareImportWizard.fillName /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/tests/models/wizard.ts:80:16
       at /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/v2v/vmware/v2v.vmware.import.incorrect.names.negative.scenario.ts:33:38
       at Generator.next <anonymous>:null:null
       at /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/v2v/vmware/v2v.vmware.import.incorrect.names.negative.scenario.ts:8:71
       at new Promise <anonymous>:null:null
       at __awaiter /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/v2v/vmware/v2v.vmware.import.incorrect.names.negative.scenario.ts:4:12
       at /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/v2v/vmware/v2v.vmware.import.incorrect.names.negative.scenario.ts:32:68
       at Object.<anonymous> /home/ansy/projects/openshift/console/frontend/packages/console-shared/src/test-utils/utils.ts:159:19
       at Generator.next <anonymous>:null:null
       at /home/ansy/projects/openshift/console/frontend/packages/console-shared/src/test-utils/utils.ts:8:71
       at new Promise <anonymous>:null:null
       at __awaiter /home/ansy/projects/openshift/console/frontend/packages/console-shared/src/test-utils/utils.ts:4:12
       at Object.asyncForEach /home/ansy/projects/openshift/console/frontend/packages/console-shared/src/test-utils/utils.ts:156:12
       at /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/v2v/vmware/v2v.vmware.import.incorrect.names.negative.scenario.ts:32:23
       at Generator.next <anonymous>:null:null
       at fulfilled /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/v2v/vmware/v2v.vmware.import.incorrect.names.negative.scenario.ts:5:58
       at processTicksAndRejections internal/process/task_queues.js:93:5
   
   From: Task: Run it"VMWare - Import Wizard shows warning when using incorrect VM name" in control flow
       at UserContext.<anonymous> /home/ansy/projects/openshift/console/frontend/node_modules/jasminewd2/index.js:94:19
       at /home/ansy/projects/openshift/console/frontend/node_modules/jasminewd2/index.js:64:48
       at ControlFlow.emit /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/events.js:62:21
       at ControlFlow.shutdown_ /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:2674:10
       at /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:2599:53
       at /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:2728:9
       at /home/ansy/projects/openshift/console/frontend/node_modules/selenium-webdriver/lib/promise.js:668:7
       at processTicksAndRejections internal/process/task_queues.js:93:5
   
   From asynchronous test: 
   Error
       at Suite.<anonymous> /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/v2v/vmware/v2v.vmware.import.incorrect.names.negative.scenario.ts:30:5
       at Env.<anonymous> /home/ansy/projects/openshift/console/frontend/node_modules/jasmine-fail-fast/dist/jasmine-fail-fast.js:57:26
       at Env.wrapper [as describe] /home/ansy/projects/openshift/console/frontend/node_modules/jasmine-fail-fast/node_modules/lodash/index.js:3592:19
       at Object.<anonymous> /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/v2v/vmware/v2v.vmware.import.incorrect.names.negative.scenario.ts:22:1
       at Module._compile internal/modules/cjs/loader.js:1063:30
       at Module.m._compile /home/ansy/projects/openshift/console/frontend/node_modules/ts-node/dist/index.js:234:29
       at Module._extensions..js internal/modules/cjs/loader.js:1092:10
       at Object.require.extensions.<computed> [as .ts] /home/ansy/projects/openshift/console/frontend/node_modules/ts-node/dist/index.js:236:16
       at Module.load /home/ansy/projects/openshift/console/frontend/node_modules/coffeescript/lib/coffee-script/register.js:45:36
       at Function.Module._load internal/modules/cjs/loader.js:769:14
       at Module.require internal/modules/cjs/loader.js:952:19
       at require internal/modules/cjs/helpers.js:88:18
       at /home/ansy/projects/openshift/console/frontend/node_modules/jasmine/lib/jasmine.js:93:5
       at Array.forEach <anonymous>:null:null
       at Jasmine.loadSpecs /home/ansy/projects/openshift/console/frontend/node_modules/jasmine/lib/jasmine.js:92:18
       at Jasmine.execute /home/ansy/projects/openshift/console/frontend/node_modules/jasmine/lib/jasmine.js:197:8
       at /home/ansy/projects/openshift/console/frontend/node_modules/protractor/built/frameworks/jasmine.js:132:15
       at Function.promise /home/ansy/projects/openshift/console/frontend/node_modules/q/q.js:682:9
       at /home/ansy/projects/openshift/console/frontend/node_modules/protractor/built/frameworks/jasmine.js:104:14
       at _fulfilled /home/ansy/projects/openshift/console/frontend/node_modules/q/q.js:834:54
       at /home/ansy/projects/openshift/console/frontend/node_modules/q/q.js:863:30
       at Promise.promise.promiseDispatch /home/ansy/projects/openshift/console/frontend/node_modules/q/q.js:796:13
       at /home/ansy/projects/openshift/console/frontend/node_modules/q/q.js:556:49
       at runSingle /home/ansy/projects/openshift/console/frontend/node_modules/q/q.js:137:13
       at flush /home/ansy/projects/openshift/console/frontend/node_modules/q/q.js:125:13
       at processTicksAndRejections internal/process/task_queues.js:75:11
   

await this.selectProvider(provider);
await this.waitForSpinner();
await this.configureInstance(instanceConfig);

await this.connectToInstance();
// await this.connectToInstance();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comment?

'Imports VM from VMware Instance, basic test',
async () => {
const vm = await wizard.import(vmwareVMConfig);
await withResource(leakedResources, vm.asResource(), async () => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tried to run this on 2 envs and it always fails for me at the end of test with:

1) Kubevirt migrate VM from VMWare using wizard : Imports VM from VMware Instance, basic test
   Error: Failed expectation
       at VmwareImportWizard.<anonymous> /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/tests/models/wizard.ts:219:73
       at Generator.next <anonymous>:null:null
       at fulfilled /home/ansy/projects/openshift/console/frontend/packages/kubevirt-plugin/integration-tests/tests/models/wizard.ts:5:58
       at runMicrotasks <anonymous>:null:null
       at processTicksAndRejections internal/process/task_queues.js:93:5

Does this happen to you as well? I can give you an env where it is reproducible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never seen this before, I will try to re-run again

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am still seeing this on Igor's env, and since he is not seeing it - there might be something wrong with my setup. Since other things are working fine, we should not block this PR on this and resolve this one once it can be reproduced.

@atiratree
Copy link
Member

looks good @ibragins thanks for your work

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Nov 9, 2020
@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ibragins, suomiy

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 9, 2020
@ibragins ibragins changed the title Added negative tests with wrong names for VM, NIC and disk both for R… Bug 1896101: Added negative tests for migration from VMWare and RHV Nov 9, 2020
@openshift-ci-robot openshift-ci-robot added bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. labels Nov 9, 2020
@openshift-ci-robot
Copy link
Contributor

@ibragins: This pull request references Bugzilla bug 1896101, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.7.0) matches configured target release for branch (4.7.0)
  • bug is in the state NEW, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

Bug 1896101: Added negative tests for migration from VMWare and RHV

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ibragins
Copy link
Contributor Author

ibragins commented Nov 9, 2020

looks good @ibragins thanks for your work

/lgtm

@suomiy Thank you for YOUR hard work and helping

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

1 similar comment
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit bb51ca6 into openshift:master Nov 10, 2020
@openshift-ci-robot
Copy link
Contributor

@ibragins: All pull requests linked via external trackers have merged:

Bugzilla bug 1896101 has been moved to the MODIFIED state.

In response to this:

Bug 1896101: Added negative tests for migration from VMWare and RHV

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

export const RHV_PROVIDER = 'Red Hat Virtualization (RHV)';
export const VMWARE_PROVIDER = 'VMware';

// Web-UI Exceptions
export const RHV_WIZARD_CREATE_SUCCESS = 'Started import of virtual machine';
export const WIZARD_CREATE_SUCCESS = 'Successfully created virtual machine';
export const WIZARD_CREATE_SUCCESS = 'Started import of virtual machine';
// export const WIZARD_CREATE_SUCCESS = 'Successfully created virtual machine';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This breaks tests which has VM created via wizard, fix it in #7169.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this one slipped through, thanks for fixing it

@spadgett spadgett added this to the v4.7 milestone Nov 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. component/ceph Related to ceph-storage-plugin component/core Related to console core functionality component/dashboard Related to dashboard component/dev-console Related to dev-console component/knative Related to knative-plugin component/kubevirt Related to kubevirt-plugin component/metal3 Related to metal3-plugin component/noobaa Related to noobaa-storage-plugin component/olm Related to OLM component/shared Related to console-shared lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

10 participants