Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run acceptance tests on firefox #1213

Closed
wants to merge 1 commit into from
Closed

Run acceptance tests on firefox #1213

wants to merge 1 commit into from

Conversation

paurakhsharma
Copy link
Member

@paurakhsharma paurakhsharma commented Jun 3, 2019 •
edited

Description

Run acceptance tests on firefox

moveToElement() is removed because it causes 'move target out of bounds' on firefox.
According to documentation when running click() method

The element is scrolled into view if it is not already pointer-interactable.

So, the tests should still function as usual.

Related Issue

Fixes #1197

Motivation and Context

Make the tests more robust.

How Has This Been Tested?

馃

Screenshots (if appropriate):

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Technical debt
  • Tests

Checklist:

  • Code changes
  • Unit tests added
  • Acceptance tests added
  • Documentation ticket raised:

@paurakhsharma paurakhsharma self-assigned this Jun 3, 2019
@paurakhsharma paurakhsharma force-pushed the run-on-firefox branch 4 times, most recently from eabb7c0 to 6e8ede9 Compare June 3, 2019 10:19
@ownclouders
Copy link
Contributor

馃挜 Acceptance tests webUISharingInternalUsers failed. Please find the screenshots inside ...

20190603-102602-844.png
20190603-102603-031.png

@paurakhsharma paurakhsharma added Status:Needs-Review Needs review from a maintainer and removed Status:In-Progress labels Jun 3, 2019
@paurakhsharma paurakhsharma requested review from DeepDiver1975 and individual-it and removed request for DeepDiver1975 June 3, 2019 11:05
@paurakhsharma paurakhsharma marked this pull request as ready for review June 3, 2019 11:06
@individual-it
Copy link
Member

FF is crashing

17:02:31.196 INFO [ProtocolHandshake.createSession] - Detected dialect: W3C
17:02:31.226 INFO [RemoteSession$Factory.lambda$performHandshake$0] - Started new session 7d22c1f5-6060-4604-9d48-5fec24509fd1 (org.openqa.selenium.firefox.GeckoDriverService)
JavaScript error: http://phoenix:8300/#/login, line 36: ReferenceError: wgxpath is not defined
JavaScript error: http://phoenix:8300/index.html#/oidc-callback?access_token=Y3DzV6sfH0UNb6o4cSqSJT720sYOUdOerVeR1z4c8QPxW7yxKaxaeJZxhe9I9aPx&expires_in=3600&state=adfc56985b884229a860d033c520ccfc, line 36: ReferenceError: wgxpath is not defined
JavaScript error: http://phoenix:8300/, line 36: ReferenceError: wgxpath is not defined
[Parent 252, Gecko_IOThread] WARNING: pipe error (98): Connection reset by peer: file /builds/worker/workspace/build/src/ipc/chromium/src/chrome/common/ipc_channel_posix.cc, line 357

###!!! [Parent][MessageChannel] Error: (msgtype=0x1F0088,name=PBrowser::Msg_Destroy) Channel error: cannot send/recv

A content process crashed and MOZ_CRASHREPORTER_SHUTDOWN is set, shutting down
1559581362760 Marionette INFO Stopped listening on port 34652

will restart the build to see how often it happens

individual-it
individual-it reviewed Jun 4, 2019
View reviewed changes
tests/acceptance/pageObjects/FilesPageElement/sharingDialog.js
if (value.ELEMENT) {
webElementIdList.push(value.ELEMENT)
} else {
webElementIdList.push(value['element-6066-11e4-a52e-4f735466cecf'])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is that magic number always the same?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you can read more about it here: https://www.w3.org/TR/webdriver/#elements

@ownclouders
Copy link
Contributor

馃挜 Acceptance tests webUISharingInternalUsers failed. Please find the screenshots inside ...

20190604-031019-663.png

@ownclouders
Copy link
Contributor

馃挜 Acceptance tests webUISharingInternalUsers failed. Please find the screenshots inside ...

20190604-065656-301.png
20190604-065656-513.png
20190604-065727-221.png
20190604-065727-425.png

@paurakhsharma paurakhsharma force-pushed the run-on-firefox branch 2 times, most recently from 45e274e to 0020c64 Compare June 4, 2019 08:09
@paurakhsharma paurakhsharma changed the title Run acceptance tests on firefox [WIP] Run acceptance tests on firefox Jun 4, 2019
@paurakhsharma paurakhsharma force-pushed the run-on-firefox branch 3 times, most recently from f1348ca to 3b5b85d Compare June 4, 2019 09:52
@ownclouders
Copy link
Contributor

馃挜 Acceptance tests webUIRenameFiles failed. Please find the screenshots inside ...

20190604-095844-618.png
20190604-095844-733.png

@paurakhsharma paurakhsharma force-pushed the run-on-firefox branch 2 times, most recently from 32a7d90 to 3fc782c Compare June 4, 2019 10:55
@paurakhsharma
Copy link
Member Author

paurakhsharma commented Jun 4, 2019 •
edited

There is an error in some suites with an error message

{ state: 'unknown error',
     sessionId: null,
     value: 
      { systemInformation: 'System info: host: \'606a6335e49d\', ip: \'172.17.0.10\', os.name: \'Linux\', os.arch: \'amd64\', os.version: \'4.4.0-148-generic\', java.version: \'1.8.0_212\'',
        message: 'Failed to decode response from marionette',
        error: 
         [ 'Build info: version: \'3.141.59\', revision: \'e82be7d358\', time: \'2018-11-14T08:25:53\'',
           'System info: host: \'606a6335e49d\', ip: \'172.17.0.10\', os.name: \'Linux\', os.arch: \'amd64\', os.version: \'4.4.0-148-generic\', java.version: \'1.8.0_212\'',
           'Driver info: driver.version: unknown' ] },
     status: 13 }

Which might be caused due to memory problem in drone,
according to this issue. mozilla/geckodriver#1193 (comment)

Edit: Usually this occurs in the same suites, and other suites seem to pass without a problem
So, this could be caused by the other problem as well.

@ownclouders
Copy link
Contributor

馃挜 Acceptance tests webUISharingInternalUsers failed. Please find the screenshots inside ...

20190611-085038-749.png
20190611-085038-936.png
20190611-085203-190.png
20190611-085203-387.png

@paurakhsharma paurakhsharma force-pushed the run-on-firefox branch 5 times, most recently from 903c908 to 9ec0e8b Compare June 12, 2019 08:43
@ownclouders
Copy link
Contributor

馃挜 Acceptance tests webUISharingInternalUsers failed. Please find the screenshots inside ...

20190612-084924-811.png
20190612-084925-002.png
20190612-085049-209.png
20190612-085049-382.png

@ownclouders
Copy link
Contributor

馃挜 Acceptance tests webUISharingInternalUsers failed. Please find the screenshots inside ...

20190613-060519-479.png
20190613-060519-644.png
20190613-060557-418.png
20190613-060557-580.png

@ownclouders
Copy link
Contributor

馃挜 Acceptance tests webUISharingInternalUsers failed. Please find the screenshots inside ...

20190613-075010-538.png
20190613-075010-701.png
20190613-075036-922.png
20190613-075037-115.png

@paurakhsharma paurakhsharma force-pushed the run-on-firefox branch 3 times, most recently from e0e6f38 to 3d28ac4 Compare June 13, 2019 11:13
@ownclouders
Copy link
Contributor

馃挜 Acceptance tests webUIFiles failed. Please find the screenshots inside ...

20190613-113449-512.png
20190613-113521-831.png
20190613-113618-354.png
20190613-113649-638.png

@paurakhsharma
Copy link
Member Author

paurakhsharma commented Jun 14, 2019 •
edited

Acceptance tests webUIFiles failed. Please find the screenshots inside ...
20190613-113449-512.png

Another problem while running tests in firefox is that the user gets logged out when running
the step When the user reloads the current page of the webUI this happens locally too.
c.c @individual-it

In case of Scenario: Search for a shared file
When the user reloads the current page of the webUI is not the only cause of log out here,
If we remove And the user has browsed to the files page from background, the tests seem to run
just fine.

When the user receives a share and then receiver searches for the file the file seems to be already available to the user. Which means we can remove When the user reloads the current page of the webUI altogether in case of searching a newly received share.

@paurakhsharma paurakhsharma force-pushed the run-on-firefox branch 2 times, most recently from d772947 to cb014b0 Compare June 14, 2019 06:41
@paurakhsharma paurakhsharma changed the title [WIP] Run acceptance tests on firefox Run acceptance tests on firefox Jun 14, 2019
@ownclouders
Copy link
Contributor

馃挜 Acceptance tests webUIFiles failed. Please find the screenshots inside ...

20190614-064914-825.png
20190614-065159-424.png
20190614-065624-892.png
20190614-065717-141.png
20190614-065801-971.png
20190614-065802-652.png
20190614-065803-430.png
20190614-070119-124.png
20190614-070119-502.png
20190614-070119-782.png
20190614-070134-454.png
20190614-070158-422.png

individual-it
individual-it requested changes Jun 14, 2019
View reviewed changes
Copy link
Member

@individual-it individual-it left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all in all this looks a bit haky to me and might make the tests more flaky in the future. Should we maybe just archive this and try again in a couple of month?

tests/acceptance/features/webUIFiles/createFolderEdgeCases.feature
| "^#29][29@({" |
| "+-{$(882)" |
| "home" |
# Scenario Outline: Create a folder using special characters
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why commenting that whole test out?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a mistake, that should have@skipOnFIREFOX

.drone.yml
BROWSER: chrome

firefox:
image: selenium/standalone-firefox-debug:3.141.59-mercury
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not :latest?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I used for testing purpose. I should keep the latest one there.

@paurakhsharma
Copy link
Member Author

all in all this looks a bit haky to me and might make the tests more flaky in the future. Should we maybe just archive this and try again in a couple of month?

Yeah its haky, we have skipped many scenarios that are failing on firefox without knowing the actual
cause. Yes, closing this PR for now and then reopening again later when the geckodriver is more stable makes sense to me.

@paurakhsharma
Copy link
Member Author

Closing this for now. We can retry after mozilla/geckodriver#285 is fixed.

@pascalwengerter pascalwengerter deleted the run-on-firefox branch April 8, 2021 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@individual-it individual-it individual-it requested changes

@DeepDiver1975 DeepDiver1975 Awaiting requested review from DeepDiver1975

Assignees

@paurakhsharma paurakhsharma

Labels
QA:team Status:Needs-Review Needs review from a maintainer
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

run UI tests in Firefox
3 participants
@paurakhsharma @ownclouders @individual-it