Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Tests Failing when using torture .We tried on our self hosted server and then tried on jitsi meet (https://meet.jit.si/) and getting the same errors as below #387

Open
rvvssvraju opened this issue Jun 1, 2021 · 5 comments
Open

Tests Failing when using torture .We tried on our self hosted server and then tried on jitsi meet (https://meet.jit.si/) and getting the same errors as below #387

rvvssvraju opened this issue Jun 1, 2021 · 5 comments

Comments

@rvvssvraju
Copy link 

[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   ActiveSpeakerTest>AbstractBaseTest.setupClassPrivate:188->setupClass:45->WebTestBase.ensureThreeParticipants:199->WebTestBase.ensureThreeParticipants:180->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   AudioOnlyTest>AbstractBaseTest.setupClassPrivate:188->setupClass:44->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   AvatarTest>AbstractBaseTest.setupClassPrivate:188->setupClass:51->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   ChatPanelTest>AbstractBaseTest.setupClassPrivate:188->setupClass:43->WebTestBase.ensureOneParticipant:56->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   DesktopSharingTest.testDesktopSharingStart:47->WebTestBase.ensureOneParticipant:56->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   DisplayNameTest>AbstractBaseTest.setupClassPrivate:188->setupClass:49->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   EndConferenceTest>AbstractBaseTest.setupClassPrivate:188->setupClass:36->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   EtherpadTest>AbstractBaseTest.setupClassPrivate:188->setupClass:49->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   FollowMeTest>AbstractBaseTest.setupClassPrivate:188->setupClass:48->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   InviteTest>AbstractBaseTest.setupClassPrivate:188->setupClass:47->WebTestBase.ensureOneParticipant:56->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   KickTest>AbstractBaseTest.setupClassPrivate:188->setupClass:41->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   LastNTest.testLastN:48->WebTestBase.ensureOneParticipant:68->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   LipSyncTest.testLipSync:114->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   LobbyTest>AbstractBaseTest.setupClassPrivate:188->setupClass:45->WebTestBase.ensureOneParticipant:56->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   LockRoomDigitsOnlyTest.lockRoomWithDigitsOnly:44->WebTestBase.ensureOneParticipant:68->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   LockRoomTest>AbstractBaseTest.setupClassPrivate:188->setupClass:50->WebTestBase.ensureOneParticipant:56->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   ModeratorTest>AbstractBaseTest.setupClassPrivate:188->setupClass:49->WebTestBase.ensureOneParticipant:56->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   MuteTest>AbstractBaseTest.setupClassPrivate:188->setupClass:58->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   OneOnOneTest.testFilmstripHiddenInOneOnOne:58->WebTestBase.ensureTwoParticipants:104->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   Peer2PeerTest.testSwitchToP2P:49->WebTestBase.ensureTwoParticipants:104->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   PreJoinTest.testJoinWithoutAudio:84->WebTestBase.joinFirstParticipant:347->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   PreJoinTest.testPreJoinForRoomWithoutLobby:65->WebTestBase.joinFirstParticipant:347->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   PreJoinTest.testPreJoinWhenDisplayedNameRequired:40->WebTestBase.joinFirstParticipant:347->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   SharedVideoTest>AbstractBaseTest.setupClassPrivate:188->setupClass:72->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   SinglePortTest>AbstractBaseTest.setupClassPrivate:188->setupClass:38->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   StartMutedTest>AbstractBaseTest.setupClassPrivate:188->setupClass:39->WebTestBase.ensureOneParticipant:56->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   StopVideoTest>AbstractBaseTest.setupClassPrivate:188->setupClass:57->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   SwitchVideoTest>AbstractBaseTest.setupClassPrivate:188->setupClass:57->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   TileViewTest>AbstractBaseTest.setupClassPrivate:188->setupClass:54->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   UDPTest>AbstractBaseTest.setupClassPrivate:188->setupClass:37->WebTestBase.ensureTwoParticipants:91->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:151->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[ERROR]   VideoLayoutTest>AbstractBaseTest.setupClassPrivate:188->setupClass:44->WebTestBase.ensureOneParticipant:56->WebTestBase.ensureOneParticipant:82->WebTestBase.joinParticipantAndWait:423->WebTestBase.joinParticipant:324 » InvalidArgument
[INFO] 
[ERROR] Tests run: 290, Failures: 31, Errors: 0, Skipped: 256
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  45.882 s
[INFO] Finished at: 2021-06-01T19:30:15+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.20:test (default-test) on project jitsi-meet-torture: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/vignesh/jitsi-meet-torture/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
@dkgrieshammer
Copy link

Same here; any updates?

@damencho
Copy link
Member

How are you running the test?

@dkgrieshammer
Copy link

How are you running the test?

@damencho I just cloned the repo & run MuteTest & TCPTest, fails against own install as well as on meet.jit.si;
mvn test -Djitsi-meet.instance.url="https://meet.jit.si" -Djitsi-meet.tests.toRun="MuteTest,TCPTest"

I did that to narrow down the errors I got using the docker-hub version on local macbook as well as on an AWS Debian;
any hints would be highly appreciated 🙏

Error Log:

[1] End tcpTest.
[1] Closing web.participant1
[1] Closing web.participant2
[ERROR] Tests run: 415, Failures: 2, Errors: 0, Skipped: 406, Time elapsed: 88.967 s <<< FAILURE! - in TestSuite
[ERROR] muteAfterJoinCanShareAndUnmute(org.jitsi.meet.test.MuteTest)  Time elapsed: 25.645 s  <<< FAILURE!
org.openqa.selenium.TimeoutException:
Expected condition failed: waiting for org.jitsi.meet.test.util.TestUtils$$Lambda$344/0x0000000800e89390@7dd611c8 (tried for 10 second(s) with 500 milliseconds interval)
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03'
System info: host: 'David-Air.local', ip: '**IP**', os.name: 'Mac OS X', os.arch: 'aarch64', os.version: '12.0.1', java.version: '17.0.1'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 96.0.4664.55, chrome: {chromedriverVersion: 96.0.4664.45 (76e4c1bb2ab46..., userDataDir: /var/folders/6g/4hb15fqd6x7...}, goog:chromeOptions: {debuggerAddress: localhost:54971}, javascriptEnabled: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: MAC, platformName: MAC, proxy: Proxy(), setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}
Session ID: b1cc9df88ce281bcdce8f04a90ea983a
	at org.jitsi.meet.test.MuteTest.muteParticipant1BeforeParticipant2JoinsAndScreenshare(MuteTest.java:281)
	at org.jitsi.meet.test.MuteTest.muteAfterJoinCanShareAndUnmute(MuteTest.java:231)

[ERROR] tcpTest(org.jitsi.meet.test.TCPTest)  Time elapsed: 17.649 s  <<< FAILURE!
org.openqa.selenium.TimeoutException:
Expected condition failed: waiting for WebParticipant[web.participant2]@424106351#isIceConnected (tried for 15 second(s) with 500 milliseconds interval)
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03'
System info: host: 'David-Air.local', ip: '**IP**', os.name: 'Mac OS X', os.arch: 'aarch64', os.version: '12.0.1', java.version: '17.0.1'
Driver info: org.openqa.selenium.chrome.ChromeDriver
Capabilities {acceptInsecureCerts: false, browserName: chrome, browserVersion: 96.0.4664.55, chrome: {chromedriverVersion: 96.0.4664.45 (76e4c1bb2ab46..., userDataDir: /var/folders/6g/4hb15fqd6x7...}, goog:chromeOptions: {debuggerAddress: localhost:55275}, javascriptEnabled: true, networkConnectionEnabled: false, pageLoadStrategy: normal, platform: MAC, platformName: MAC, proxy: Proxy(), setWindowRect: true, strictFileInteractability: false, timeouts: {implicit: 0, pageLoad: 300000, script: 30000}, unhandledPromptBehavior: dismiss and notify, webauthn:extension:credBlob: true, webauthn:extension:largeBlob: true, webauthn:virtualAuthenticators: true}
Session ID: 085c268efbdf155d25a6688815a7784f
	at org.jitsi.meet.test.TCPTest.tcpTest(TCPTest.java:82)

[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR]   MuteTest.muteAfterJoinCanShareAndUnmute:231->muteParticipant1BeforeParticipant2JoinsAndScreenshare:281 » Timeout
[ERROR]   TCPTest.tcpTest:82->WebTestBase.ensureTwoParticipants:104->WebTestBase.ensureTwoParticipants:126->WebTestBase.ensureTwoParticipantsInternal:157 » Timeout
[INFO]
[ERROR] Tests run: 411, Failures: 2, Errors: 0, Skipped: 402
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:30 min
[INFO] Finished at: 2021-11-28T10:25:24Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.20:test (default-test) on project jitsi-meet-torture: There are test failures.

@damencho
Copy link
Member

The one muteAfterJoinCanShareAndUnmute is known to fail sometimes and we will be working on fixing it in the following weeks or we will disable it, as for the TCPTest ... not sure we run it anymore ... maybe something with it is not working it needs debugging the client ...

@bgrozev
Copy link
Member

bgrozev commented Nov 29, 2021 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@dkgrieshammer @damencho @bgrozev @rvvssvraju