Skip to content

Conversation

pzrq
Copy link
Contributor

@pzrq pzrq commented Oct 4, 2017

This PR should resolve several sporadic hard to reproduce failures seen in Compass at various times.

More available resources in a build container

The build containers in our legacy build infrastructure have had 1.5 cores (with burst capacity) and 3GB of memory. The CPU resources are now guaranteed, which means less impact by noisy neighbors on the same host machine. Build times throughout the day should be much more consistent on the new container stack.

The new containers have 2 dedicated cores available and 4 GB of memory.

https://blog.travis-ci.com/2014-12-17-faster-builds-with-container-based-infrastructure/

It also removes some redundant calls, e.g. dropDatabase('music'), within the functional selenium / webdriverio / spectron test suite, which should hopefully make them easier to understand and more predictable, please see the individual commits within this PR for more details.

See also #1247

pzrq added 5 commits October 4, 2017 13:10
It was not created by this test, so if it's needed it would be for earlier test runs? I don't see value in keeping it.

Plus - it might be how the chain of test timeouts gets broken? Let's find out with another test.
This should be the second failure in at least one of the "main failure chain" pairs, i.e.
https://travis-ci.com/10gen/compass/jobs/92384423

(but not)
https://travis-ci.com/10gen/compass/jobs/92384429
They are redundant as they are already done by goToCollection, which is at least a code smell worthwhile removing.
@pzrq pzrq self-assigned this Oct 4, 2017
@pzrq
Copy link
Contributor Author

pzrq commented Oct 4, 2017

Results are 6 reds in 44 runs

These issues may also have been present before on non-container-based infrastructure, I will update the tickets if I find any evidence that they were present both before and after the switch.

CollectionsStore "before all" hook: Error: Timeout of 2000ms exceeded.

Tracked as COMPASS-2163

  1) CollectionsStore "before all" hook:
     Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

https://travis-ci.com/10gen/compass/jobs/93094619
https://travis-ci.com/10gen/compass/jobs/93096547

#query-bar: 'returns only the fields included in the project plus _id'

(Ticket to be created)

1) #query-bar when using advanced query options when applying queries from the documents tab when applying a projection returns only the fields included in the project plus _id:
      AssertionError: expected [ '_id', 'member_id' ] to deeply equal [ '_id', 'member_id', 'name' ]
      + expected - actual
       [
         "_id"
         "member_id"
      +  "name"
       ]

https://travis-ci.com/10gen/compass/jobs/93096552

#indexes: 'adds the index to the list'

(Ticket yet to be created)

  1) #indexes when navigating to the indexes tab when creating an index when the index is valid when the indexes are sorted adds the index to the list:
     Error: element ("[data-test-id='index-table-name'][title=name_1]") still not visible after 8000ms
      at elements("[data-test-id='index-table-name'][title=name_1]") - isVisible.js:54:17
      at isVisible("[data-test-id='index-table-name'][title=name_1]") - waitForVisible.js:73:22

https://travis-ci.com/10gen/compass/jobs/93096592

Renderer tests hang

(Ticket feels like https://jira.mongodb.org/browse/COMPASS-707)

Running renderer tests.
Xlib:  extension "RANDR" missing on display ":99.0".
Xlib:  extension "RANDR" missing on display ":99.0".
process 5879: D-Bus library appears to be incorrectly set up; failed to read machine uuid: Failed to open "/etc/machine-id": No such file or directory
See the manual page for dbus-uuidgen to correct this issue.
No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received
The build has been terminated

https://travis-ci.com/10gen/compass/jobs/93097249

#launch "before all" hook: TypeError: Cannot read property 'client' of undefined

(Ticket yet to be created)

  1) #launch "before all" hook:
     TypeError: Cannot read property 'client' of undefined
      at test/functional/_launch.test.js:12:27
      at process._tickDomainCallback (internal/process/next_tick.js:129:7)

https://travis-ci.com/10gen/compass/jobs/93097269

@pzrq
Copy link
Contributor Author

pzrq commented Oct 4, 2017

Comparison - before was 16 reds in 40 runs

Comparing these results to #1247 which had 16 reds in the first 40 runs, I'd call this PR an improvement and recommend the switch.

@pzrq pzrq requested a review from matt-d-rat October 4, 2017 03:29
@pzrq
Copy link
Contributor Author

pzrq commented Oct 4, 2017

@durran Please feel welcome to review and/or merge if you think this would make your work on COMPASS-1878 easier too.

Copy link
Contributor

@matt-d-rat matt-d-rat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work on this @pzrq very thorough investigation. The query bar tests will no longer effect the main compass build following the externalisation of that as a plugin. LGTM 👍

@pzrq pzrq merged commit cb11b62 into master Oct 5, 2017
@pzrq pzrq deleted the COMPASS-2159-travis branch October 5, 2017 03:00
pzrq added a commit that referenced this pull request Oct 5, 2017
…tructure (#1255)

* 🔥 Remove #databases dropDatabase('music') call

It was not created by this test, so if it's needed it would be for earlier test runs? I don't see value in keeping it.

Plus - it might be how the chain of test timeouts gets broken? Let's find out with another test.

* 🐛 Handle the case where app is undefined

This should be the second failure in at least one of the "main failure chain" pairs, i.e.
https://travis-ci.com/10gen/compass/jobs/92384423

(but not)
https://travis-ci.com/10gen/compass/jobs/92384429

* 🐛 Handle app null as well as undefined

https://travis-ci.com/10gen/compass/jobs/92556220

* 🚧 Try container-based infrastructure

Just seeing if it's a cheap win today.
https://blog.travis-ci.com/2014-12-17-faster-builds-with-container-based-infrastructure/

* 🎨 Remove clickDatabaseInSidebar/waitForSidebar calls

They are redundant as they are already done by goToCollection, which is at least a code smell worthwhile removing.

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"
pzrq added a commit that referenced this pull request Oct 5, 2017
…tructure (#1255)

* 🔥 Remove #databases dropDatabase('music') call

It was not created by this test, so if it's needed it would be for earlier test runs? I don't see value in keeping it.

Plus - it might be how the chain of test timeouts gets broken? Let's find out with another test.

* 🐛 Handle the case where app is undefined

This should be the second failure in at least one of the "main failure chain" pairs, i.e.
https://travis-ci.com/10gen/compass/jobs/92384423

(but not)
https://travis-ci.com/10gen/compass/jobs/92384429

* 🐛 Handle app null as well as undefined

https://travis-ci.com/10gen/compass/jobs/92556220

* 🚧 Try container-based infrastructure

Just seeing if it's a cheap win today.
https://blog.travis-ci.com/2014-12-17-faster-builds-with-container-based-infrastructure/

* 🎨 Remove clickDatabaseInSidebar/waitForSidebar calls

They are redundant as they are already done by goToCollection, which is at least a code smell worthwhile removing.

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"
pzrq added a commit that referenced this pull request Oct 5, 2017
…tructure (#1255)

* 🔥 Remove #databases dropDatabase('music') call

It was not created by this test, so if it's needed it would be for earlier test runs? I don't see value in keeping it.

Plus - it might be how the chain of test timeouts gets broken? Let's find out with another test.

* 🐛 Handle the case where app is undefined

This should be the second failure in at least one of the "main failure chain" pairs, i.e.
https://travis-ci.com/10gen/compass/jobs/92384423

(but not)
https://travis-ci.com/10gen/compass/jobs/92384429

* 🐛 Handle app null as well as undefined

https://travis-ci.com/10gen/compass/jobs/92556220

* 🚧 Try container-based infrastructure

Just seeing if it's a cheap win today.
https://blog.travis-ci.com/2014-12-17-faster-builds-with-container-based-infrastructure/

* 🎨 Remove clickDatabaseInSidebar/waitForSidebar calls

They are redundant as they are already done by goToCollection, which is at least a code smell worthwhile removing.

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

#retry
  1) #collections when creating & deleting collections when viewing the database when creating a collection when deleting a collection pressing enter on correct collection name removes collection:
     Error: element ("[data-test-id='drop-collection-modal']") still not visible after 8000ms
https://travis-ci.com/10gen/compass/jobs/93287440
pzrq added a commit that referenced this pull request Oct 5, 2017
…tructure (#1255)

* 🔥 Remove #databases dropDatabase('music') call

It was not created by this test, so if it's needed it would be for earlier test runs? I don't see value in keeping it.

Plus - it might be how the chain of test timeouts gets broken? Let's find out with another test.

* 🐛 Handle the case where app is undefined

This should be the second failure in at least one of the "main failure chain" pairs, i.e.
https://travis-ci.com/10gen/compass/jobs/92384423

(but not)
https://travis-ci.com/10gen/compass/jobs/92384429

* 🐛 Handle app null as well as undefined

https://travis-ci.com/10gen/compass/jobs/92556220

* 🚧 Try container-based infrastructure

Just seeing if it's a cheap win today.
https://blog.travis-ci.com/2014-12-17-faster-builds-with-container-based-infrastructure/

* 🎨 Remove clickDatabaseInSidebar/waitForSidebar calls

They are redundant as they are already done by goToCollection, which is at least a code smell worthwhile removing.

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

#retry
  1) CollectionsStore "before all" hook:
     Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
https://travis-ci.com/10gen/compass/jobs/93287442
pzrq added a commit that referenced this pull request Oct 5, 2017
…tructure (#1255)

* 🔥 Remove #databases dropDatabase('music') call

It was not created by this test, so if it's needed it would be for earlier test runs? I don't see value in keeping it.

Plus - it might be how the chain of test timeouts gets broken? Let's find out with another test.

* 🐛 Handle the case where app is undefined

This should be the second failure in at least one of the "main failure chain" pairs, i.e.
https://travis-ci.com/10gen/compass/jobs/92384423

(but not)
https://travis-ci.com/10gen/compass/jobs/92384429

* 🐛 Handle app null as well as undefined

https://travis-ci.com/10gen/compass/jobs/92556220

* 🚧 Try container-based infrastructure

Just seeing if it's a cheap win today.
https://blog.travis-ci.com/2014-12-17-faster-builds-with-container-based-infrastructure/

* 🎨 Remove clickDatabaseInSidebar/waitForSidebar calls

They are redundant as they are already done by goToCollection, which is at least a code smell worthwhile removing.

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

#retry
  1) #collections when creating & deleting collections when viewing the database when creating a collection when deleting a collection pressing enter on correct collection name removes collection:
     Error: element ("[data-test-id='drop-collection-modal']") still not visible after 8000ms
https://travis-ci.com/10gen/compass/jobs/93287440

  1) #schema when applying a filter in the schema tab "before all" hook:
https://travis-ci.com/10gen/compass/jobs/93288256
pzrq added a commit that referenced this pull request Oct 5, 2017
…tructure (#1255) (#1258)

* 🔥 Remove #databases dropDatabase('music') call

It was not created by this test, so if it's needed it would be for earlier test runs? I don't see value in keeping it.

Plus - it might be how the chain of test timeouts gets broken? Let's find out with another test.

* 🐛 Handle the case where app is undefined

This should be the second failure in at least one of the "main failure chain" pairs, i.e.
https://travis-ci.com/10gen/compass/jobs/92384423

(but not)
https://travis-ci.com/10gen/compass/jobs/92384429

* 🐛 Handle app null as well as undefined

https://travis-ci.com/10gen/compass/jobs/92556220

* 🚧 Try container-based infrastructure

Just seeing if it's a cheap win today.
https://blog.travis-ci.com/2014-12-17-faster-builds-with-container-based-infrastructure/

* 🎨 Remove clickDatabaseInSidebar/waitForSidebar calls

They are redundant as they are already done by goToCollection, which is at least a code smell worthwhile removing.

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

#retry
  1) CollectionsStore "before all" hook:
     Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.
https://travis-ci.com/10gen/compass/jobs/93287442
pzrq added a commit that referenced this pull request Oct 5, 2017
…odal' functional test race + Switch to TravisCI container-based infras… (#1165) (#1259)

* [Backport 1.8] COMPASS 2159 Switch to TravisCI container-based infrastructure (#1255)

* 🔥 Remove #databases dropDatabase('music') call

It was not created by this test, so if it's needed it would be for earlier test runs? I don't see value in keeping it.

Plus - it might be how the chain of test timeouts gets broken? Let's find out with another test.

* 🐛 Handle the case where app is undefined

This should be the second failure in at least one of the "main failure chain" pairs, i.e.
https://travis-ci.com/10gen/compass/jobs/92384423

(but not)
https://travis-ci.com/10gen/compass/jobs/92384429

* 🐛 Handle app null as well as undefined

https://travis-ci.com/10gen/compass/jobs/92556220

* 🚧 Try container-based infrastructure

Just seeing if it's a cheap win today.
https://blog.travis-ci.com/2014-12-17-faster-builds-with-container-based-infrastructure/

* 🎨 Remove clickDatabaseInSidebar/waitForSidebar calls

They are redundant as they are already done by goToCollection, which is at least a code smell worthwhile removing.

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

#retry
  1) #collections when creating & deleting collections when viewing the database when creating a collection when deleting a collection pressing enter on correct collection name removes collection:
     Error: element ("[data-test-id='drop-collection-modal']") still not visible after 8000ms
https://travis-ci.com/10gen/compass/jobs/93287440

  1) #schema when applying a filter in the schema tab "before all" hook:
https://travis-ci.com/10gen/compass/jobs/93288256

* [Backport 1.8] COMPASS 1424: Resolve 'drop-collection-modal' functional test race (#1165)

* 🐛 Resolve 'drop-collection-modal functional test race

e.g. https://travis-ci.com/10gen/compass/jobs/83567154

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"

* noop

* Revert "noop"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants