WIP: Turn on render process tests on Linux in script/test

[daviwil]

Experimenting with turning on render process tests on Linux in CI.

[maxbrunsfeld]

Oh wow there's really not too many failures!

styling changes

These look like differences in the font or the rendering engine that Electron is using on Linux. We're expecting Korean characters like 세, Chinese characters like 我 and Kanji characters like ﾊ to have different widths than latin characters like x, but we're instead finding that they all have the same measured width.

[daviwil]

Now that the renderer process tests actually run on VSTS, I'd definitely love some help fixing the TextEditorComponent tests. If @maxbrunsfeld's idea is correct and this has to do with rendering engine differences on Linux, how would we go about figuring out the fixes for those specs?

[daviwil]

Here are the specific failures we're seeing from render process tests on Linux builds for both Travis and Azure Pipelines:

TextEditorComponent
  mouse input
    on the lines
      when there is only one cursor
        it positions the cursor on single-click or when middle-clicking
          Expected { row : 0, column : 25 } to equal [ 0, 0 ].
            at it (/home/travis/build/atom/atom/spec/text-editor-component-spec.js:2832:54)
            at <anonymous>
          Expected { row : 0, column : 54 } to equal [ 0, 79 ].
            at it (/home/travis/build/atom/atom/spec/text-editor-component-spec.js:2850:54)
            at <anonymous>
          Expected { row : 1, column : 25 } to equal [ 1, 0 ].
            at it (/home/travis/build/atom/atom/spec/text-editor-component-spec.js:2858:54)
            at <anonymous>
          Expected { row : 3, column : 39 } to equal [ 3, 14 ].
            at it (/home/travis/build/atom/atom/spec/text-editor-component-spec.js:2866:54)
            at <anonymous>
          Expected { row : 3, column : 40 } to equal [ 3, 15 ].
            at it (/home/travis/build/atom/atom/spec/text-editor-component-spec.js:2874:54)
            at <anonymous>
          Expected { row : 3, column : 39 } to equal [ 3, 14 ].
            at it (/home/travis/build/atom/atom/spec/text-editor-component-spec.js:2885:54)
            at <anonymous>
          Expected { row : 3, column : 41 } to equal [ 3, 16 ].
            at it (/home/travis/build/atom/atom/spec/text-editor-component-spec.js:2893:54)
            at <anonymous>
          Expected [ { screenRange : { start : { row : 0, column : 0 }, end : { row : 0, column : 1 } }, options : { clip : false } }, { screenRange : { start : { row : 0, column : 25 }, end : { row : 0, column : 26 } }, options : { clip : false } }, { screenRange : { start : { row : 12, column : 2 }, end : { row : 12, column : 3 } }, options : { clip : false } }, { screenRange : { start : { row : 12, column : 27 }, end : { row : 12, column : 28 } }, options : { clip : false } }, { screenRange : { start : { row : 0, column : 29 }, end : { row : 0, column : 30 } }, options : { clip : false } }, { screenRange : { start : { row : 0, column : 54 }, end : { row : 0, column : 55 } }, options : { clip : false } }, { screenRange : { start : { row : 1, column : 0 }, end : { row : 1, column : 1 } }, options : { clip : false } }, { screenRange : { start : { row : 1, column : 25 }, end : { row : 1, column : 26 } }, options : { clip : false } }, { screenRange : { start : { row : 3, column : 14 }, end : { row : 3, column : 15 } }, options : { clip : false } }, { screenRange : { start : { row : 3, column : 39 }, end : { row : 3, column : 40 } }, options : { clip : false } }, { screenRange : { start : { row : 3, column : 15 }, end : { row : 3, column : 16 } }, options : { clip : false } }, { screenRange : { start : { row : 3, column : 40 }, end : { row : 3, column : 41 } }, options : { clip : false } }, { screenRange : { start : { row : 3, column : 14 }, end : { row : 3, column : 15 } }, options : { clip : false } }, { screenRange : { start : { row : 3, column : 39 }, end : { row : 3, column : 40 } }, options : { clip : false } }, { screenRange : { start : { row : 3, column : 16 }, end : { row : 3, column : 17 } }, options : { clip : false } }, { screenRange : { start : { row : 3, column : 41 }, end : { row : 3, column : 42 } }, options : { clip : false } } ] to equal [  ].
            at it (/home/travis/build/atom/atom/spec/text-editor-component-spec.js:2895:51)
            at <anonymous>
  styling changes
    it updates the rendered content based on new measurements when the font dimensions change
      Expected 7.21875 not to be 7.21875.
        at it (/home/travis/build/atom/atom/spec/text-editor-component-spec.js:3832:47)
        at <anonymous>
      Expected 7.21875 not to be 7.21875.
        at it (/home/travis/build/atom/atom/spec/text-editor-component-spec.js:3833:45)
        at <anonymous>
      Expected 7.21875 not to be 7.21875.
        at it (/home/travis/build/atom/atom/spec/text-editor-component-spec.js:3834:47)
        at <anonymous>

[daviwil]

Grrr, Pipelines incorrectly reported a failure on the Linux tests, perhaps something else caused the exit code to be nonzero?

[daviwil]

Great, looks like main process tests crashed again, thought I had fixed that 😞

[daviwil]

Some context on this PR for anyone who picks it up later:

I've disabled the consistently failing tests here so that Linux test runs (both main and render process) are green on Travis CI. The only remaining work is figuring out why the main and/or render process tests fail to run on Azure Pipelines. I switched from gcc over to clang 3.5 for building the native components and that seemed to work at first but now the issue has come back.

In theory it shouldn't take much work to get these tests re-enabled!

[aminya]

An improved version of this is included in #21109. So we won't need this anymore.

[sadick254]

#21109 was already merged.