Search includes node_modules folder and does not filter on Windows

[michelkaporin]

VS Code: Insiders

First and second bullet item of the smoke test are broken on Windows:
https://github.com/Microsoft/vscode/wiki/Smoke-Test#search

Detected by automated Smoke Test in build ID 29998.

[ghost]

I'm having problems searching in a large repository, DefinitelyTyped.
cd DefinitelyTyped, code ., ctrl-shift-F, and search for declare module with "files to include" of types.
VSCode 1.13.1: Tons of results.
VSCode 1.14.0-Insider: No results.
There doesn't seem to be a problem in smaller repositories.
Update: It works on linux with 1.14.0-insider.

[roblourens]

This should be fixed in today's insiders. I'll try the smoke test.

[roblourens]

I'm having trouble running the smoke test on Windows. Almost every test fails with a timeout or generic error. Not sure what I did wrong.

[michelkaporin]

@roblourens I was doing some big refactorings around the smoke test on Friday, so that can happen. I will take a look at it on Monday

[michelkaporin]

@roblourens did you run it locally? Don't see a triggered build from you for the smoke test. If yes, it would be good to see which errors did you get.

[roblourens]

Yeah it was local, I'll try again today.

[roblourens]

Ran with
npm test -- --latest "C:\Program Files (x86)\Microsoft VS Code Insiders\Code - Insiders.exe" --stable "C:\Program Files (x86)\Microsoft VS Code\Code.exe"

Almost all failed, results below. Are there any other logs to check?

``` 1) Smoke Test Suite Data Migration checks if the Untitled file is restored migrating from stable to latest: AssertionError [ERR_ASSERTION]: Untitled-1 tab was not restored after migration. at Context. (out\tests\data-migration.js:53:24) at Generator.next () at fulfilled (out\tests\data-migration.js:8:58) at at process._tickDomainCallback (internal/process/next_tick.js:208:7)

2. Smoke Test Suite Data Migration checks if the newly created dirty file is restored migrating from stable to latest:

   AssertionError [ERR_ASSERTION]: Entered text was not correctly restored after migration.

   • expected - actual

   -This is going to be an unsaved files_that is dirty
   +This is going to be an unsaved file_that is dirty.

   at Context. (out\tests\data-migration.js:78:24)
   at Generator.next ()
   at fulfilled (out\tests\data-migration.js:8:58)
   at
   at process._tickDomainCallback (internal/process/next_tick.js:208:7)

3. Smoke Test Suite Data Loss verifies that 'hot exit' works for dirty files:
   AssertionError [ERR_ASSERTION]: Untitled-1 tab is not present after reopening.
   at Context. (out\tests\data-loss.js:49:24)
   at Generator.next ()
   at fulfilled (out\tests\data-loss.js:8:58)
   at
   at process._tickDomainCallback (internal/process/next_tick.js:208:7)

4. Smoke Test Suite Data Loss verifies that contents of the dirty files are restored after 'hot exit':
   AssertionError [ERR_ASSERTION]: Active file contents are different after restore.
   at Context. (out\tests\data-loss.js:71:24)
   at Generator.next ()
   at fulfilled (out\tests\data-loss.js:8:58)
   at
   at process._tickDomainCallback (internal/process/next_tick.js:208:7)

5. Smoke Test Suite Explorer quick open search produces correct result:

   AssertionError [ERR_ASSERTION]: 0 == 7

   • expected - actual

   -0
   +7

   at Context. (out\tests\explorer.js:40:24)
   at Generator.next ()
   at fulfilled (out\tests\explorer.js:8:58)
   at

6. Smoke Test Suite Explorer quick open respects fuzzy matching:

   AssertionError [ERR_ASSERTION]: 0 == 3

   • expected - actual

   -0
   +3

   at Context. (out\tests\explorer.js:49:24)
   at Generator.next ()
   at fulfilled (out\tests\explorer.js:8:58)
   at

7. Smoke Test Suite Configuration and views turns off editor line numbers and verifies the live change:

   AssertionError [ERR_ASSERTION]: Line numbers are not present in the editor before disabling them.

   • expected - actual

   -0
   +1

   at Context. (out\tests\configuration-views.js:42:24)
   at Generator.next ()
   at fulfilled (out\tests\configuration-views.js:8:58)
   at
   at process._tickDomainCallback (internal/process/next_tick.js:208:7)

8. Smoke Test Suite Configuration and views changes 'workbench.action.toggleSidebarPosition' command key binding and verifies it:
   Error: the string "Could not retrieve the element in 45 seconds." was thrown, throw an Error :)
   at

9. Smoke Test Suite Search searches for body & checks for correct result number:
   Error: the string "Could not retrieve the element in 45 seconds." was thrown, throw an Error :)
   at

10. Smoke Test Suite Search searches only for *.js files & checks for correct result number:
    An element could not be located on the page using the given search parameters.
    Error

11. Smoke Test Suite Search dismisses result & checks for correct result number:
    An element could not be located on the page using the given search parameters.
    Error

12. Smoke Test Suite Search replaces first search result with a replace term:
    An element could not be located on the page using the given search parameters.
    Error

13. Smoke Test Suite CSS verifies quick outline:

    AssertionError [ERR_ASSERTION]: Quick outline symbol count is wrong.

    • expected - actual

    -0
    +2

    at Context. (out\tests\css.js:43:24)
    at Generator.next ()
    at fulfilled (out\tests\css.js:8:58)
    at

14. Smoke Test Suite CSS verifies warnings for the empty rule:
    AssertionError [ERR_ASSERTION]: Warning squiggle is not shown in 'style.css'.
    at Context. (out\tests\css.js:52:24)
    at Generator.next ()
    at fulfilled (out\tests\css.js:8:58)
    at
    at process._tickDomainCallback (internal/process/next_tick.js:208:7)

15. Smoke Test Suite CSS verifies that warning becomes an error once setting changed:
    Error: the string "Failed to select settings editor to add a setting." was thrown, throw an Error :)
    at
    at process._tickDomainCallback (internal/process/next_tick.js:208:7)

16. Smoke Test Suite JavaScript shows correct quick outline:

    AssertionError [ERR_ASSERTION]: Quick outline elements count does not match to expected.

    • expected - actual

    -0
    +12

    at Context. (out\tests\javascript.js:43:24)
    at Generator.next ()
    at fulfilled (out\tests\javascript.js:8:58)
    at

17. Smoke Test Suite JavaScript finds 'All References' to 'app':
    Error: Timeout of 360000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

18. Smoke Test Suite JavaScript renames local 'app' variable:
    Error: Timeout of 360000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

19. Smoke Test Suite JavaScript folds/unfolds the code correctly:
    Error: Timeout of 360000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

20. Smoke Test Suite JavaScript verifies that 'Go To Definition' works:
    Error: Timeout of 360000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

21. Smoke Test Suite JavaScript verifies that 'Peek Definition' works:
    Error: Timeout of 360000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

22. Smoke Test Suite Debugging JavaScript autodetects program attribute for launch.json:
    Error: the string "Clicking on debug configuration gear failed." was thrown, throw an Error :)
    at

23. Smoke Test Suite Debugging JavaScript can set a breakpoint and verify if it's set:
    An element could not be located on the page using the given search parameters.
    Error

24. Smoke Test Suite Git verifies current changes are picked up by Git viewlet:
    AssertionError [ERR_ASSERTION]: app.js change does not appear in SCM viewlet.
    at Context. (out\tests\git.js:42:24)
    at Generator.next ()
    at fulfilled (out\tests\git.js:8:58)
    at
    at process._tickDomainCallback (internal/process/next_tick.js:208:7)

25. Smoke Test Suite Git verifies 'app.js' diff viewer changes:
    Error: the string "Cannot fine app.js in a viewlet." was thrown, throw an Error :)
    at

26. Smoke Test Suite Git stages 'app.js' changes and checks stage count:
    Error: the string "app.js was not found in SCM viewlet" was thrown, throw an Error :)
    at
    at process._tickDomainCallback (internal/process/next_tick.js:208:7)

27. Smoke Test Suite Git stages, commits change to 'app.js' locally and verifies outgoing change:
    Error: the string "app.js was not found in SCM viewlet" was thrown, throw an Error :)
    at
    at process._tickDomainCallback (internal/process/next_tick.js:208:7)

28. Smoke Test Suite Integrated Terminal opens terminal, runs 'echo' and verifies the output:

    AssertionError [ERR_ASSERTION]: Terminal output does not contain echo.

    • expected - actual

    -false
    +true

    at Context. (out\tests\integrated-terminal.js:45:24)
    at Generator.next ()
    at fulfilled (out\tests\integrated-terminal.js:8:58)
    at
    at process._tickDomainCallback (internal/process/next_tick.js:208:7)

29. Smoke Test Suite Status Bar verifies presence of all default status bar elements:

    AssertionError [ERR_ASSERTION]: Encoding indicator is not visible.

    • expected - actual

    -false
    +true

    at Context. (out\tests\statusbar.js:45:24)
    at Generator.next ()
    at fulfilled (out\tests\statusbar.js:8:58)
    at
    at process._tickDomainCallback (internal/process/next_tick.js:208:7)

30. Smoke Test Suite Status Bar verifies that 'quick open' opens when clicking on status bar elements:
    element not visible
    Error

31. Smoke Test Suite Status Bar checks if 'Go to Line' works if called from the status bar:
    element not visible
    Error

32. Smoke Test Suite Status Bar verifies if changing EOL is reflected in the status bar:
    element not visible
    Error

33. Smoke Test Suite Tasks verifies that eslint task results in 1 problem:
    Error: the string "Could not retrieve the element in 45 seconds." was thrown, throw an Error :)
    at

34. Smoke Test Suite Tasks is able to select 'Git' output:
    An element could not be located on the page using the given search parameters.
    Error

35. Smoke Test Suite Tasks ensures that build task produces error in index.js:
    Error: the string "Could not retrieve the element in 45 seconds." was thrown, throw an Error :)
    at

36. Smoke Test Suite Tasks verifies build error is reflected in 'Problems View':
    Error: the string "Could not retrieve the element in 45 seconds." was thrown, throw an Error :)
    at

37. Smoke Test Suite Extensions installs 'vscode-smoketest-check' extension and verifies reload is prompted:
    An element could not be located on the page using the given search parameters ("div[id="workbench.view.extensions"] .search-box").
    Error

38. Smoke Test Suite Extensions installs an extension and checks if it works on restart:
    An element could not be located on the page using the given search parameters ("div[id="workbench.view.extensions"] .search-box").
    Error

39. Smoke Test Suite Localization starts with 'DE' locale and verifies title and viewlets text is in German:

    AssertionError [ERR_ASSERTION]: 'explorer' == 'suchen'

    • expected - actual

    -explorer
    +suchen

    at Context. (out\tests\localization.js:39:24)
    at Generator.next ()
    at fulfilled (out\tests\localization.js:8:58)
    at
    at process._tickDomainCallback (internal/process/next_tick.js:208:7)

[michelkaporin]

@roblourens did you close all instances of VS Code when running a test?

The is the only log to check. You can take a look at screenshots under test\smoke\test_data\screenshots, which are captured on every test action. You can send them to me if you closed all VS Code instances before running and think you ran them correctly.

[michelkaporin]

@roblourens still can reproduce an issue on Windows only.
Triggered smoke test buildId=30077 which also showed a problem. Other OS are fine.