Running all the unit tests crashes brackets

[WebsiteDeveloper]

When i run the unit tests in the current master, brackets will crash.

1. Open Brackets and open the unit test window
2. Run all unit tests

Result:
Brackets stops working

Expected:
Just running all tests

[TuckerWhitehouse]

I'm experiencing something similar, but with the Integration tests, not the unit tests.
I think it's failing when it gets to the Live Development tests and only if Developer Tools are open.

OS: Mac OS X 10.8.2
Brackets: sprint 22 development build 0.22.0-0 (master 05eed60)

1. Open Brackets

2. Run Integration unit tests
   Results: Tests pass successfully

3. Open Brackets

4. Show Developer Tools

5. Run Integration unit tests
   Results: All windows (including Jasmine Spec Runner) go gray

Running the Integration tests again, I get 1 error in Live Development: "should translate urls to/from local paths"

Not sure if this is similar to what your experiencing @WebsiteDeveloper.

[WebsiteDeveloper]

actually it was something different, for me but i can also repoduce the issue you described.
Some additional Information:
I am on a windows 7 machine and when i run all unit tests
my processor load jumps from 5% to 30% and when i run each test suite on it's own it works fine.
on thing i also noted is that these problems seem to be only with the unit tests involving an editor

Note: I have a virtual 8 core Intel Xeon 3.6GHz processor so it is very much processor load

[njx]

One thing to try is to make sure that you don't have any dev tools windows open if you're trying to run the full unit test suite. Having dev tools open seems to make things use up a bunch more memory and run a lot slower--normally not a problem for just one suite, but when you run them all together the memory usage seems to accumulate.

[WebsiteDeveloper]

i tried that out but still the window went grey when running all unit test suites

[redmunds]

@TuckerWhitehouse I have submitted a pull request to fix the "should translate urls to/from local paths" test, so that will be fixed soon.

@WebsiteDeveloper
Before running tests, make sure your brackets repo is clean using 'git status .' Sometimes test files get corrupted, so make sure there are none from previous test failures. It's also not a good idea to have uncommitted changes while running tests as you could lose them.

Do you build brackets-shell locally or run the version from the previous build? I don't think there have been any changes yet to brackets-shell in Sprint 22, but there were a few changes in Sprint 21.

Next try closing Chrome. Note that tests run a little faster if open a Chrome window with a single empty tab, but try it both ways. Definitely do not leave a Live Development connection open or have the Dev Tools open.

Once you open the Jasmine Window, which tab are you on: Unit, Integration, or Extensions? I was just able to run the Integration and Extensions tests, but I did have a couple time outs in the Integration tests for Live Development.

When the tests fail, how many Brackets windows do you have open? Sometimes a test will fail and leave a Brackets test window open that is "grey", but you can simply close those.

If the tests are still failing when you run All, then try running each group of test one at a time to try to narrow down which test is causing the problem for you.

Let us know your results.

[WebsiteDeveloper]

@redmunds

• i am using the sprint 21 build
• it happens with and without chrome open
• i'm running the Unit tests only Integration and extension test work fine (exept a couple of timeouts)
• i have the normal brackets window and the unit test window open, but both go grey and can not be closed normally i have to kill the process if i want to close the window
• the Editor, EditorCommandHandlers, EditorManager, FindReplace and MultiRangeInlineEditor seem to be the root of the problem, because they really slow down the entire brackest process
  maybe it has something to do with the handling of the editor, because all of them involve an editor?
  Anybody an idea?

[redmunds]

What happens if you run each set of test individually, one at a time? Which pass and which fail?

Have you tried building brackets-shell locally? There are build scripts that make it easy. Do you have Visual Studio installed? If not you can download VS Express for free.

Standard Windows response: try rebooting and trying again with no other apps open.

[WebsiteDeveloper]

all test pass, and i'll try building the shell and also rebooting.
i'll notify you of any results.

[redmunds]

Also uninstall all extensions.

[njx]

Reviewed. @redmunds -- if it turns out there's a real bug here, please prioritize accordingly or let us know and we'll take it up in a future bug review.

[WebsiteDeveloper]

i can reproduce it most of the time with my own built shell, it seems that brackets crashes only if the FileSystemIO unit tests fail due to timeouts.

[redmunds]

I have seen the problem with Brackets windows graying out and needing to be force closed when running tests, but it was on Mac 10.8 and I was purposefully stress testing the system: I had Dev Tools open and a Live Dev session going when I started running the Integration tests.

This sounds like it may be a resource deadlock, but that's a wild guess.

Be sure to build brackets-shell using Release mode during normal usage (because Debug mode is really slow), but it might be worth trying a Debug build to see if it might be able to help figure out what's going wrong.

[WebsiteDeveloper]

i made sure that i'm using release mode, but i can still reproduce the issue,
what is interesting, is that when Debuging the shell code in Visual Studio 2012 where the code runs much more slower than in the release build, i actually can't reproduce the issue.
It seems that something happens which causes the FileSystemIO unit tests to fail (due to timeouts), and thats pretty much the indicator for something like a resource deadlock when running all tests, note that this doesn't happen if i run each unit test suite alone.

[redmunds]

FYI, I hit this once on Mac 10.8 running tests under normal conditions. I think it was running All Integration tests.

[redmunds]

I trapped a case in debugger where Brackets went gray. Note that I was not running unit tests, I was paused in debugger:

Uncaught Error: Load timeout for modules: thirdparty/jstree_pre1.0_fix_1/jquery.jstree, preferences/PreferencesDialogs, utils/StringUtils, utils/ViewUtils, utils/CollectionUtils, file/NativeFileError, i18n!nls/urls_unnormalized5, i18n, utils/KeyEvent, editor/Editor, editor/InlineTextEditor, language/HTMLUtils, editor/MultiRangeInlineEditor, text!base-config/keyboard.json_unnormalized6, editor/CodeHintList, thirdparty/jslint/jslint, widgets/StatusBar, widgets/ModalBar, utils/StringMatch, widgets/PopUpManager, text!base-config/keyboard.json, i18n!nls/strings_unnormalized7, text!htmlContent/update-dialog.html, text!htmlContent/update-list.html, preferences/PreferenceStorage, utils/TokenUtils, utils/BuildInfoUtils, text!htmlContent/about-dialog.html, text!htmlContent/contributors-list.html, LiveDevelopment/Documents/CSSDocument, LiveDevelopment/Documents/HTMLDocument, LiveDevelopment/Documents/JSDocument, LiveDevelopment/Agents/ConsoleAgent, LiveDevelopment/Agents/RemoteAgent, LiveDevelopment/Agents/NetworkAgent, LiveDevelopment/Agents/CSSAgent, LiveDevelopment/Agents/ScriptAgent, LiveDevelopment/Agents/HighlightAgent, LiveDevelopment/Agents/GotoAgent, LiveDevelopment/Agents/EditAgent, LiveDevelopment/Agents/DOMNode, LiveDevelopment/Agents/DOMHelpers

[peterflynn]

In the past when we saw unit test crashes when running the full suite of tests, it was usually an out-of-memory error. Increasing memory usage also makes the process run slower, which could explain the console output Randy saw.

As NJ mentioned, make doubly sure you don't have a dev tools tab open because that seems to make V8 use a lot more memory and makes unit test crashes much easier to hit.

[WebsiteDeveloper]

I made sure no other windows at all were open, but still Brackets crashes.
Another thing is that all previously saved preferences get trashed after having to kill the process, and also new preferences don't seem to be saved anymore.
I have to clean the cef_data directory to reenable the saving of preferences.

[redmunds]

Once you get into the state where preferences won't save, then manually delete your cache folder: https://github.com/adobe/brackets/wiki/Cache-Folder .

[WebsiteDeveloper]

yeah i already knew that as mentioned above.

[TomMalbran]

There is an easier way to restore the old preferences since the problem is that the local storage is still storing the tests key. What I do is open the Developer tools, go to the resources tab, go to Local Storage -> Local Files, and you need to delete the last entrance with the tests key. This saves you from clearing your cache if you want to keep your preferences.

[peterflynn]

The preferences issue is #972. The simplest workaround to get unstuck from that state is to simply launch Brackets again and run any 'integration' test suite to completion. This essentially does automatically what Tom suggested.

[WebsiteDeveloper]

@redmunds closing this since it isn't reproduceable anymore.