Update jasmine to 2.x

[kiskoza]

Hi. As mentioned before on #966 the current test suite is not in a very good shape. I've encountered most of the issues while I was creating other contributions, all my PRs (#785 and #954) had problems when I tried to cover the changes with specs.

What do we have already?

First of all, we have a running CI. I can only imagine the amount of work you had to put in to make it work 🥇 🙇

On the other hand, the tools behind the test suite got a bit outdated and messy over time. If you check the package.json file, you can see that we have three packages: jasmine-json, jasmine-reporters and jasmine-tagged as direct dependencies. Two of them seem to be abandoned already and the third one is pinned on a pretty old version. To make it more complicated, jasmine-tagged depends on jasmine-focused (also abandoned) which depends on a direct commit of jasmine-node from github which seems to be v1.10.2 with some changes.

Unfortunately, that's not all yet. As I tried to understand how the test suite works, I've found that we're only using a few code snippets from the jasmine package and most of the jasmine codes are coming from the vendor folder - we have a vendored jasmine v1.3.1. Also, the file starts with an ominous comment: "Modified line". Of course, if you make a proper diff with an official jasmine v1.3.1 release, there are more differences than the documented ones. We also have a vendored jasmine-jquery and some helper methods that are not related to jasmine, but all the methods defined here are available on window / global, so the vendored jasmine file was used as a place to put global helper methods. Uff.

The rest of the setup is easy to understand break: we have spec/jasmine-test-runner.js and spec/spec-helper.js to set up the runtime and to add a lot of helper methods. Some of them are handling the test logging and the integration with the atom environment, but a huge chunk of it just tries to solve problems already solved by more recent jasmine versions, like re-running failed tests and handling async specs. If you feel like some codes are hard to read, then it's most likely an auto-generated javascript coming from the old coffescript files.

What should we have?

Easy: a test suite without all the workarounds in the codebase. 🙃

To be more serious: The old jasmine version lacks a lot of quality of life features that a modern codebase / test suite should be able to rely on. I'm talking about retries without custom filtering, better spies and expectations on complex objects, supported asnyc tests, etc, etc. So in an ideal world, we should be able to use the latest-ish jasmine to write new tests or make the existing more reliable.

In this PR, I'm trying to replace all the mess with a working jasmine 2.x version. It won't solve all the issues immediately, but could be a good start.

In my commits

• I tried to remove as many lines as I was able to with minimum number of failing scenarios (e.g.: I dropped some formatters that seemingly nothing requires, dropped the jasmine-tagged package that I need to replace with something later).
• I have some code changes to make a common entrypoint for jasmine as we should import it only once (and from only the npm package!)
• Moved helper functions from the vendored jasmine file to actual helper files
• I switched to jasmine 2.x (also kept the old jasmine-node for debugging purposes)
• And finally I fixed the first spec file to make it pass 🎉

I would like to continue my work here - there's 70+ spec files in the spec folder and we haven't talked about package specs. As this could be a huge task, I can't promise anything. Also, I'd appreciate some feedback from the current state of the PR. Do you like the idea? Is there anything that you would do differently?

[savetheclocktower]

I think this project has a guardian angel. Because every time I think “we don't have enough contributors to the core codebase,” someone comes in with a PR and says “I want to rewrite this extremely complex thing just because it's been bugging me.”

I think you've bitten off quite a bit here, but if you can pull this off, I'd want to find a way to give you the Nobel Prize of Open-Source Contribution, an award that doesn't actually exist.

One suggestion: every package, even built-in packages, can change their test runner independent of others. So it might be easier to introduce Jasmine 2.x as another option for test-runners; that way we'd be able to merge this PR earlier and migrate things over incrementally. We'd probably have to keep the Jasmine 1.x runner around anyway for community packages — unless we found a way to keep 100% backward-compatibility.

[kiskoza]

I've been working on this one and have some updates:

I've introduced jasmine2-test-runner and moved over pulsar's tests to run using it. Almost all tests are passing, there's only a few flaky specs that are failing here and there. I checked a few previous runs and they seem to be flaky already and sometimes the retry mechanism just skips them. I'm planning to re-introduce a retry workflow to make them pass again. Currently all the test fixes are in one gigantic commit, I'm planning to break it up into smaller ones and write an upgrade guide using / following the commits. Also, I broke the spec-helper file into several helper files. This way we can write some comments on why they are required, what are they doing and all of them holds one single responsibility.

I've also kept jamine1-test-runner and made it the default for now if the package.json doesn't set anything else. I'm planning to convert at least one package to jasmine2 to make sure that it can pick up the new test runner. Also, I'd like to add a deprecation warning and set a date or milestone to remove it from the project.

[UziTech]

I did create an atom-jasmine2-test-runner and atom-jasmine3-test-runner for running tests in packages for atom a while ago if those can help.

Those include updated packages for:

• custom-matchers
• jasmine-focused
• jasmine-json
• jasmine-tagged
• and others