Javascript Runner Broken

[xcthulhu]

Not sure how this happened, I hope it wasn't one of my commits...

But when I run:

docker build -t codewars/cli-runner . ; docker run -i --entrypoint mocha codewars/cli-runner test/*

I get:

...
  2 failing

  1) javascript runner .run cw-2 error handling should gracefully handle reference errors:
     Uncaught AssertionError: expected '<DESCRIBE::>test\n<IT::>test1\n<ERROR::>ReferenceError: b is not defined<:LF:>        at Test.it<:LF:>        at Test.describe<:LF:>            <:LF:>            \n<COMPLETEDIN::>2\n<IT::>test2\n<PASSED::>Test Passed\n<COMPLETEDIN::>1\n<COMPLETEDIN::>9\n' to include '\\n'
      at /codewars/test/runners/javascript_spec.js:77:50
      at /codewars/lib/shovel.js:16:9
      at ChildProcess.<anonymous> (/codewars/lib/shovel.js:106:9)
      at ChildProcess.emit (events.js:98:17)
      at Process.ChildProcess._handle.onexit (child_process.js:809:12)

  2) javascript runner .run cw-2 error handling should gracefully top level handle reference errors:
     Uncaught AssertionError: expected '<ERROR::>ReferenceError: b is not defined<:LF:>        <:LF:>            \n' to include '\\n'
      at /codewars/test/runners/javascript_spec.js:91:50
      at /codewars/lib/shovel.js:16:9
      at ChildProcess.<anonymous> (/codewars/lib/shovel.js:106:9)
      at ChildProcess.emit (events.js:98:17)
      at Process.ChildProcess._handle.onexit (child_process.js:809:12)

[xcthulhu]

Looks like the newlines aren't escaped anymore... is that the only problem?

[jhoffner]

Mine are all passing when running node test. Try pulling latest and re-running. Maybe you are out of sync?

Also yes newlines are no longer escaped but instead replaced with the <:LF:> token. We ran into issues with escape levels and we found it easier to do it this way then to update the entire code stack to handle multiple levels of escaping via "JSON.stringify". We still need to update the Julia/Haskel/Clojure implementations to support <:LF:>

[xcthulhu]

So for <:LF:>, is this for user output, or test-framework output, or both?

[jhoffner]

test-framework output. Basically, each stdout line within the runner output is considered a new data item. However if a single data item is supposed to have multiple lines within it, then things get messed up.
So in the following example, we actually want there to be two "data items" but since the passed message includes a \n, its not possible to for the HTML formatter to know.

<PASSED::>Good\nJob
<FAILED::>Some error

So we change it to

<PASSED::>Good<:LF:>Job
<FAILED::>Some error

[xcthulhu]

Changing the test frameworks to support this is hard. I don’t know if this is possible in Julia.

Can this be done in the exec function in util.js?

On Jul 15, 2014, at 5:59 PM, Jake Hoffner notifications@github.com wrote:

test-framework output. Basically, each stdout line within the runner output is considered a new data item. However if a single data item is supposed to have multiple lines within it, then things get messed up.
So in the following example, we actually want there to be two "data items" but since the passed message includes a \n, its not possible to for the HTML formatter to know.

PASSED::Good\nJob
FAILED::Some error
So we change it to

PASSED::Good<:LF:>Job
FAILED::Some error
—
Reply to this email directly or view it on GitHub.

[jhoffner]

If we are able to format the <IT::>, <PASSED::> stuff then this should be able to be done in the same place. For example in Julia you could do it within the format_assertion method. Checkout the JS and Ruby version for how this is done in those.

It couldn't be done in util.js because at that point, the data is already written to stdout and we have no way of knowing if \n is meant to be escaped or not.

[xcthulhu]

Ok… well in that case there’s nothing to be done for Clojure or Julia, as I believe I don’t actually emit newlines in either one of those test-frameworks. I’ll double check.

This is pretty hard to do for Haskell, however...

On Jul 15, 2014, at 6:37 PM, Jake Hoffner notifications@github.com wrote:

If we are able to format the IT::, PASSED:: stuff then this should be able to be done in the same place. For example in Julia you could do it within the format_assertion method. Checkout the JS and Ruby version for how this is done in those.

It couldn't be done in util.js because at that point, the data is already written to stdout and we have no way of knowing if \n is meant to be escaped or not.

—
Reply to this email directly or view it on GitHub.

[jhoffner]

We need new lines though.

For example:

<PASSED::>Passed<PASSED::>Passed

would end up being shown to users as a single passed statement saying Passed<PASSED::>Passed.

[jhoffner]

Looks like Haskell should be possible. The Formatters.hs just needs to have its exampleSucceeded, exampleFailed and formatFailure methods tweaked.

[xcthulhu]

Well, hspec wraps a lot of other utilities that have newlines in their output, so I was worried about that.

[xcthulhu]

This appears to have been largely closed by d051ae9 and 04e3c66 (along with other commits)