Notebook javascript test suite using CasperJS #4285

ivanov · 2013-09-26T16:50:53Z

With this PR, you can just iptest js, and if casperjs is found, and the
ipython notebook dependencies are met, the test suite will run. There are
still a couple of things left to polish off, but it works.

closes #2292

@minrk's input on the following two would be great:

redirect stdout and stderr of the notebook server (mitigated by --log-level=0)
AssertionError: Request closed (mitigated by not deleting individual notebooks)

     ERROR:tornado.application:Exception in callback <functools.partial object at 0x57bdf18>
    Traceback (most recent call last):
      File "/home/pi/code/ipython-sloan/pyzmq/zmq/eventloop/minitornado/ioloop.py", line 463, in _run_callback
        callback()
      File "/home/pi/code/ipython-sloan/pyzmq/zmq/eventloop/minitornado/stack_context.py", line 331, in wrapped
        raise_exc_info(exc)
      File "/home/pi/code/ipython-sloan/pyzmq/zmq/eventloop/minitornado/stack_context.py", line 302, in wrapped
        ret = fn(*args, **kwargs)
      File "/usr/lib/python2.7/dist-packages/tornado/iostream.py", line 311, in wrapper
        callback(*args)
      File "/usr/lib/python2.7/dist-packages/tornado/httpserver.py", line 268, in _on_headers
        self.request_callback(self._request)
      File "/usr/lib/python2.7/dist-packages/tornado/web.py", line 1427, in __call__
        handler._execute(transforms, *args, **kwargs)
      File "/usr/lib/python2.7/dist-packages/tornado/web.py", line 1046, in _execute
        self._handle_request_exception(e)
      File "/usr/lib/python2.7/dist-packages/tornado/web.py", line 1086, in _handle_request_exception
        self.send_error(500, exc_info=sys.exc_info())
      File "/usr/lib/python2.7/dist-packages/tornado/web.py", line 741, in send_error
        self.finish()
      File "/usr/lib/python2.7/dist-packages/tornado/web.py", line 721, in finish
        self.flush(include_footers=True)
      File "/usr/lib/python2.7/dist-packages/tornado/web.py", line 681, in flush
        self.request.write(headers + chunk, callback=callback)
      File "/usr/lib/python2.7/dist-packages/tornado/httpserver.py", line 418, in write
        self.connection.write(chunk, callback=callback)
      File "/usr/lib/python2.7/dist-packages/tornado/httpserver.py", line 183, in write
        assert self._request, "Request closed"
    AssertionError: Request closed

minrk · 2013-09-26T17:07:58Z

Hooray!

ellisonbg · 2013-09-26T17:10:33Z

:-)

On Thu, Sep 26, 2013 at 10:07 AM, Min RK notifications@github.com wrote:

Hooray!

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/4285#issuecomment-25185169
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
bgranger@calpoly.edu and ellisonbg@gmail.com

ivanov · 2013-09-26T17:33:04Z

rebased on top of master

ellisonbg · 2013-09-26T20:53:20Z

Should we close #3125 ?

ellisonbg · 2013-09-26T21:01:56Z

IPython/html/tests/casperjs/README.md

+
+The file `util.js` contains utility functions for tests,
+including a hardcoded path to a running notebook server
+(http://127.0.0.1:8888 by default).


We should run the test suite using a notebook server on a different port than our default. Otherwise, if someone has a notebook server running already, the test suite would use it, potentially clobbering their notebooks. Pick some high random port number like 10345

sorry, that README was outdated, we actually grab a new port number as usual, and pass that along to the test suite

ellisonbg · 2013-09-26T21:15:13Z

I love the look of this - I think that writing tests, with a bit more infrastructure in utils.py is going to be pretty nice. I should note that when I run the tests, I get lots of failures though. Is that expected or do you think it is a timing issue with the starting of the notebook server?

ivanov · 2013-09-26T21:43:13Z

Should we close #3125 ?

There's a commit in here that will do that on merge.

dwyde · 2013-09-29T04:27:16Z

Glad to see this has been revived :-)

I just realized that the code cell execution test fails in Python 3 because of the print statement...

takluyver · 2013-10-04T22:46:05Z

IPython/html/tests/casperjs/test_cases/execute_code_cell.js

+casper.notebook_test(function () {
+    this.evaluate(function () {
+        var cell = IPython.notebook.get_cell(0);
+        cell.set_text('a=10; print a');


pablooliveira · 2013-10-09T07:12:56Z

First, this is really nice, I hope it will be merged soon :)

I tried writing some new js tests. When tests fail, I get sometimes (but not always) the following traceback:

2013-10-09 09:04:02.153 [tornado.application] ERROR | Uncaught exception, closing connection.
Traceback (most recent call last):
  File "/home/poliveira/ipython/env/local/lib/python2.7/site-packages/tornado/iostream.py", line 341, in wrapper
    callback(*args)
  File "/home/poliveira/ipython/env/local/lib/python2.7/site-packages/tornado/stack_context.py", line 331, in wrapped
    raise_exc_info(exc)
  File "/home/poliveira/ipython/env/local/lib/python2.7/site-packages/tornado/stack_context.py", line 302, in wrapped
    ret = fn(*args, **kwargs)
  File "/home/poliveira/ipython/env/local/lib/python2.7/site-packages/tornado/websocket.py", line 450, in _on_end_delimiter
    self._receive_message()
  File "/home/poliveira/ipython/env/local/lib/python2.7/site-packages/tornado/websocket.py", line 434, in _receive_message
    self.stream.read_bytes(1, self._on_frame_type)
  File "/home/poliveira/ipython/env/local/lib/python2.7/site-packages/tornado/iostream.py", line 168, in read_bytes
    self._try_inline_read()
  File "/home/poliveira/ipython/env/local/lib/python2.7/site-packages/tornado/iostream.py", line 418, in _try_inline_read
    self._check_closed()
  File "/home/poliveira/ipython/env/local/lib/python2.7/site-packages/tornado/iostream.py", line 580, in _check_closed
    raise StreamClosedError("Stream is closed")
StreamClosedError: Stream is closed
ERROR:tornado.application:Uncaught exception, closing connection.

Nevertheless, the number of passed/failed tests is correct, so maybe this is not that important.

@ellisonbg

as suggested by @ellisonbg in ipython#4285

ivanov · 2013-10-14T17:54:01Z

@pablooliveira I'm also seeing these messages, trying to track down what's causing them, but you're definitely not alone

ivanov · 2013-10-19T06:23:17Z

ok, I've added some more tests here, including testing of interrupts via "mouse click" and via keyboard shortcuts. This took a while, since the combination of JavaScript, JQuery, CasperJS and phantomJS is a bit tricky to figure out who should be doing what... The uncaught exception stuff that @pablooliveira and I were seeing earlier are also gone now, and notebook kernels shutdown properly at the end of the tests.

sychan · 2013-10-22T19:28:57Z

Paul - do you have any sense of whether it would be workable to backport the diffs to the 1.x branch? I tried a test merge of the 1.x branch against this branch and of course there were deltas all over the place. When we tried to run our notebook against IPython master branch the other week all kinds of things were unhappy so I don't think we can use the master branch as-is.

takluyver · 2013-10-22T19:52:38Z

I'm pretty sure this is not going to be backported to 1.x - that branch is only getting bugfixes.

sychan · 2013-10-22T19:59:00Z

I'm willing to do the work myself, if Paul thinks that it is not too onerous. Our project needs front end testing over the next couple of months - but if it makes more sense to just code up a separate, lightweight CasperJS based test setup until until 2.0 gets into a release candidate form, I can go that route as well.

sychan · 2013-10-22T20:43:26Z

I tried the quick and dirty test of just running the casperjs tests directly against a 1.x server, and they seem to function standalone (though about half of the tests seem to fail - but I can work with that), so I guess that seems adequate for now.

ivanov · 2013-10-23T00:02:35Z

@sychan yeah, it shouldn't be too bad to get this working against 1.x, Steve. After all, this is a continuation of work that was sitting in a PR pre 1.0 (7 months ago).

takluyver · 2013-10-23T17:58:38Z

Merging this now, people can keep adding tests in new branches.

takluyver · 2013-10-23T18:00:01Z

(actually, it needs a rebase)

Markdown cell execution. Code cell test is based on https://gist.github.com/ellisonbg/3329929.

pressed in an empty notebook.

this line causes noise in the test suite, but if we just ignore it, we'll never get to the bottom of it. It seems to only happen when running 'iptest js', and *not* when running the 'casperjs test' command directly, with a notebookserver that was launched manually.

the PPA we use on Travis CI doesn't have CasperJS 1.1.0-DEV yet, so we're better off not using it for now.

Notebook javascript test suite using CasperJS

…so that they would run with the current 1.x branch. Currently passes the basic tests

@ellisonbg

as suggested by @ellisonbg in ipython#4285

Notebook javascript test suite using CasperJS

ellisonbg reviewed Sep 26, 2013
View reviewed changes

minrk mentioned this pull request Sep 27, 2013

Notebook CasperJS tests #3125

Closed

takluyver reviewed Oct 4, 2013
View reviewed changes

ivanov added a commit to ivanov/ipython that referenced this pull request Oct 9, 2013

added a shorthand way to trigger keyboard events

6851a59

as suggested by @ellisonbg in ipython#4285

minrk mentioned this pull request Oct 22, 2013

Client side tests for the notebook #2292

Closed

dwyde added 3 commits October 23, 2013 11:03

Add CasperJS utility functions, and tests for code and

18def9d

Markdown cell execution. Code cell test is based on https://gist.github.com/ellisonbg/3329929.

Add a README for CasperJS tests.

416eb44

Add a test to demonstrate errors when up/down arrow keys are

e464bd8

pressed in an empty notebook.

ivanov added 25 commits October 23, 2013 11:05

use dashboard to simulate clicking new notebook

0d48d40

don't shutdown kernel after every test

09f8021

XXX: speedup testing only javascript on travis

0347a4e

make tests work on TravisCI

3d21396

test kernel interrupt

21810de

make sure kernel started running

dccf954

run all travis tests

e2dc90d

repeat test with keyboard shortcut

e737024

utility functions + tests for shift- & ctrl-enter

b4e4c46

wait_for_output utility function

04e06a5

use wait_for_output util function

1770835

add Ctrl-M keyboard shortcut

cdbacdc

test keyboard interrupt shortcuts

532de54

minor shortening

877c22b

refactor of get_output_cell

b406226

add checks for new cell added using shift-enter

b6bc351

adds test of clicking the play button

b4a7925

created a kernel_running method

780043a

add test for shutdown notebook

d18e40b

use test.begin to give test group a name

3bae281

make js tests quieter, silence notebook server

ee550cb

don't use test.begin

b906c2c

the PPA we use on Travis CI doesn't have CasperJS 1.1.0-DEV yet, so we're better off not using it for now.

fix: JS kernel state should reflect killed state

a7e71ea

give travis 10 seconds to start server

91bba24

takluyver added a commit that referenced this pull request Oct 23, 2013

Merge pull request #4285 from ivanov/notebook-casperjs-tests

459371d

Notebook javascript test suite using CasperJS

takluyver merged commit 459371d into ipython:master Oct 23, 2013

brettin pushed a commit to kbase/narrative that referenced this pull request Jun 12, 2014

NAR-35 #resolve Tweaked the casperjs tests from ipython/ipython#4285 …

8e4d1d6

…so that they would run with the current 1.x branch. Currently passes the basic tests

mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this pull request Nov 3, 2014

added a shorthand way to trigger keyboard events

73e0e41

as suggested by @ellisonbg in ipython#4285

mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this pull request Nov 3, 2014

Merge pull request ipython#4285 from ivanov/notebook-casperjs-tests

f0d8fa4

Notebook javascript test suite using CasperJS

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Notebook javascript test suite using CasperJS #4285

Notebook javascript test suite using CasperJS #4285

ivanov commented Sep 26, 2013

minrk commented Sep 26, 2013

ellisonbg commented Sep 26, 2013

ivanov commented Sep 26, 2013

ellisonbg commented Sep 26, 2013

ellisonbg Sep 26, 2013

ivanov Sep 26, 2013

ellisonbg commented Sep 26, 2013

ivanov commented Sep 26, 2013

dwyde commented Sep 29, 2013

takluyver Oct 4, 2013

pablooliveira commented Oct 9, 2013

ivanov commented Oct 14, 2013

ivanov commented Oct 19, 2013

sychan commented Oct 22, 2013

takluyver commented Oct 22, 2013

sychan commented Oct 22, 2013

sychan commented Oct 22, 2013

ivanov commented Oct 23, 2013

takluyver commented Oct 23, 2013

takluyver commented Oct 23, 2013

Notebook javascript test suite using CasperJS #4285

Notebook javascript test suite using CasperJS #4285

Conversation

ivanov commented Sep 26, 2013

minrk commented Sep 26, 2013

ellisonbg commented Sep 26, 2013

ivanov commented Sep 26, 2013

ellisonbg commented Sep 26, 2013

ellisonbg Sep 26, 2013

Choose a reason for hiding this comment

ivanov Sep 26, 2013

Choose a reason for hiding this comment

ellisonbg commented Sep 26, 2013

ivanov commented Sep 26, 2013

dwyde commented Sep 29, 2013

takluyver Oct 4, 2013

Choose a reason for hiding this comment

pablooliveira commented Oct 9, 2013

ivanov commented Oct 14, 2013

ivanov commented Oct 19, 2013

sychan commented Oct 22, 2013

takluyver commented Oct 22, 2013

sychan commented Oct 22, 2013

sychan commented Oct 22, 2013

ivanov commented Oct 23, 2013

takluyver commented Oct 23, 2013

takluyver commented Oct 23, 2013