Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Selenium utils + markdown rendering tests #3458

Merged
merged 40 commits into from Mar 28, 2018

Conversation

Projects
None yet
4 participants
@mpacer
Copy link
Member

mpacer commented Mar 22, 2018

This PR introduces a Notebook class as a helper for running selenium functions. It still needs work to document it more completely, but… it works and makes it a lot easier to create the relevant tests inside notebooks.

I'm interested to hear people's reviews.

I think @takluyver @Carreau @minrk and @rgbkrk would be particularly interested in this.

@mpacer mpacer force-pushed the mpacer:selenium_utils branch from 51c2d27 to 1889e56 Mar 22, 2018

@rgbkrk

rgbkrk approved these changes Mar 22, 2018

Copy link
Member

rgbkrk left a comment

Awesome

@rgbkrk

This comment has been minimized.

Copy link
Member

rgbkrk commented Mar 22, 2018

I'm happy these at least make it easy for Python people to write tests for the JS. 😅

@mpacer mpacer force-pushed the mpacer:selenium_utils branch from 1889e56 to 715d9e3 Mar 22, 2018

@mpacer mpacer force-pushed the mpacer:selenium_utils branch from 715d9e3 to ebef7ba Mar 23, 2018

@mpacer

This comment has been minimized.

Copy link
Member Author

mpacer commented Mar 23, 2018

This is now rebased on #3412 to take into account the issues that arise when you create a new notebook in the first place.

except IndexError as e:
e.message = 'You need a server running before you can run this command'
driver = Firefox()
auth_url = f'{server["url"]}?token={server["token"]}'

This comment has been minimized.

@takluyver

takluyver Mar 23, 2018

Member

f strings are great, but we're still supporting Python 3.4 and 3.5 for now, so we'll need to stick with the explicit .format() method for now.

This comment has been minimized.

@mpacer

mpacer Mar 23, 2018

Author Member

@takluyver
Copy link
Member

takluyver left a comment

Looks good, just a few points about clarity.

"""Gets all cells once they are visible.
"""
wait_for_selector(self.browser, ".cell")

This comment has been minimized.

@takluyver

takluyver Mar 23, 2018

Member

I don't like having a wait inside a property - the rule of thumb for properties is that anything they need to do should be near-instant. If it needs to do something that's not, let's make it a regular method - the call gives people reading the code a hint that there's something going on underneath.

This comment has been minimized.

@mpacer

mpacer Mar 23, 2018

Author Member

I was wrong in my belief that it was needed here. One of the reasons I really didn't like it was that it couldn't handle notebooks with no cells; so this always seemed wrong. So it's gone!


@contextmanager
def new_window(browser, selector=None):
"""Creates new window, switches you to that window, waits for selector if set.

This comment has been minimized.

@takluyver

takluyver Mar 23, 2018

Member

Neat! The docstring needs some work, though - this doesn't create a new window, rather it expects that code called in its context will create a new window, and then it switches to that on leaving the context.

This is worth explaining clearly, because most context managers do the interesting stuff when you create/enter them, and exiting just cleans something up. Here the interesting stuff is on exit.

This comment has been minimized.

@mpacer

mpacer Mar 23, 2018

Author Member

I think I took care of this one.

# initial_window_handles = browser.window_handles
with new_window(browser, selector=".cell"):
select_kernel(browser, kernel_name=kernel_name)
browser.execute_script("Jupyter.notebook.set_autosave_interval(0)")

This comment has been minimized.

@takluyver

takluyver Mar 23, 2018

Member

Is it OK that this is in new_notebook() but remove_safety_check() is called from __init__? I'd expect they'd go together.

This comment has been minimized.

@mpacer

mpacer Mar 23, 2018

Author Member

yea… I think you're right on that account.



@classmethod
def new_notebook(cls, browser, kernel_name='kernel-python3'):

This comment has been minimized.

@takluyver

takluyver Mar 23, 2018

Member

Docstring! It should mention that this expects to be called with the browser on the dashboard page. You could use your other utilities to verify that and throw an error if it's not.

def current_cell_index(self):
return self.cells.index(self.current_cell)

def remove_safety_check(self):

This comment has been minimized.

@takluyver

takluyver Mar 23, 2018

Member

'safety' is rather vague. remove_unsaved_check, maybe? Or base it on the hook name: remove_onbeforeunload.

if visible:
conditional = EC.visibility_of_all_elements_located
else:
conditional = EC.presence_of_all_elements_located

This comment has been minimized.

@takluyver

takluyver Mar 23, 2018

Member

This is steadily heading towards replicating the complexity of the underlying Selenium API. It's fine for now, but we should keep in mind that it may be better to use the underlying APIs directly in some cases.

This comment has been minimized.

@mpacer

mpacer Mar 23, 2018

Author Member

Agreed — however, I think this covers all the cases where we wanted it and it hides a tonne of the selenium boilerplate so I think it's still ok.

@mpacer

This comment has been minimized.

Copy link
Member Author

mpacer commented Mar 23, 2018

I think this now stands on its own. It's not fully handling everything that the markdown tests are doing, but it handles all the basic tests of our markdown rendering.

The mathjax rendering is a different story — however, I think we might be able to test that even more directly in pure javascript code as it's not really about what's rendered on the page but is entirely about what is parsed on the javascript side.

@mpacer mpacer force-pushed the mpacer:selenium_utils branch from 64a2619 to 74af79c Mar 23, 2018

@mpacer mpacer changed the title Selenium utils + 1 markdown test Selenium utils + markdown tests Mar 23, 2018

@mpacer mpacer changed the title Selenium utils + markdown tests Selenium utils + markdown rendering tests Mar 23, 2018

"""Contextmanager for switching to & waiting for a window created.
This context manager gives you the ability to create a new window inside
the created context and it will switch you to that new window.abs

This comment has been minimized.

@takluyver
def index(self, cell):
return self.cells.index(cell)

def remove_safety_check(self):

This comment has been minimized.

@takluyver

takluyver Mar 25, 2018

Member

Let's find a more representative name for this.

'<pre><code class="cm-s-ipython language-aaaa">x = 1\n</code></pre>'
]
for i, cell in enumerate(nb):
nb.append(*cell_text, cell_type="markdown")

This comment has been minimized.

@takluyver

takluyver Mar 25, 2018

Member

I don't think this needs to be in a loop now - the .append() call adds all the cells in one go.

@@ -0,0 +1,43 @@
from selenium.webdriver import Firefox

This comment has been minimized.

@takluyver

takluyver Mar 25, 2018

Member

Let's have a module level docstring to explain that these utilities are not used by the tests, but they're intended for interactive exploration while writing tests.

This comment has been minimized.

@mpacer

mpacer Mar 30, 2018

Author Member

I had intended to add this.

@takluyver

This comment has been minimized.

Copy link
Member

takluyver commented Mar 27, 2018

The last few things on this are all docstring and naming points. If you've got other things to do, let me know and I can tidy them up.

@takluyver takluyver merged commit faa0cab into jupyter:master Mar 28, 2018

4 checks passed

codecov/patch 0% of diff hit (target 0%)
Details
codecov/project 76.07% (-1.51%) compared to dc1eeab
Details
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@takluyver takluyver added this to the 5.5 milestone Mar 28, 2018

@takluyver

This comment has been minimized.

Copy link
Member

takluyver commented Mar 28, 2018

@mpacer sorry to take over the PR, but I wanted to get it merged in quickly.

@rgbkrk

This comment has been minimized.

Copy link
Member

rgbkrk commented Mar 28, 2018

Thanks @mpacer and @takluyver!

self.add_cell(index, cell_type="markdown")
self.edit_cell(index=index, content=content, render=render)

def append(self, *values, cell_type="code"):

This comment has been minimized.

@hroncok

hroncok May 11, 2018

Contributor

This is invalid syntax on Python 2. Are those files intended to be supported on all the Pythons as notebook itself?

This comment has been minimized.

@rgbkrk

rgbkrk May 11, 2018

Member

Notebook development is Python3 only now. Your runtime / kernel will always be allowed to be in Python2.

This comment has been minimized.

@hroncok

hroncok May 11, 2018

Contributor

I'm planning to get python2-notebook out of Fedora, however I was wondering what's the upstream status on this. The docs say Python 2.7 is supported. And except the selenium tests it seems to be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.