Using Spynner you would able to simulate a web browser with no GUI (though a browsing window can be opened for debugging purposes), so it may be used to implement crawlers or acceptance testing tools.
- Leo Lou <https://github.com/l4u>
Throught regular easy_install / buildout:
The bleeding edge version is hosted on github:
git clone https://github.com/makinacorpus/spynner.git cd spynner python setup.py install
You can generate the API locally (will create docs/api directory):
python setup.py gen_doc
A basic example:
Sometimes you'll want to see what is going on:
browser = spynner.Browser() browser.debug_level = spynner.DEBUG browser.create_webview() browser.show()
See more examples in the repository: https://github.com/kiorky/spynner/tree/master/examples
- See the implementation docstrings or examples !
- You have three levels of control:
- webkit methods which are recommended to us (wk_fill_*, wk_click_*) which are jquery based
- classical methods (fill, click_*) which are jquery based
- low level using QT raw events which are not that well working ATM. At least, you can move the mouse
- JQuery core Amongst other things, it adds the powerful JQuery selectors, which are used internally by some Spynner methods. Of course you can also use jQuery when you inject your own code into a page.
- Simulate jQuery plugin: Makes it possible to simulate mouse and keyboard events (for now spynner uses it only in the _click_ action). Look up the library code to see which kind of events you can fire.
Note that you must use __jQuery(...)_ instead of _jQuery(...)_ or the common shortcut _$(...)_. That prevents name clashing with the jQuery library used by the page.
import spynner import pyquery browser = spynner.Browser() ... d = pyquery.Pyquery(browser.html) d.make_links_absolute(browser.get_url()) href = d("#somelink").attr("href") browser.download(href, open("/path/outputfile", "w"))
Spynner needs a X11 server to run. If you are running it in a server without X11 you must install the virtual Xvfb server. Debian users can use the small wrapper (xvfb-run). If you are not using Debian, you can download it here: http://email@example.com/msg69632/x-run
xvfb-run python myscript_using_spynner.py
You can also use tightvnc.