Skip to content
A Ruia plugin for loading javascript - pyppeteer
Branch: master
Clone or download
Latest commit 8f5456d Feb 14, 2019
Type Name Latest commit message Commit time
Failed to load latest commit information.
example v0.0.4 Feb 14, 2019
ruia_pyppeteer v0.0.4 Feb 14, 2019
.gitignore Init Nov 22, 2018
Pipfile v0.0.4 Feb 14, 2019 v0.0.4 Feb 14, 2019


A Ruia plugin for loading javascript

Notice: Works on ruia >= 0.4.9


pip install ruia_pyppeteer
# New features
pip install git+


ruia_pyppeteer will load js by using pyppeteer.

You need to pay attention when you use load_js, it will download a recent version of Chromium (~100MB). This only happens once.

Load JavaScript

import asyncio

from ruia_pyppeteer import PyppeteerRequest as Request

request = Request("", load_js=True)
response = asyncio.get_event_loop().run_until_complete(request.fetch())

Complete example

from ruia import AttrField, TextField, Item

from ruia_pyppeteer import PyppeteerSpider as Spider

class JianshuItem(Item):
    target_item = TextField(css_select='ul.list>li')
    author_name = TextField(css_select='')
    author_url = AttrField(attr='href', css_select='')

    async def clean_author_url(self, author_url):
        return f"{author_url}"

class JianshuSpider(Spider):
    start_urls = ['']
    concurrency = 10

    async def parse(self, response):
        async for item in JianshuItem.get_items(html=response.html):
            # Loading js by using PyppeteerRequest

if __name__ == '__main__':

Enjoy it :)

You can’t perform that action at this time.