Skip to content

Commit

Permalink
Add examples
Browse files Browse the repository at this point in the history
  • Loading branch information
Šarūnas Navickas committed Sep 7, 2016
1 parent a8277bb commit e8354eb
Showing 1 changed file with 35 additions and 1 deletion.
36 changes: 35 additions & 1 deletion docs/config.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@

Configuration
=============

Expand All @@ -20,9 +19,31 @@ parser (Optional) Select parser for web pages.
auth (Optional) `Auth` object
============ ================= ======================================

.. code-block:: yaml
name: Python Org scrapper
description: Just scrape it for testing
site_root: https://www.python.org
start_page: /blogs
cookies:
authToken: abc1234
remember: true
headers:
"User-Agent": "Mozilla/5"
auth: ...
pages:
- ...
Auth
----
.. code-block:: yaml
url: /login
method: POST
params:
user: {{ secret_user }}
password: {{ secret_password }}
============ ================ ======================================
Value Notes Description
Expand All @@ -43,6 +64,13 @@ link_pattern Pattern which allows to detect which pag
mappings (Array) Array of `Mapping`
============ ================ ======================================

.. code-block:: yaml
- name: Blog
link_pattern: /blog%
mappings:
- ...
Mapping
-------

Expand All @@ -52,3 +80,9 @@ Value Notes Description
name Name of mapping
path Path to element
============ ================ ======================================


.. code-block:: yaml
- name: title
path: h3[class="event-title"]/a

0 comments on commit e8354eb

Please sign in to comment.