Skip to content

Commit

Permalink
update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
jadbin committed Oct 24, 2017
1 parent 9c26c5a commit 823cb28
Show file tree
Hide file tree
Showing 5 changed files with 20 additions and 17 deletions.
5 changes: 0 additions & 5 deletions docs/_static/docs.css

This file was deleted.

8 changes: 4 additions & 4 deletions docs/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,18 @@ Change log
New features
~~~~~~~~~~~~

- 使用继承Dupefilter的去重过滤器来实现去重功能,系统配置新增``dupefilter_cls``项,用于替换默认的去重过滤器
- ``xpaw.utils.run``模块中新增``run_crawler``函数,便于在python代码中控制开启爬虫
- 使用继承Dupefilter的去重过滤器来实现去重功能,系统配置新增 ``dupefilter_cls`` 项,用于替换默认的去重过滤器
- ``xpaw.utils.run`` 模块中新增 ``run_crawler`` 函数,便于在python代码中控制开启爬虫

Update
~~~~~~

- 使用config.py替代config.yaml作为配置文件,移除对pyyaml的依赖
- ForwardedForMiddleware移动到``xpaw.downloadermws.headers``模块下
- ForwardedForMiddleware移动到 ``xpaw.downloadermws.headers`` 模块下
- 修改aiohttp的版本限制到>=2.2.0
- 更新了中间件的错误处理机制
- 不再采用中间件的形式实现请求的去重功能,并移除相关的中间件
- ProxyAgentMiddleware的``proxy_agnet``配置下面``addr``字段更名为``agent_addr``
- ProxyAgentMiddleware的 ``proxy_agnet`` 配置下面 ``addr`` 字段更名为 ``agent_addr``


0.6.5 (2017-05-09)
Expand Down
13 changes: 10 additions & 3 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -95,14 +95,21 @@
# documentation.
#
html_theme_options = {
'font_family': '"Helvetica Neue", Helvetica, "PingFang SC", "Hiragino Sans GB", "Microsoft YaHei", "微软雅黑", Arial, sans-serif;',
'font_size': '14px'
'font_family': "'goudy old style', 'minion pro', 'bell mt', Georgia, 'Hiragino Mincho Pro', 'Hiragino Sans GB', 'PingFang SC', 'Microsoft YaHei', '微软雅黑', serif",
'font_size': '16px'
}

# Custom sidebar templates, maps document names to template names.
html_sidebars = {
'**': [
'about.html', 'navigation.html', 'searchbox.html',
]
}

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# html_static_path = ['_static']


# -- Options for HTMLHelp output ------------------------------------------
Expand Down
7 changes: 4 additions & 3 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,11 @@
xpaw |version| documentation
============================

Overview
========
Key Features
============

xpaw is a web scraping framework, used to crawl web pages and extract structured data from them.
- Provides a web scraping framework used to crawl web pages.
- Provides data extraction tools used to extract structured data from web pages.

Getting started
===============
Expand Down
4 changes: 2 additions & 2 deletions docs/intro/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@
Installation guide
==================

Installing xpaw
===============
Installation
============

安装xpaw需要Python 3.5及更高版本的环境支持。

Expand Down

0 comments on commit 823cb28

Please sign in to comment.