canonicalize_url: add to w3lib.url.__all__ + doctest

[redapple]

No description provided.

[codecov-io]

Current coverage is 94.10% (diff: 100%)

Merging #67 into master will increase coverage by 0.01%

Powered by Codecov. Last update 3c56191...aadc0de

[kmike]

Thanks Paul 👍
Is that correct that we don't actually execute doctests for w3lib?

[redapple]

@kmike , it looks like we do not, indeed.

[redapple]

@kmike , I can't find a way to run them with Python 3 and py.test.
Any idea?

This works with Python 2.7 (140 tests vs. 121):

diff --git a/tox.ini b/tox.ini
index 5fb4f78..4ef28b1 100644
--- a/tox.ini
+++ b/tox.ini
@@ -11,4 +11,7 @@ deps =
     pytest
     pytest-cov
 commands =
-    py.test --cov=w3lib --cov-report= {posargs:w3lib tests}
+    py.test \
+        --doctest-modules --doctest-ignore-import-errors \
+        --cov=w3lib --cov-report=term \
+        {posargs:w3lib tests}

Tox output:

$ tox -e py27
GLOB sdist-make: /home/paul/src/w3lib/setup.py
py27 inst-nodeps: /home/paul/src/w3lib/.tox/dist/w3lib-1.14.3.zip
py27 installed: coverage==4.1,pkg-resources==0.0.0,py==1.4.31,pytest==2.9.2,pytest-cov==2.2.1,six==1.10.0,w3lib==1.14.3
py27 runtests: PYTHONHASHSEED='599567335'
py27 runtests: commands[0] | py.test --doctest-modules --doctest-ignore-import-errors --cov=w3lib --cov-report=term w3lib tests
===================================================================================== test session starts ======================================================================================
platform linux2 -- Python 2.7.11+, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /home/paul/src/w3lib, inifile: 
plugins: cov-2.2.1
collected 140 items 

w3lib/encoding.py .....
w3lib/form.py .
w3lib/html.py .....
w3lib/http.py ...
w3lib/url.py .....
tests/test_encoding.py ..................
tests/test_form.py ...
tests/test_html.py ....................................................
tests/test_http.py ......
tests/test_url.py ..........................................
----------------------------------------------------------------------- coverage: platform linux2, python 2.7.11-final-0 -----------------------------------------------------------------------
Name                Stmts   Miss Branch BrPart  Cover
-----------------------------------------------------
w3lib/__init__.py       2      0      2      0   100%
w3lib/encoding.py      68      0     34      0   100%
w3lib/form.py          28      1     10      2    92%
w3lib/html.py         110      6     34      2    94%
w3lib/http.py          23      1     14      1    95%
w3lib/url.py          142     26     44      5    78%
w3lib/util.py          33      8     22      4    71%
-----------------------------------------------------
TOTAL                 406     42    160     14    88%

================================================================================== 140 passed in 0.40 seconds ==================================================================================
___________________________________________________________________________________________ summary ____________________________________________________________________________________________
  py27: commands succeeded
  congratulations :)

But with Python 3, still only the 121 tests run:

$ tox -e py35
GLOB sdist-make: /home/paul/src/w3lib/setup.py
py35 recreate: /home/paul/src/w3lib/.tox/py35
py35 installdeps: pytest, pytest-cov
py35 inst: /home/paul/src/w3lib/.tox/dist/w3lib-1.14.3.zip
py35 installed: coverage==4.2,pkg-resources==0.0.0,py==1.4.31,pytest==2.9.2,pytest-cov==2.3.0,six==1.10.0,w3lib==1.14.3
py35 runtests: PYTHONHASHSEED='1309043954'
py35 runtests: commands[0] | py.test --doctest-modules --doctest-ignore-import-errors --cov=w3lib --cov-report=term w3lib tests
===================================================================================== test session starts ======================================================================================
platform linux -- Python 3.5.2, pytest-2.9.2, py-1.4.31, pluggy-0.3.1
rootdir: /home/paul/src/w3lib, inifile: 
plugins: cov-2.3.0
collected 121 items 

tests/test_encoding.py ..................
tests/test_form.py ...
tests/test_html.py ....................................................
tests/test_http.py ......
tests/test_url.py ..........................................

----------- coverage: platform linux, python 3.5.2-final-0 -----------
Name                Stmts   Miss Branch BrPart  Cover
-----------------------------------------------------
w3lib/__init__.py       2      0      2      0   100%
w3lib/encoding.py      68      0     34      0   100%
w3lib/form.py          28      1     10      2    92%
w3lib/html.py         110      7     36      2    94%
w3lib/http.py          23      0     18      1    98%
w3lib/url.py          142      5     46      6    94%
w3lib/util.py          33      9     22      5    67%
-----------------------------------------------------
TOTAL                 406     22    168     16    93%


================================================================================== 121 passed in 0.67 seconds ==================================================================================
___________________________________________________________________________________________ summary ____________________________________________________________________________________________
  py35: commands succeeded
  congratulations :)

[redapple]

ok... this is due to conftest.py excluding w3lib modules in Python3