Skip to content

better Python 3.7 support #97

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 2, 2018
Merged

better Python 3.7 support #97

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ matrix:
env: TOXENV=py35
- python: 3.6
env: TOXENV=py36
- python: 3.7
env: TOXENV=py37
dist: xenial
sudo: true

# command to install dependencies
install:
Expand Down
2 changes: 1 addition & 1 deletion extruct/_extruct.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ def extract(htmlstring, base_url=None, encoding="UTF-8",
schema_context: schema's context for current page"""
if base_url is None and 'url' in kwargs:
warnings.warn('"url" argument is deprecated, please use "base_url"',
DeprecationWarning)
DeprecationWarning, stacklevel=2)
base_url = kwargs.pop('url')
if kwargs:
raise TypeError('Unexpected keyword arguments')
Expand Down
2 changes: 1 addition & 1 deletion extruct/jsonld.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

from extruct.utils import parse_html

HTML_OR_JS_COMMENTLINE = re.compile('^\s*(//.*|<!--.*-->)')
HTML_OR_JS_COMMENTLINE = re.compile(r'^\s*(//.*|<!--.*-->)')


class JsonLdExtractor(object):
Expand Down
4 changes: 2 additions & 2 deletions extruct/tool.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ def metadata_from_url(url, syntaxes=SYNTAXES, uniform=False,
resp.raise_for_status()
except requests.exceptions.HTTPError:
return result
result.update(extruct.extract(resp.content,
url=url,
result.update(extruct.extract(resp.content,
base_url=url, # FIXME: use base url
Copy link
Member

@lopuhin lopuhin Nov 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more reason to deprecate the tool - I was going to suggest using get_base_url from w3lib, but it seems this would require more extra code to handle encodings correctly (and we are already throwing encoding information away here by not using headers).

Copy link
Member Author

@kmike kmike Nov 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, actually I started adding get_base_url, then realized encodings need to be handled, etc., removed that, and just added FIXME comment instead :)

syntaxes=syntaxes,
uniform=uniform,
schema_context=schema_context,
Expand Down
1 change: 1 addition & 0 deletions setup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,5 +57,6 @@ def get_version():
'Programming Language :: Python :: 3.4',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
],
)
7 changes: 5 additions & 2 deletions tests/test_extruct.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
import json
import unittest

import pytest

import extruct
from tests import get_testdata, jsonize_dict, replace_node_ref_with_node_id

Expand Down Expand Up @@ -31,8 +33,9 @@ def test_microdata_with_returning_node(self):

def test_deprecated_url(self):
body, expected = self._microdata_custom_url('product_custom_url.json')
data = extruct.extract(body, url='http://some-example.com',
syntaxes=['microdata'])
with pytest.warns(DeprecationWarning):
data = extruct.extract(body, url='http://some-example.com',
syntaxes=['microdata'])
self.assertEqual(data, expected)

def test_extra_kwargs(self):
Expand Down
2 changes: 1 addition & 1 deletion tox.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[tox]
envlist = py27, py34, py35, py36
envlist = py27, py34, py35, py36, py37

[testenv]
deps =
Expand Down