New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add huge_tree support to HTMLParser. #261

Merged
merged 2 commits into from Mar 8, 2018

Conversation

Projects
None yet
2 participants
@stranac
Copy link
Contributor

stranac commented Mar 7, 2018

HTMLParser currently has no support for the huge_tree argument, which makes it impossible to correctly parse html documents with large text nodes.
This has been reported as an issue for Scrapy (scrapy/scrapy#3077), and I was able to trace it back to lxml.

@stranac

This comment has been minimized.

Copy link
Contributor

stranac commented Mar 7, 2018

The argument is already documented in http://lxml.de/parsing.html#parser-options, so I don't think documentation changes are necessary, other than the already-changed docstring.

There also seem to be no tests for the huge_tree option of the XMLParser, so I haven't added any for this (not that I can see a reasonable way to test this).

@scoder

This comment has been minimized.

Copy link
Member

scoder commented Mar 8, 2018

Thanks!

@scoder scoder merged commit 3fac834 into lxml:master Mar 8, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment