Skip to content
Switch branches/tags
Go to file
Was occuring due to casting an EOF token to a string (from #540).
3 contributors

Users who have contributed to this file

@jhy @krystiangorecki @exviva
jsoup changelog
*** Release 1.14.1 [PENDING]
* Change: updated the minimum supported Java version from Java 7 to Java 8.
* Change: updated the minimum Android API level from 8 to 10.
* Change: although Node#childNodes() returns an UnmodifiableList as a view into its children, it was still
directly backed by the internal child list. That made some uses, such as looping and moving those children to
another element, throw a ConcurrentModificationException. Now this method returns its own list so that they are
separated and changes to the parent's contents will not impact the children view. This aligns with similar methods
such as Element#children(). If you have code that iterates this list and makes parenting changes to its contents,
you may need to make a code update.
* Improvement: renamed the Whitelist class to Safelist, with the goal of more inclusive language. A shim is provided
for backwards compatibility (source and binary). This shim is marked as deprecated and will be removed in the
jsoup 1.15.1 release.
* Improvement: added support for Internationalized Domain Names (IDNs) in Jsoup.Connect.
* Improvement: added support for loading and parsing gzipped HTML files in Jsoup.parse(File in, charset, baseUri).
* Improvement: reduced thread contention in HttpConnection and Document.
* Improvement: better parsing performance when under high thread concurrency
* Improvement: added Element#id(String) ID attribute setter.
* Improvement: in Document, #body() and #head() accessors will now automatically create those elements, if they were
missing (e.g. if the Document was not parsed from HTML). Additionally, the #body() method returns the frameset
element (instead of null) for frameset documents.
* Improvement: when cleaning a document, the output settings of the original document are cloned into the cleaned
* Improvement: when parsing XML, disable pretty-printing by default.
* Improvement: much better performance in Node#clone() for large and deeply nested documents. Complexity was O(n^2) or
worse, now O(n).
* Improvement: during traversal using the NodeTraversor, nodes may now be replaced with Node#replaceWith(Node).
* Improvement: added Element#insertChildren and Elment#prependChildren, as convenience methods in addition to
Element#insertChildren(index, children), for bulk moving nodes.
* Build Improvement: moved to GitHub Workflows for build verification.
* Build Improvement: updated Jetty (used for integration tests; not bundled) to 9.4.35.v2020112.
* Build Improvement: added nullability annotations and initial settings.
* Bugfix: when parsing HTML, could throw NPEs on some tags (isindex or table>input).
* Bugfix: in HttpConnection.Request, headers beginning with "sec-" (e.g. Sec-Fetch-Mode) were silently discarded by
the underlying Java HttpURLConnection. These are now settable correctly.
* Bugfix: when adding child Nodes to a Node, could incorrectly reparent all nodes if the first parent had the same
length of children as the incoming node list.
* Bugfix: when wrapping an orphaned element, would throw an NPE.
* Bugfix: when wrapping an element with HTML that included multiple sibling elements, those siblings were incorrectly
added as children of the wrapper instead of siblings.
* Bugfix: when setting the content of a script or style tag via the Element#html(String) method, the content is now
treated as a DataNode, not a TextNode. This means that characters like '<' will no longer be incorrectly escaped.
As a related ergonomic improvement, the same behavior applies for Element#text(String) (i.e. the content will be
treated as a DataNode, despite calling the text() method.
* Bugfix: when wrapping HTML around an existing element with Element#wrap(String), will now take the content as
provided and ignore normal HTML tree-building rules. This allows for e.g. a div tag to be placed inside of p tags.
* Bugfix: the Elements#forms() method should return the selected immediate elements that are Forms, not children.
* Bugfix: when creating a selector for an element with Element#cssSelector, if the element used a non-unique ID
attribute, the returned selector may not match the desired element.
* Bugfix: corrected the toString() methods of the Evaluator classes.
* Bugfix: when converting a jsoup document to a W3C document (in W3CDom#convert), if a tag had XML illegal characters,
a DOMException would be thown. Now instead, that tag is represented as a text node.
* Bugfix: if a HTML file ended with an open noscript tag, an "EOF" string would appear in the HTML output.
*** Release 1.13.1 [2020-Feb-29]
* Improvement: added Element#closest(selector), which walks up the tree to find the nearest element matching the
* Improvement: memory optimizations, reducing the retained size of a Document by ~ 39%, and allocations by ~ 9%:
1. Attributes holder in Elements is only created if the element has attributes
2. Only track the baseUri in an element when it is set via DOM to a new value for a given tree
3. After parsing, do not retain the input character reader (and associated buffers) in the Document#parser
* Improvement: substantial parse speed improvements vs 1.12.x (bringing back to par with previous releases).
* Improvement: when pretty-printing, comments in inline tags are not pushed to a newline
* Improvement: added Attributes#hasDeclaredValueForKey(key) and Attribute#hasDeclaredValueForKeyIgnoreCase(), to check
if an attribute is set but has no value. Useful in place of the deprecated and removed BooleanAttribute class and
instanceof test.
* Improvement: removed old methods and classes that were marked deprecated in previous releases.
* Improvement: added Element#select(Evaluator) and Element#selectFirst(Evaluator), to allow re-use of a parsed CSS
selector if using the same evaluator many times.
* Improvement: added Elements#forms(), Elements#textNodes(), Elements#dataNodes(), and Elements#comments(), as a
convenient way to get access to these node types directly from an element selection.
* Improvement: preserve whitespace before html and head tag, if pretty-printing is off.
* Bugfix: in a <select> tag, a second <optgroup> would not automatically close an earlier open <optgroup>
* Bugfix: in CharacterReader when parsing an input stream, could throw a Mark Invalid exception if the reader was
marked, a bufferUp occurred, and then the reader was rewound.
* Bugfix: empty tags and form tags did not have their attributes normalized (lower-cased by default)
* Bugfix: when preserve case was set to on, the HTML pretty-print formatter didn't indent capitalized tags correctly.
* Bugfix: ensure that script and style contents are parsed into DataNodes, not TextNodes, when in case-sensitive
parse mode.
**** Release 1.12.2 [2020-Feb-08]
* Improvement: the :has() selector now supports relative selectors. For example, the query
"div:has(> a)" will select all "div" elements that have at least one direct child "a" element.
* Improvement: added Element chaining methods for various overridden methods on Node.
* Improvement: ensure HTTP keepalives work when fetching content via body() and bodyAsBytes().
* Improvement: set the default max body size in Jsoup.Connection to 2MB (up from 1MB) so fewer people get trimmed
content if they have not set it, but still in sensible bounds. Also updated the default user-agent to improve
default compatibility.
* Improvement: dramatic speed improvement when bulk inserting child nodes into an element (wrapping contents).
* Improvement: added Element#childrenSize() as a convenience to get the size of an element's element children.
* Improvement: in W3CDom.asString, allow the output mode to be specified as HTML or as XML. It will default to
checking the content, and automatically selecting.
* Improvement: added a Document#documentType() method, to get a doc's doctype.
* Improvement: To DocumentType, added #name(), #publicID(), and #systemId() methods to fetch those fields.
* Improvement: in W3CDom conversions from jsoup documents, retain the DocumentType, and be able to serialize it.
* Bugfix: on pages fetch by Jsoup.Connection, a "Mark Invalid" exception might be incorrectly thrown, or the page may
miss some data. This occurred on larger pages when the file transfer was chunked, and an invalid HTML entity
happened to cross a chunk boundary.
* Bugfix: if duplicate attributes in an element exist, retain the first vs the last attribute with the same name. Case
aware (HTML case-insensitive names, XML are case-sensitive).
* Bugfix: don't submit input type=button form elements.
* Bugfix: handle error position reporting correctly and don't blow up in some edge cases.
* Bugfix: handle the ^= (starts with) selector correctly when the prefix starts with a space.
* Bugfix: don't strip out zero-width-joiners (or zero-width-non-joiners) when normalizing text. That breaks combined
emoji (and other text semantics). 🤦‍♂️
* Bugfix: Evaluator.TagEndsWith (namespaced elements) and Tag disagreed in case-sensitivity. Now correctly matches
* Bugfix: Don't throw an exception if a selector ends in a space, just trim it.
* Bugfix: HTML parser adds redundant text when parsing self-closing textarea.
* Bugfix: Don't add spurious whitespace or newlines to HTML or text for inline tags.
* Bugfix: TextNode.outerHtml() wouldn't normalize correctly without a parent.
* Bugfix: Removed binary input detection as it was causing too many false positives.
* Bugfix: when cloning a TextNode, if .attributes() was hit before the clone() method, the text value would only be a
shallow clone.
* Various code hygiene updates.
**** Release 1.12.1 [2019-May-12]
* Change: removed deprecated method to disable TLS cert checking Connection.validateTLSCertificates().
* Change: some internal methods have been rearranged; if you extended any of the Jsoup internals you may need to make
* Improvement: documents now remember their parser, so when later manipulating them, the correct HTML or XML tree
builder is reused, as are the parser settings like case preservation.
* Improvement: Jsoup now detects the character set of the input if specified in an XML Declaration, when using the
HTML parser. Previously that only happened when the XML parser was specified.
* Improvement: if the document's input character set does not support encoding, flip it to one that does.
* Improvement: if a start tag is missing a > and a new tag is seen with a <, treat that as a new tag. (This differs
from the HTML5 spec, which would make at attribute with a name beginning with <, but in practice this impacts too
many pages.
* Improvement: performance tweaks when parsing start tags, data, tables.
* Improvement: added Element.nextElementSiblings() and Element.previousElementSiblings()
* Improvement: treat center tags as block tags.
* Improvement: allow forms to be submitted with Content-Type=multipart/form-data without requiring a file upload;
automatically set the mime boundary.
* Improvement: Jsoup will now detect if an input file or URL is binary, and will refuse to attempt to parse it, with
an IO exception. This prevents runaway processing time and wasted effort creating meaningless parsed DOM trees.
* Bugfix: when using the tag case preserving parsing settings, certain HTML tree building rules where not followed
for upper case tags.
* Bugfix: when converting a Jsoup document to a W3C DOM, if an element is namespaced but not in a defined namespace,
set it to the global namespace.
* Bugfix: attributes created with the Attribute constructor with just spaces for names would incorrectly pass
* Bugfix: some pseudo XML Declarations were incorrectly handled when using the XML Parser, leading to an IOOB
exception when parsing.
* Bugfix: when parsing URL parameter names in an attribute that is not correctly HTML encoded, and near the end of the
current buffer, those parameters may be incorrectly dropped. (Improved CharacterReader mark/reset support.)
* Bugfix: boolean attribute values would be returned as null, vs an empty string, when accessed via the
Attribute#getValue() method.
* Bugfix: orphan Attribute objects (i.e. created outside of a parse or an Element) would throw an NPE on
* Bugfix: Element.shallowClone() was not making a clone of its attributes.
* Bugfix: fixed an ArrayIndexOutOfBoundsException in HttpConnection.looksLikeUtf8 when testing small strings in
specific ranges.
* Updated jetty-server (which is used for integration tests) to latest 9.2 series (9.2.28).
*** Release 1.11.3 [2018-Apr-15]
* Improvement: CDATA sections are now treated as whitespace preserving (regardless of the containing element), and are
round-tripped into output HTML.
* Improvement: added support for Deflate encoding.
* Improvement: when parsing <pre> tags, skip the first newline if present.
* Improvement: support nested quotes for attribute selection queries.
* Improvement: character references from Windows-1252 that are not valid Unicode are mapped to the appropriate
Unicode replacement.
* Improvement: accept a custom SSL socket factory in Jsoup.Connection.
* Bugfix: "Mark has been invalidated" exception was thrown when parsing some URLs on Android <= 6.
* Bugfix: The Element.text() for <div>One</div>Two was "OneTwo", not "One Two".
* Bugfix: boolean attributes with empty string values were not collapsing in HTML output.
* Bugfix: when using the XML Parser set to lowercase normalize tags, uppercase closing tags were not correctly
* Bugfix: when parsing from a URL, an end tag could be read incorrectly if it started on a buffer boundary.
* Bugfix: when parsing from a URL, if the remote server failed to complete its write (i.e. it writes less than the
Content Length header promised on a gzipped stream), the parse method would incorrectly throw an unchecked
exception. It now throws the declared IOException.
* Bugfix: leaf nodes (such as text nodes) where throwing an unsupported operation exception on childNodes(), instead
of just returning an empty list.
* Bugfix: documents with a leading UTF-8 BOM did not have that BOM consumed, so it acted as a zero width no-break
space, which could impact the parse tree.
* Bugfix: when parsing an invalid XML declaration, the parse would fail.
*** Release 1.11.2 [2017-Nov-19]
* Improvement: added a new pseudo selector :matchText, which allows text nodes to match as if they were elements.
This enables finding text that is only marked by a "br" tag, for example.
* Change: marked Connection.validateTLSCertificates() as deprecated.
* Improvement: normalize invisible characters (like soft-hyphens) in Element.text().
* Improvement: added Element.wholeText(), to easily get the un-normalized text value of an element and its children.
* Bugfix: in a deep DOM stack, a StackOverFlow exception could occur when generating implied end tags.
* Bugfix: when parsing attribute values that happened to cross a buffer boundary, a character was dropped.
* Bugfix: fixed an issue that prevented using infinite timeouts in Jsoup.Connection.
* Bugfix: whitespace preserving tags were not honoured when nested deeper than two levels deep.
* Bugfix: an unterminated comment token at the end of the HTML input would cause an out of bounds exception.
* Bugfix: an NPE in the Cleaner which would occur if an <a href> attribute value was missing.
* Bugfix: when serializing the same document in a multiple threads, on Android, with a character set that is not ascii
or UTF-8, an encoding exception could occur.
* Bugfix: removing a form value from the DOM would not remove it from FormData.
* Bugfix: in the W3CDom transformer, siblings were incorrectly inheriting namespaces defined on previous siblings.
*** Release 1.11.1 [2017-Nov-06]
* Updated language level to Java 7 from Java 5. To maintain Android support (of minversion 8), try-with-resources are
not used.
* When loading content from a URL or a file, the content is now parsed as it streams in from the network or disk,
rather than being fully buffered before parsing. This substantially reduces memory consumption & large garbage
objects when loading large files. Note that this change means that a response, once parsed, may not be parsed
again from the same response object unless you call response.bufferUp() first, which will buffer the full response
into memory.
* Added Connection.Response.bodyStream(), a method to get the response body as an input stream. This is useful for
saving a large response straight to a file, without buffering fully into memory first.
* Performance improvements in text and HTML generation (through less GC).
* Reduced memory consumption of text, scripts, and comments in the DOM by 40%, by refactoring the node
hierarchy to not track childnodes or attributes by default for lead nodes. For the average document, that's about a
30% memory reduction.
* Reduced memory consumption of Elements by refactoring their Attributes to be a simple pair of arrays, vs a
* Added support for Element.selectFirst(query), to efficiently find the first matching element.
* Added Element.appendTo(parent) to simplify slinging elements about.
* Added support for multiple headers with the same name in Jsoup.Connect
* Added Element.shallowClone() and Node.shallowClone(), to allow cloning nodes without getting all their children.
* Updated Element.text() and the :contains(text) selector to consider &nbsp; character as spaces.
* Updated Jsoup.connect().timeout() to implement a total connect + combined read timeout. Previously it specified
connect and buffer read times only, so to implement a combined total timeout, you had to have another thread send
an interrupt.
* Improved performance of Node.addChildren (was quadratic)
* Added missing support for template tags in tables
* In Jsoup.connect file uploads, added the ability to set the uploaded files' mimetype.
* Improved Node traversal, including less object creation, and partial and filtering traversor support.
* Bugfix: if a document was was re-decoded after character set detection, the HTML parser was not reset correctly,
which could lead to an incorrect DOM.
* Bugfix: attributes with the same name but different case would be incorrectly treated as different attributes.
* Bugfix: self-closing tags for known empty elements were incorrectly treated as errors.
* Bugfix: fixed an issue where a self-closing title, noframes, or style tag would cause the rest of the page to be
incorrectly parsed as data or text.
* Bugfix: fixed an issue with unknown mixed-case tags
* Bugfix: fixed an issue where the entity resources were left open after startup, causing a warning.
* Bugfix: fixed an issue where Element.getElementsByIndexLessThan(index) would incorrectly provide the root element
* Improved parse time for pages with exceptionally deeply nested tags.
* Improvement / workaround: modified the Entities implementation to load its data from a .class vs from a jar resource.
Faster, and safer on Android.
*** Release 1.10.3 [2017-Jun-11]
* Added Elements.eachText() and Elements.eachAttr(name), which return a list of Element's text or attribute values,
respectively. This makes it simpler to for example get a list of each URL on a page:
List<String> urls ="a").eachAttr("abs:href"");
* Improved selector validation for :contains(...) with unbalanced quotes.
* Improved the speed of index based CSS selectors and other methods that use elementSiblingIndex, by a factor of 34x.
* Added Node.clearAttributes(), to simplify removing of all attributes of a Node / Element.
* Bugfix: if an attribute name started or ended with a control character, the parse would fail with a validation
* Bugfix: Element.hasClass() and the ".classname" selector would not find the class attribute case-insensitively.
* Bugfix: In Jsoup.Connection, if a redirect contained a query string with %xx escapes, they would be double escaped
before the redirect was followed, leading to fetching an incorrect location.
* Bugfix: In Jsoup.Connection, if a request body was set and the connection was redirected, the body would incorrectly
still be sent.
* Bugfix: In DataUtil when detecting the character set from meta data, and there are two Content-Types defined, use
the one that defines a character set.
* Bugfix: when parsing unknown tags in case-sensitive HTML mode, end tags would not close scope correctly.
* In Jsoup.Connection, ensure there is no Content-Type set when being redirected to a GET.
* Bugfix: in certain locales (Turkey specifically), lowercasing and case insensitivity could fail for specific items.
* Bugfix: after an element was cloned, changes to its child list where not notifying the element correctly.
*** Release 1.10.2 [2017-Jan-02]
* Improved startup time, particularly on Android, by reducing garbage generation and CPU execution time when loading
the HTML entity files. About 1.72x faster in this area.
* Added to check if an element matches this CSS query.
* Added new methods to Elements: next(query), nextAll(query), prev(query), prevAll(query) to select next and previous
element siblings from a current selection, with optional selectors.
* Added Node.root() to get the topmost ancestor of a Node.
* Added the new selector :containsData(), to find elements that hold data, like script and style tags.
* Changed Jsoup.isValid(bodyHtml) to validate that the input contains only body HTML that is safe according to the
safelist, and does not include HTML errors. And in the Jsoup.Cleaner.isValid(Document) method, make sure the doc
only includes body HTML.
* In Safelists, validate that a removed protocol exists before removing said protocol.
* Allow the Jsoup.Connect thread to be interrupted when reading the input stream; helps when reading from a long stream
of data that doesn't read timeout.
* Jsoup.Connect now uses a desktop user agent by default. Many developers were getting caught by not specifying the
user agent, and sending the default 'Java'. That causes many servers to return different content than what they would
to a desktop browser, and what the developer was expecting.
* Increased the default connect/read timeout in Jsoup.Connect to 30 seconds.
* Jsoup.Connect now detects if a header value is actually in UTF-8 vs the HTTP spec of ISO-8859, and converts
the header value appropriately. This improves compatibility with servers that are configured incorrectly.
* Bugfix: in Jsoup.Connect, URLs containing non-URL-safe characters were not encoded to URL safe correctly.
* Bugfix: a "SYSTEM" flag in doctype tags would be incorrectly removed.
* Bugfix: removing attributes from an Element with removeAttr() would cause a ConcurrentModificationException.
* Bugfix: the contents of Comment nodes were not returned by
* Bugfix: if source checked out on Windows with git autocrlf=true, Entities.load would fail because of the \r char.
*** Release 1.10.1 [2016-Oct-23]
* New feature: added the option to preserve case for tags and/or attributes, with ParseSettings. By default, the HTML
parser will continue to normalize tag names and attribute names to lower case, and the XML parser will now preserve
case, according to the relevant spec. The CSS selectors for tags and attributes remain case insensitive, per the CSS
* Improved support for extended HTML entities, including supplemental characters and multiple character references.
Also reduced memory consumption of the entity tables.
* Added support for *|E wildcard namespace selectors.
* Added support for setting multiple connection headers at once with Connection.headers(Map)
* Added support for setting/overriding the response character set in Connection.Response, for cases where the charset
is not defined by the server, or is defined incorrectly.
* Improved performance of class selectors by reducing memory allocation and garbage collection.
* Improved performance of HTML output by reducing the creation of temporary attribute list iterators.
* Fixed an issue when converting to the W3CDom XML, where valid (but ugly) HTML attribute names containing characters
like '"' could not be converted into valid XML attribute names. These attribute names are now normalized if possible,
or not added to the XML DOM.
* Fixed an OOB exception when loading an empty-body URL and parsing with the XML parser.
* Fixed an issue where attribute names starting with a slash would be parsed incorrectly.
* Don't reuse charset encoders from OutputSettings, to make threadsafe.
* Fixed an issue in connections with a requestBody where a custom content-type header could be ignored.
*** Release 1.9.2 [2016-May-17]
* Fixed an issue where tag names that contained non-ascii characters but started with an ascii character
would cause the parser to get stuck in an infinite loop.
* In XML documents, detect the charset from the XML prolog - <?xml encoding="UTF-8"?>
* Fixed an issue where created XML documents would have an incorrect prolog.
* Fixed an issue where you could not use an attribute selector to find values containing unbalanced braces or
* Fixed an issue where namespaced tags (like <fb:comment>) would cause Element.cssSelector() to fail.
*** Release 1.9.1 [2016-Apr-16]
* Added support for HTTP and SOCKS request proxies, specifiable per connection.
* Added support for sending plain HTTP request bodies in POST and PUT requests, with Connection.requestBody(String).
* Added support in Jsoup.Connect for HEAD, OPTIONS, TRACE.
* Added support for HTTP 307 Temporary Redirect (replays posts, if applicable).
* Performance improvements when parsing HTML, particularly for Android Dalvik.
* Added support for writing HTML into Appendable objects (like OutputStreamWriter), to enable stream serialization.
* Added support for XML namespaces when converting jsoup documents to W3C documents.
* Added support for UTF-16 and UTF-32 character set detection from byte-order-marks (BOM).
* Added support for tags with non-ascii (unicode) letters.
* Added to retrieve a data KeyVal by its key. Useful to update form data before submission.
* Fixed an issue in the Parent selector where it would not match against the root element it was applied to.
* Fix an issue where would not return every matching element if they had the same content.
* Added not-null validators to Element.appendText() and Element.prependText()
* Fixed an issue when moving moving nodes using Element.insert(index, children) where the sibling index would be set
incorrectly, leading to the original loads being lost.
* Reverted Node.equals() and Node.hashCode() back to identity (object) comparisons, as deep content inspection
had negative performance impacts and hashkey stability problems. Functionality replaced with Node.hasSameContent().
* In Jsoup.Connect, if the same header key is seen multiple times, combine their values with a comma per the HTTP RFC,
instead of keeping just one value. Also fixes an issue where header values could be out of order.
*** Release 1.8.3 [2015-Aug-02]
* Added support for custom boolean attributes.
* When fetching XML URLs, automatically switch to the XML parser instead of the HTML parser.
* Performance improvement on parsing larger HTML pages. On Android KitKat, around 1.7x times faster. On Android
Lollipop, ~ 1.3x faster. Improvements largely from re-ordering the HtmlTreeBuilder methods based on analysis of
various websites; also from further memory reduction for nodes with no children, and other tweaks.
* Fixed an issue in Element.getElementSiblingIndex (and related methods) where sibling elements with the same content
would incorrectly have the same sibling index.
* Fixed an issue where unexpected elements in a badly nested table could be moved to the wrong location in the
* Fixed an issue where a table nested within a TH cell would parse to an incorrect tree.
* When serializing a document using the XHTML encoding entities, if the character set did not support &nbsp; chars
(such as Shift_JIS), the character would be skipped. For visibility, will now always output &xa0; when using XHTML
encoding entities (as &nbsp; is not defined), regardless of the output character set.
* Fixed an issue when resolving URLs, if the absolute URL had no path, the relative URL was not normalized correctly.
Also fixed an issue where connections that were redirected to a relative URL did not have the same normalization
rules as a URL read from Nodes.absUrl(String).
* When serialising XML, ensure that '<' characters in attributes are escaped, per spec. Not required in HTML.
*** Release 1.8.2 [2015-Apr-13]
* Performance improvements for parsing HTML on Android, of 1.5x to 1.9x, with larger parses getting a bigger
speed increase. For non-Android JREs, around 1.1x to 1.2x.
* Dramatic performance improvement in HTML serialization on Android (KitKat and later), of 115x. Improvement by working
around a character set encoding speed regression in Android.
* Performance improvement for the class name selector on Android (.class) of 2.5x to 14x. Around 1.2x
on non-Android JREs.
* File upload support. Added the ability to specify input streams for POST data, which will upload content in
MIME multipart/form-data encoding.
* Add a meta-charset element to documents when setting the character set, so that the document's charset is
* Added ability to disable TLS (SSL) certificate validation. Helpful if you're hitting a host with a bad cert,
or your JDK doesn't support SNI.
* Added ability to further tweak the canned Cleaner Safelists by removing existing settings.
* Added option in Cleaner Safelist to allow linking to in-page anchors (#)
* Use a lowercase doctype tag for HTML5 documents.
* Add support for 201 Created with redirect, and other status codes. Treats any HTTP status code 2xx or 3xx as an OK
response, and follow redirects whenever there is a Location header.
* Added support for HTTP method verbs PUT, DELETE, and PATCH.
* Added support for overriding the default POST character of UTF-8
* W3C DOM support: added ability to convert from a jsoup document to a W3C document, with the W3Dom helper class.
* In the HtmlToPlainText example program, added the ability to filter using a CSS selector. Also clarified
the usage documentation.
* Fixed validation of cookie names in HttpConnection cookie methods.
* Fixed an issue where <option> tags would be missed when preparing a form for submission if missing a selected
* Fixed an issue where submitting a form would incorrectly include radio and checkbox values without the checked
* Fixed an issue where Element.classNames() would return a set containing an empty class; and may have extraneous
* Fixed an issue where attributes selected by value were not correctly space normalized.
* In head+noscript elements, treat content as character data, instead of jumping out of head parsing.
* Fixed performance issue when parsing HTML with elements with many children that need re-parenting.
* Fixed an issue where a server returning an unsupported character set response would cause a runtime
UnsupportedCharsetException, instead of falling back to the default UTF-8 charset.
* Fixed an issue where Jsoup.Connection would throw an IO Exception when reading a page with zero content-length.
* Improved the equals() and hashcode() methods in Node, to consider all their child content, for DOM tree comparisons.
* Improved performance in Selector when searching multiple roots.
*** Release 1.8.1 [2014-Sep-27]
* Introduced the ability to chose between HTML and XML output, and made HTML the default. This means img tags are
output as <img>, not <img />. XML is the default when using the XmlTreeBuilder. Control this with the
Document.OutputSettings.syntax() method.
* Improved the performance of Element.text() by 3.2x
* Improved the performance of Element.html() by 1.7x
* Improved file read time by 2x, giving around a 10% speed improvement to file parses.
* Tightened the scope of what characters are escaped in attributes and textnodes, to align with the spec. Also, when
using the extended escape entities map, only escape a character if the current output charset does not support it.
This produces smaller, more legible HTML, with greater control over the output (by setting charset and escape mode).
* If pretty-print is disabled, don't trim outer whitespace in Element.html()
* In the HTML Cleaner, allow span tags in the basic safelist, and span and div tags in the relaxed safelist.
* Added Element.cssSelector(), which returns a unique CSS selector/path for an element.
* Fixed an issue where <svg><img/></svg> was parsed as <svg><image/></svg>
* Fixed an issue where a UTF-8 BOM character was not detected if the HTTP response did not specify a charset, and
the HTML body did, leading to the head contents incorrectly being parsed into the body. Changed the behavior so that
when the UTF-8 BOM is detected, it will take precedence for determining the charset to decode with.
* Relaxed doctype validation, allowing doctypes to not specify a name.
* Fixed an issue in parsing a base URI when loading a URL containing a http-equiv element.
* Fixed an issue for Java 1.5 / Android 2.2 compatibility, and verify it doesn't regress.
* Fixed an issue that would throw an NPE when trying to set invalid HTML into a title element.
* Added support for quoted attribute values in CSS Selectors
* Fixed support for nth-of-type selectors with unknown tags.
* Added support for 'application/*+xml' mimetypes.
* Fixed support for allowing script tags in cleaner Safelists.
* In FormElements, don't submit disabled inputs, and use 'on' as checkbox value default.
*** Release 1.7.3 [2013-Nov-10]
* Introduced FormElement, providing easy access to form controls and their data, and the ability to submit forms
with Jsoup.Connect.
* Reduced GC impact during HTML parsing, with 17% fewer objects created, and 3% faster parses.
* Reduced CSS selection time by 26% for common queries.
* Improved HTTP character set detection.
<> <>
* Added Document.location, to get the URL the document was retrieved from. Helpful if connection was redirected.
* Fixed support for self-closing script tags.
* Fixed a crash when reading an unterminated CDATA section.
* Fixed an issue where elements added via the adoption agency algorithm did not preserve their attributes.
* Fixed an issue when cloning a document with extremely nested elements that could cause a stack-overflow.
* Fixed an issue when connecting or redirecting to a URL that contains a space.
<> <>
* Added support for the HTTP/1.1 Temporary Redirect (307) status code.
*** Release 1.7.2 [2013-Jan-27]
* Added support for supplementary characters outside of the Basic Multilingual Plane.
<> <>
* Added support for structural pseudo CSS selectors, including :first-child, :last-child, :nth-child, :nth-last-child,
:first-of-type, :last-of-type, :nth-of-type, :nth-last-of-type, :only-child, :only-of-type, :empty, and :root
* Added a maximum body response size to Jsoup.Connection, to prevent running out of memory when trying to read
extremely large documents. The default is 1MB.
* Refactored the Cleaner to traverse rather than recurse child nodes, to avoid the risk of overflowing the stack.
* Added Element.insertChildren(), to easily insert a list of child nodes at a specific index.
* Added Node.childNodesCopy(), to create an independent copy of a Node's children.
* When parsing in XML mode, preserve XML declarations (<?xml ... ?>).
* Introduced Parser.parseXmlFragment(), to allow easy parsing of XML fragments.
* Allow Safelist test methods to be extended
* Added Document.OutputSettings.outline mode, to aid HTML debugging by printing out in outline mode, similar to
browser HTML inspectors.
* When parsing, allow all tags to self-close. Tags that aren't expected to self-close will get an end tag.
* Fixed an issue when parsing <textarea>/RCData tags containing unescaped closing tags that would drop the trailing >.
* Corrected the javadoc for Element#child() to note that it throws IndexOutOfBounds.
* When cloning an Element, reset the classnames set so as not to hold a pointer to the source's.
* Limit how far up the stack the formatting adoption agency algorithm will travel, to prevent the chance of a run-away
parse when the HTML stack is hopelessly deep.
* Modified Element.text() to build text by traversing child nodes rather than recursing. This avoids stack-overflow
errors when the DOM is very deep and the VM stack-size is low.
*** Release 1.7.1 [2012-Sep-23]
* Improved parse time, now 2.3x faster than previous release, with lower memory consumption.
* Reduced memory consumption when selecting elements.
* Introduced finer granularity of exceptions in Jsoup.connect, including HttpStatusException and
* Fixed an issue when determining the Windows-1254 character-set from a meta tag when run in the Turkish locale.
* Fixed whitespace preservation in <textarea> tags.
* In jsoup.connect, fail faster if the return content type is not supported.
* In jsoup.clean, allow custom OutputSettings, to control pretty printing, character set, and entity escaping.
* Fixed an issue that prevented frameset documents to be cleaned by the Cleaner.
* Fixed an issue when normalising whitespace for strings containing high-surrogate characters.
* If a server doesn't specify a content-type header, treat that as OK.
* If a server returns an unsupported character-set header, attempt to decode the content with the default charset
(UTF8), instead of bailing with an unsupported charset exception.
* Removed an unnecessary synchronisation in Tag.valueOf, allowing multi-threaded parsing to run faster.
* Made entity decoding less greedy, so that non-entities are less likely to be incorrectly treated as entities.
* Whitespace normalise document.title() output.
* In Jsoup.connection, enforce a connection disconnect after every connect. This precludes keep-alive connections to
the same host, but in practise many implementations will leak connections, particularly on error.
*** Release 1.6.3 [2012-May-28]
* Fixed parsing of group-or commas in CSS selectors, to correctly handle sub-queries containing commas.
* If a node has no parent, return null on previousSibling and nextSibling instead of throwing a null pointer exception.
* Updated Node.siblingNodes() and Element.siblingElements() to exclude the current node (a node is not its own sibling).
* Fixed HTML entity parser to correctly parse entities like frac14 (letter + number combo).
* Fixed issue where contents of a script tag within a comment could be incorrectly parsed.
* Fixed GAE support: load HTML entities from a file on startup, instead of embedding in the class.
* Fixed NPE when HTML fragment parsing a <style> tag
* Fixed issue with :all pseudo-tag in HTML sanitizer when cleaning tags previously defined in safelist
* Fixed NPE in Parser.parseFragment() when context parameter is null.
* In HTML Safelists, when defining allowed attributes for a tag, automatically add the tag to the allowed list.
*** Release 1.6.2 [2012-Mar-27]
* Added a simplified XML parsing mode, which can usefully parse valid and invalid XML, but does not enforce any HTML
document structure or special tag behaviour.
* Added the optional ability to track errors when tokenising and parsing.
* Added jsoup.connect.cookies(Map) method, to set multiple cookies at once, possibly from a prior request.
* Added Element.textNodes() and Element.dataNodes(), to easily access an element's children text nodes and data nodes.
* Added an example program that demonstrates how to format HTML as plain-text, and the use of the NodeVisitor interface.
* Added Node.traverse() and Elements.traverse() methods, to iterate through a node's descendants.
* Updated jsoup.connect so that when requests made as POSTs are redirected, the redirect is followed as a GET.
* Updated the Cleaner and Safelists to optionally preserve related links in elements, instead of converting them
to absolute links.
* Updated the Cleaner to support custom allowed protocols such as "cid:" and "data:".
* Updated handling of <base href> tags, to act on only the first one seen when parsing, to align with modern browsers.
* Updated Node.setBaseUri(), to recursively set on all the node's descendants.
* Fixed handling of null characters within comments.
* Tweaked escaped entity detection in attributes to not treat &entity_... as an entity form.
* Fixed doctype tokeniser to allow whitespace between name and public identifier.
* Fixed issue where comments within a table tag would be duplicate-fostered into body.
* Fixed an issue where a spurious byte-order-mark at the start of a document would cause the parser to miss head
* Fixed an issue where content after a frameset could cause a NPE crash. Now correctly implements spec and ignores
the trailing content.
* Tweaked whitespace checks to align with HTML spec
* Tweaked HTML output of closing script and style tags to not add an extraneous newline when pretty-printing.
* Substantially reduced default memory allocation within Node.outerHtml, to reduce memory pressure when serialising
smaller DOMs.
*** Release 1.6.1 [2011-Jul-02]
* Fixed Java 1.5 compatibility.
* Fixed an issue when parsing <script> tags in body where the tokeniser wouldn't switch to the InScript state, which
meant that data wasn't parsed correctly.
* Fixed an issue with a missing quote when serialising DocumentType nodes.
* Fixed issue where a single 0 character was lexed incorrectly as a null character.
* Fixed normalisation of carriage returns to newlines on input HTML.
* Disabled memory mapped files when loading files from disk, to improve compatibility in Windows environments.
*** Release 1.6.0 [2011-Jun-13]
* HTML5 conformant parser. Complete reimplementation of HTML tokenisation and parsing, to implement the spec. This ensures jsoup parses HTML identically to current modern browsers.
* When parsing files from disk, files are loaded via memory mapping, to increase parse speed.
* Reduced memory overhead and lowered garbage collector pressure with Attribute, Node and Element model optimisations.
* Improved "abs:" absolute URL handling in Elements.attr("abs:href") and Node.hasAttr("abs:href").
* Fixed cookie handling issue in jsoup.Connect where empty cookies would cause a validation exception.
* Added jsoup.Connect configuration options to allow HTTP errors to be ignored, and the content-type to be ignored.
Contributed by Jesse Piascik (piascikj)
* Added Node.before(node) and Node.after(node), to allow existing nodes to be moved, or new nodes to be inserted, into
precise DOM positions.
* Added Node.unwrap() and Elements.unwrap(), to remove a node but keep its contents. Useful for e.g. removing unwanted
formatting tags.
* Now handles unclosed <title> tags in document by breaking out of the title at the next start tag, instead of
eating up to the end of the document.
* Added OSGi bundle support to the jsoup package jar.
*** Release 1.5.2 [2011-Feb-27]
* Fixed issue with selector parser where some boolean AND + OR combined queries (e.g. "meta[http-equiv], meta[content]")
were being parsed incorrectly as OR only queries (e.g. former as "meta, [http-equiv], meta[content]")
* Fixed issue where a content-type specified in a meta tag may not be reliably detected, due to the above issue.
* Updated Element.text() and Element.ownText() methods to ensure <br> tags output as whitespace.
* Tweaked Element.outerHtml() method to not generate initial newline on first output element.
*** Release 1.5.1 [2011-Feb-19]
* Integrated new single-pass selector evaluators, contributed by knz (Anton Kazennikov). This significantly speeds up
the execution of combined selector queries.
* Implemented workaround to fix Scala support. Contributed by bbeck (Brandon Beck).
* Added ability to change an element's tag with Element.tagName(String), and to change many at once
with Elements.tagName(String).
* Added Node.wrap(html), Node.before(html), and Node.after(html), to allow HTML to be easily added to all nodes. These
functions were previously supported on Elements only.
* Added TextNode.splitText(index), which allows a text node to be split into two nodes at a specified index point.
This is convenient if you need to surround some text in an element.
* Updated Jsoup.Connection so that cookies set on a redirect response will be included on both the redirected request
and response.
* Infinite redirection loops in Jsoup.Connect are now prevented.
* Allow Jsoup.Connect to parse application/xml and application/xhtml+xml responses.
* Modified Jsoup.Connect to always follow relative links, regardless of the underlying HTTP sub-system.
* Defined U (underline) element as an inline tag.
* Force strict entity matching (must be &xxx; and not &xxx) in element attributes.
* Implemented clone method for Elements (contributed by knz).
* Fixed tokeniser optimisation when scanning for missing data element close tags.
* Fixed issue when using descendant regex attribute selectors.
*** Release 1.4.1 [2010-Nov-23]
* Added ability to load and parse HTML from an input stream.
* Implemented Node.clone() to create deep, independent copies of Nodes, Elements, and Documents.
* Added :not() selector, to find elements that do not match the selector. E.g. div:not(.logo) finds divs that
do not have the "logo" class name.
* Added Elements.not(selector) method, to remove undesired results from selector results.
* Implemented DataNode.setWholeData() to allow updating of script and style data contents.
* Relaxed parse rules of H1 - H6, to allow nested content. This is against spec, but matches browser and publisher
* Relaxed parse rule of SPAN to treat as block, to allow nested block content.
* Fixed issue in jsoup.connect when extracting character set from content-type header; now supports quoted
charset declaration.
* Fixed support for jsoup.connect to follow redirects between http & https URLs.
* Document normalisation now more enthusiastically enforces the correct document structure.
* Support node.outerHtml() method when node has no parent (e.g. when it has been removed from its DOM tree)
* Fixed support for HTML entities with numbers in name (e.g. &frac34, &sup1).
* Fixed absolute URL generation from relative URLs which are only query strings.
*** Release 1.3.3 [2010-Sep-19]
* Implemented Elements.empty() and Elements.remove(). This allows easy element removal, like:"iframe").remove();
* Fixed issue in Entities when unescaping &#36; ("$")
* Added restricted XHTML output entity option
*** Release 1.3.2 [2010-Aug-30]
* Treat HTTP headers as case insensitive in Jsoup.Connection. Improves compatibility for HTTP responses.
* Improved malformed table parsing by implementing ignorable end tags.
*** Release 1.3.1 [2010-Aug-23]
* Removed dependency on Apache Commons-lang. Jsoup now has no external dependencies.
* Added new Connection implementation, to enable easier and richer HTTP requests that parse to Documents. This includes
support for gzip responses, cookies, headers, data parameters, user-agent, referrer, etc.
* Added Element.ownText() method, to get only the direct text of an element, not including the text of its children.
* Added support for selectors :containsOwn(text) and :matchesOwn(regex), to supplement Element.ownText().
* Added support for non-pretty-printed HTML output, to more closely mirror the input HTML.
* Further speed optimisations for parsing and output generation.
* Fixed support for case-sensitive HTML escape entities.
* Fixed issue when parsing tags with keyless attributes.
*** Release 1.2.3 [2010-Aug-04]
* Added support for automatic input character set detection and decoding. Jsoup now automatically detects the encoding
character set when parsing HTML from a File or URL. The parser checks the content-type header, then the
<meta http-equiv> or <meta charset> tag, and finally falls back to UTF-8.
* Added ability to configure the document's output charset, to control which characters are HTML escaped, and which
are kept intact. The output charset defaults to the document's input charset. This simplifies non-ascii output.
* Added full support for all new HTML5 tags.
* Added support for HTML5 dataset custom data attributes, with the Element.dataset() map.
* Added support for the [^attributePrefix] selector query, to find elements with attributes starting with a prefix.
Useful for finding elements with datasets: [^data-] matches <p data-name="jsoup">
* Added support for namespaced elements (<fb:name>) and selectors to find them (fb|name)
* Implemented Node.ownerDocument DOM method
* Improved implicit table element handling (particularly around thead, tbody, and tfoot).
* Improved HTML output format for empty elements and auto-detected self closing tags
* Changed DT & DD tags to block-mode tags, to follow practice over spec
* Added support for tag names with - and _ (<abc_foo>, <abc-foo>)
* Handle tags with internal trailing space (<foo >)
* Fixed support for character class regular expressions in [attr=~regex] selector
*** Release 1.2.2 [2010-Jul-11]
* Performance optimisation:
- core HTML parser engine now 3.5 times faster
- HTML generator now 2.5 times faster
- much lower memory use and garbage collection time
* Added support for :matches(regex) selector, to find elements containing text matching regular expression
* Added support for [key~=regex] attribute selector, to find elements with attribute values matching regular expression
* Upgraded the selector query parser to allow nested selectors like 'div:has(p:matches(regex))'
*** Release 1.2.1 [2010-Jun-21]
* Added .before(html) and .after(html) methods to Element and Elements, to insert sibling HTML
* Added :contains(text) selector, to search for elements containing the specified text
* Added :has(selector) pseudo-selector
* Added Element#parents and Elements#parents to retrieve an element's ancestor chain
* Fixes an issue where appending / prepending rows to a table (or to similar implicit
element structures) would create a redundant wrapping elements
* Improved implicit close tag heuristic detection when parsing malformed HTML
* Fixes an issue where text content after a script (or other data-node) was
incorrectly added to the data node.
* Fixes an issue where text order was incorrect when parsing pre-document
*** Release 1.1.1 [2010-Jun-08]
* Added selector support for :eq, :lt, and :gt
* Added TextNode#text and TextNode#text(String)
* Throw exception if trying to parse non-text content
* Added Node#remove and Node#replaceWith
* Allow _ and - in CSS ID selectors (per CSS spec).
* Relative links are resolved to absolute when cleaning, to normalize
output and to verify safe protocol. (Were previously discarded.)
* Allow combinators at start of selector query, for query refinements
* Added Element#val() and #val(String) methods, for form values
* Changed textarea contents to parse as TextNodes, not DataNodes,
so contents visible to text() (and val(), as treated as form input)
* Fixed support for Java 1.5
*** Release 0.3.1 (2010-Feb-20)
* New features: supports Elements#html(), html(String),
prepend(String), append(String); bulk methods for corresponding
methods in Element.
* New feature: Jsoup.isValid(html, safelist) method for user input
form validation.
* Improved Elements.attr(String) to find first matching element
with attribute.
* Fixed assertion error when cleaning HTML with empty attribute
*** Release 0.2.2 (2010-Feb-07)
* jsoup packages are now available in the Maven central repository.
* New feature: supports Element#addClass, removeClass, toggleClass;
also collection class methods on Elements.
* New feature: supports Element#wrap(html) and Elements#wrap(html).
* New selector syntax: supports E + F adjacent sibling selector
* New selector syntax: supports E ~ F preceding sibling selector
* New: supports Element#elementSiblingIndex()
* Improved document normalisation.
* Improved HTML string output format (pretty-print)
* Fixed absolute URL resolution issue when a base tag has no href.
*** Release 0.1.2 (2010-Feb-02)
* Fixed unrecognised tag handler to be more permissive
*** Release 0.1.1 (2010-Jan-31)
* Initial beta release of jsoup