Skip to content

v1.2.0 - HTML scanner update

Choose a tag to compare

@leafo leafo released this 09 Apr 19:29
· 43 commits to master since this release

https://luarocks.org/modules/leafo/web_sanitize

  • Substantial updates to the HTML scanner/updater
    • Support parsing HTML comments (they will no longer be part of text nodes, any markup inside of a comment will be completely passed over)
    • Support parsing CDATA tags, markup inside of them will not be parsed. They are emitted as individual text nodes when text nodes are enabled (with tag set to cdata and type set to text_node)
    • Support parsing "raw text" tags like script and style. The content of these tags is read as text until the respective closing tag, no nested tags are parsed inside of them.
    • Support for parsing auto closing tags as defined by the HTML spec. This includes things like auto-closing tr, td tags when defining a table, or auto closing li tags when defining a list.
    • Support for auto closing p tags when an invalid block level tag is included inside
    • The format of the attr field on node now matches the format used on the HTML sanitizer. All attributes are included in tuple form ({ key, value} including duplicates) in the array portion of the table, and then lowercase key, values are stored as fields in the table, with the right most value overwriting any duplicates
    • Add update_attributes method on Node class for replacing rewriting an element's attributes with specified ones in addition to existing ones
    • replace_attributes will write all attributes both in tuple and table form ({"hello", "world"} and { hello = "world" }, if you have multiple entries with the same name then they will all be written) The argument format is analogous to the attr field of a parsed node
    • Fix a bug for automatically closed tags due to parent closing was not working as intended. It caused the tag to be closed at the end of the document with the contents taking up the rest of the document
    • Be more diligent about reusing Lpeg patterns where possible to avoid extra allocations when running the scanner
    • Refactored parsing primitives to parse things more atomically (reducing use of Cmt)
  • The HTML entity decoder now correctly respects that HTML entities are case sensitive
  • Add more documentation for scan_html/replace_html, including interface for the node and stack
  • Add web_sanitize.patterns module with some common patterns for parsing HTML (Although this module is undocumented, the interface should be relatively stable)

Full Changelog: v1.1.0...v1.2.0