Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Loofah-integration #11218

Closed
wants to merge 192 commits into from

14 participants

Kasper Timm Hansen Rafael Mendonça França Guillermo Iguaran Łukasz Strzałkowski Paul Nikitochkin Richard Schneeman Godfrey Chan Matt Bridges Vipul A M Michael Koziarski Jeremy Kemper Xavier Noria Robin Dupret Francesco Rodríguez
Kasper Timm Hansen
Collaborator

Today Rails uses the HTML-scanner gem to do its sanitization. We will switch the gem that is used, while keeping the old API for backwards compatibility. Instead of the scanner gem, we will use Loofah. Loofah is built on top of Nokogiri, meaning we rid the implementation's reliance on regular expressions and we get speed. On large documents and fragments Loofah is around 60 to 100% faster than the current implementation.

Notes

The sanitizers used in ActionView::SanitizeHelper have been extracted to the rails-html-sanitizer gem.
https://github.com/rafaelfranca/rails-html-sanitizer

The DomAssertions and SelectorAssertions have been extracted to the rails-dom-testing gem.
https://github.com/kaspth/rails-dom-testing

The substitution values syntax in assert_select has changed.

assert_select "div#?", /\d+/
assert_select "div:match('id', ?)", /\d+/

The attribute to match should be enclosed in quotes to avoid issues with Nokogiri's css selector syntax parsing. It is not necessary to do so with the question mark.

Todos

  • Extract failing tests to mark them as pending.
  • Find areas where backwards compatibility is broken
Pending Test Fixes
  • Find out what causes Output Error: Unknown Encoding ASCII-8BIT three times in date_helper_test and once in form_helper_test. Related: sparklemotion/nokogiri#553
  • Research the other failing tests
Sanitizers
  • Implement Action View FullSanitizer, LinkSanitizer and WhiteListSanitizer in sanitizers.rb
  • Deprecate protocol_separator and bad_tags for WhiteListSanitizer
  • Make sanitize accept custom :tags and :attributes options
  • Make sanitize accept a Loofah::Scrubber via :scrubber option
  • Make PermitScrubber
  • Make TargetScrubber
Sanitizers Testing
  • Add new tests in sanitizers_test.rb
  • Complete test coverage for PermitScrubber
  • Complete test coverage for TargetScrubber
  • Test behavior of changing sanitization
  • Move testing of PermitScrubber's peculiarities from sanitizers_test.rb
Dom and Selector Assertions
  • Reimplement assert_dom_equal with Loofah
  • Move Dom and Selector assertions to Action View
  • Fix test_nested_css_select failing on line 245
  • Fix test_feed_item_encoded finding two p elements when it shouldn't
  • Marked as pending the colliding xml namespaces issue discussed here: https://groups.google.com/forum/#!topic/nokogiri-talk/Nv8kX4p_r7I

Related issues

flavorjones/loofah#44
flavorjones/loofah#45
flavorjones/loofah#46
flavorjones/loofah#47
flavorjones/loofah#51
flavorjones/loofah#52
flavorjones/loofah#54

//@rafaelfranca

Rafael Mendonça França rafaelfranca was assigned
...lib/action_view/helpers/sanitize_helper/sanitizers.rb
((28 lines not shown))
+ def sanitize(html, options = {})
+ return nil unless html
+ validate_options(options)
+
+ loofah_fragment = Loofah.fragment(html)
+ loofah_fragment.scrub!(:strip)
+ loofah_fragment.xpath("./form").each { |form| form.remove }
+ loofah_fragment.to_s
+ end
+
+ def sanitize_css(style_string)
+ Loofah::HTML5::Scrub.scrub_css style_string
+ end
+
+ def protocol_separator
+ ActiveSupport::Deprecation.warn('protocol_separator has been deprecated and has no effect.')
Rafael Mendonça França Owner

We can call self.class.protocol_separator here and in all the deprecated instance methods to avoid duplication.

Kasper Timm Hansen Collaborator
kaspth added a note
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Rafael Mendonça França

cc @carlosantoniodasilva @josevalim @jeremy @NZKoz @tenderlove mind to review this one? I think we are done

actionpack/CHANGELOG.md
@@ -65,5 +65,9 @@
* Fix removing trailing slash for mounted apps #3215
*Piotr Sarnacki*
+
+* Loofah replaces html-scanner in dom assertions
Francesco Rodríguez
frodsan added a note

New entries must be on the top.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionpack/actionpack.gemspec
@@ -22,6 +22,8 @@ Gem::Specification.new do |s|
s.add_dependency 'activesupport', version
s.add_dependency 'actionview', version
+ s.add_dependency 'loofah', '~> 1.2.1'
+
Guillermo Iguaran Owner

remove that whitespace between the dependencies :smile:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionpack/lib/action_dispatch/testing/assertions/dom.rb
((52 lines not shown))
+ # Determines further comparison via said type
+ # i.e. element node children with equal names has their attributes compared using +attributes_are_equal?+
+ def equal_children?(child, other_child)
+ return false unless child.type == other_child.type
+
+ case child.type
+ when Nokogiri::XML::Node::ELEMENT_NODE
+ child.name == other_child.name && attributes_are_equal?(child, other_child)
+ else
+ child.to_s == other_child.to_s
+ end
+ end
+
+ # +attributes_are_equal?+ sorts elements attributes by name and compares
+ # each attribute by calling +equal_attribute?+
+ # If those are true the attributes are considered equal
Francesco Rodríguez
frodsan added a note

+true+

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Guillermo Iguaran

I think we should move actionpack/lib/action_dispatch/testing/assertions/dom.rb to actionview and make to loofah only a dependency for actionview

/cc @drogus @strzalek

Kasper Timm Hansen
Collaborator

@guilleiguaran I'll second that.
Though html-scanner is still required in selector.rb and tag.rb in assertions/, which I still haven't converted to using Loofah.

Guillermo Iguaran

Let's move selector.rb and tag.rb also to actionview :smile:

Kasper Timm Hansen
Collaborator

Fine by me.

...ction_view/helpers/sanitize_helper/permit_scrubber.rb
((57 lines not shown))
+
+ def text_or_cdata_node?(node)
+ case node.type
+ when Nokogiri::XML::Node::TEXT_NODE, Nokogiri::XML::Node::CDATA_SECTION_NODE
+ return true
+ end
+ false
+ end
+
+ def validate!(var, name)
+ if var && !var.is_a?(Enumerable)
+ raise ArgumentError, "You should pass :#{name} as an Enumerable"
+ end
+ var
+ end
+end

Are these no newlines generally allowed?

Kasper Timm Hansen Collaborator
kaspth added a note

What do you mean?

Generally, there is a newline at the end of files. This is the only one missing a newline. Mostly a nitpick...

Kasper Timm Hansen Collaborator
kaspth added a note

Ah, I see. I wasn't aware of that. I'll look through my files and add newlines where needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Kasper Timm Hansen
Collaborator

The guides say that assert_tag is deprecated: http://guides.rubyonrails.org/testing.html#testing-views
So I'm guessing we shouldn't bother moving that over.

Should I remove the file assertions/tag.rb, @rafaelfranca?

Gemfile
@@ -2,6 +2,9 @@ source 'https://rubygems.org'
gemspec
+# temporary gem while working on loofah integration
+gem 'loofah', '~> 1.2.1', github: 'kaspth/loofah'
Łukasz Strzałkowski Collaborator

Why it's pointing to your fork of loofah? Did you made any changes? Are they already merged to official master branch?

I think there are changes essential for this to work if I am not wrong.

Łukasz Strzałkowski Collaborator

Yep. It would be good to merge those changes to official loofah repository before merging this PR.

There are a lot of changes that are yet to be merged, which have not received response yet from @flavorjones .
After he does that , it should be fine to change this.

Rafael Mendonça França Owner

Yes, these PR are pending, but the idea is remove this git dependency.

Kasper Timm Hansen Collaborator
kaspth added a note

@strzalek Yes, I have made lots of changes. You can check out the PRs in the description if you're interested :wink:
@vipulnsward is right, we're still waiting for Mike to pull in the changes.

Once Loofah includes them, this line will be removed.
(that's the reason for the 'temporary' comment, I see now that that wasn't clear why the gem was temporary.)

@kaspth you can now probably change this. Thanks to @flavorjones !

Kasper Timm Hansen Collaborator
kaspth added a note

Yes, I can. Much love to @flavorjones! Thank you kindly, Mike.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionview/test/template/date_helper_test.rb
@@ -2102,6 +2102,8 @@ def test_time_select_with_html_options
end
def test_time_select_with_html_options_within_fields_for
+ skip "Pending. Output error: 'unknown encoding ASCII-8BIT' makes Loofah return an empty string"
+
@post = Post.new
Łukasz Strzałkowski Collaborator

Why all of those are still skipped?

Rafael Mendonça França Owner

We need to decide what will be done with these tests. Right now Loofah behaves differently so we are not sure if we will change the behavior breaking backward compatibility or write code to make it behave like before

Kasper Timm Hansen Collaborator
kaspth added a note

Loofah uses Nokogiri for html/xml-parsing. The tests revealed that there were many differences between Nokogiri's parsing and html-scanner's. That's why we're skipping those for now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Rafael Mendonça França

@kaspth yes, lets remove it

Kasper Timm Hansen
Collaborator

@rafaelfranca Yippee!

Kasper Timm Hansen
Collaborator

@strzalek is there anything I can do to make it easier for you to move dom.rb and selector.rb in action_dispatch/testing/assertions/ into actionview while I'm changing the files, anyway?

Łukasz Strzałkowski
Collaborator

I'm actually not touching those files at all so feel free to do whatever you want with them. They unlikely be conflicted with my changes. No worries.

Kasper Timm Hansen
Collaborator

Then it's all good then.

Paul Nikitochkin

Please take in account this:

index 3b52b20..6ff7b68 100644
--- a/actionview/test/template/sanitize_helper_test.rb
+++ b/actionview/test/template/sanitize_helper_test.rb
@@ -38,6 +38,9 @@ class SanitizeHelperTest < ActionView::TestCase
     assert_equal("<<<bad html", strip_tags("<<<bad html"))
     assert_equal("<<", strip_tags("<<<bad html>"))

+    assert_equal "This is <-- not\n a comment here.",
+                 strip_tags("This is <-- not\n a comment here.")
+
     assert_equal("Weirdos", strip_tags("Wei<<a>a onclick='alert(document.cookie);'</a>/>rdos"))

from #11629

Kasper Timm Hansen
Collaborator

@pftg Sorry for the late response. The test case has been added.

Paul Nikitochkin

:cool: thanks

Richard Schneeman
Collaborator

Monster PR is monster. Where do we stand here? Are we waiting on approval of this PR to cut a release of loofah in order to update the dependency on the PR?

Rafael Mendonça França
Owner

There are more things to do @kaspth we need to update the TODO list

Kasper Timm Hansen
Collaborator

Some small number of tests, regarding the new assert_select, still aren't passing.
Yes, we are waiting for a new release of Loofah too.

@rafaelfranca I'll update it now.

actionpack/CHANGELOG.md
@@ -1,3 +1,7 @@
+* Loofah replaces html-scanner in dom assertions
+

@kaspth maybe you can expand this a little bit?

Kasper Timm Hansen Collaborator
kaspth added a note

You're right.
Since the Dom and Selector assertions have been moved to Action View, I'll move this to Action View's changelog.

Robin Dupret Collaborator
robin850 added a note

Then, this entry can be simply removed since you have added a better one to the actionview's changelog.

Kasper Timm Hansen Collaborator
kaspth added a note

You're right, I just forgot to remove it. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...ck/lib/action_dispatch/testing/assertions/selector.rb
((417 lines not shown))
- root = HTML::Document.new(part.body.to_s).root
- assert_select root, ":root", &block
- end
- end
- end
- end
-
- protected
- # +assert_select+ and +css_select+ call this to obtain the content in the HTML page.
- def response_from_page
- html_document.root
- end
- end
- end
-end
+ActiveSupport::Deprecation.warn("ActionDispatch::Assertions::SelectorAssertions has been moved to ActionView. You can find it in action_view/testing/assertions/selector.rb.")

I think it would make sense to provide class name instead of file name in action_view/testing/assertions/selector.rb

Kasper Timm Hansen Collaborator
kaspth added a note

Really? Only the location has changed. The name is still SelectorAssertions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...lib/action_view/helpers/sanitize_helper/sanitizers.rb
((1 lines not shown))
+require 'active_support/core_ext/class/attribute'
+require 'active_support/deprecation'
+require 'action_view/helpers/sanitize_helper/scrubbers'
+
+module ActionView
+ XPATHS_TO_REMOVE = %w{.//script .//form comment()}
+
+ class Sanitizer
+ # :nodoc:
+ def sanitize(html, options = {})
+ raise NotImplementedError, "subclasses must implement"
+ end
+
+ def remove_xpaths(html, xpaths)
+ if html.respond_to?(:xpath)
+ xpaths.each { |xpath| html.xpath(xpath).remove }

@kaspth Can you rename the inner variable, it seems a bit hard to read.

Kasper Timm Hansen Collaborator
kaspth added a note

The 'xpath' in the block?

right.

Kasper Timm Hansen Collaborator
kaspth added a note

Good news, I found a way to get rid of it entirely: html.xpath(*xpaths).remove

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...lib/action_view/helpers/sanitize_helper/sanitizers.rb
((4 lines not shown))
+
+module ActionView
+ XPATHS_TO_REMOVE = %w{.//script .//form comment()}
+
+ class Sanitizer
+ # :nodoc:
+ def sanitize(html, options = {})
+ raise NotImplementedError, "subclasses must implement"
+ end
+
+ def remove_xpaths(html, xpaths)
+ if html.respond_to?(:xpath)
+ xpaths.each { |xpath| html.xpath(xpath).remove }
+ html
+ else
+ remove_xpaths(Loofah.fragment(html), xpaths).to_s

Is the to_s correct here? Both conditions return different result object types.

Kasper Timm Hansen Collaborator
kaspth added a note

The idea was to return what you called it with: String => String, DocumentFragment => DocumentFragment.

Thoughts, @rafaelfranca?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...lib/action_view/helpers/sanitize_helper/sanitizers.rb
((64 lines not shown))
+ remove_xpaths(loofah_fragment, XPATHS_TO_REMOVE)
+ loofah_fragment.scrub!(:strip)
+ end
+ loofah_fragment.to_s
+ end
+
+ def sanitize_css(style_string)
+ Loofah::HTML5::Scrub.scrub_css style_string
+ end
+
+ def protocol_separator
+ self.class.protocol_separator
+ end
+
+ def protocol_separator=(value)
+ self.class.protocol_separator

This should have an assign here?

Kasper Timm Hansen Collaborator
kaspth added a note

Deprecated: self.class.protocol_separator is used to remove duplication.

Robin Dupret Collaborator
robin850 added a note

I'm not sure to understand here. Do you mean that we can't change the value of the protocol separator ? If it's the case, then we should better put a deprecation warning too, no ? Sorry if I'm misunderstanding.

Kasper Timm Hansen Collaborator
kaspth added a note
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...lib/action_view/helpers/sanitize_helper/sanitizers.rb
((77 lines not shown))
+
+ def protocol_separator=(value)
+ self.class.protocol_separator
+ end
+
+ def bad_tags
+ self.class.bad_tags
+ end
+
+ class << self
+ def protocol_separator
+ ActiveSupport::Deprecation.warn('protocol_separator has been deprecated and has no effect.')
+ end
+
+ def protocol_separator=(value)
+ protocol_separator

ditto. I guess this is to be deprecated, so could throw a warning, like above?

Kasper Timm Hansen Collaborator
kaspth added a note

It should throw a warning :wink:

Jeremy Kemper Owner
jeremy added a note

Use an explicit deprecation warning for protocol_separator= rather than leaning on protocol_separator's warning.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
.../lib/action_view/helpers/sanitize_helper/scrubbers.rb
@@ -0,0 +1,138 @@
+# === PermitScrubber
+#
+# PermitScrubber allows you to permit only your own tags and/or attributes.
+#
+# PermitScrubber can be subclassed to determine:
+# - When a node should be skipped via +skip_node?+
+# - When a node is allowed via +allowed_node?+
+# - When an attribute should be scrubbed via +scrub_attribute?+
+#
+# Subclasses don't need to worry if tags or attributes are set or not.
+# If tags or attributes are not set, Loofahs behavior will be used.

Loofah's

Kasper Timm Hansen Collaborator
kaspth added a note

Thanks for all your feedback, @vipulnsward!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionview/lib/action_view/testing/assertions/dom.rb
((6 lines not shown))
+ # # assert that the referenced method generates the appropriate HTML string
+ # assert_dom_equal '<a href="http://www.example.com">Apples</a>', link_to("Apples", "http://www.example.com")
+ def assert_dom_equal(expected, actual, message = nil)
+ assert dom_assertion(message, expected, actual)
+ end
+
+ # The negated form of +assert_dom_equal+.
+ #
+ # # assert that the referenced method does not generate the specified HTML string
+ # assert_dom_not_equal '<a href="http://www.example.com">Apples</a>', link_to("Oranges", "http://www.example.com")
+ def assert_dom_not_equal(expected, actual, message = nil)
+ assert_not dom_assertion(message, expected, actual)
+ end
+
+ protected
+ def dom_assertion(message = nil, *html_strings)

Do we need the *html_stings, since we know we are always passing 2 arguments here?

Kasper Timm Hansen Collaborator
kaspth added a note

No. You're probably right.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionview/lib/action_view/testing/assertions/dom.rb
((43 lines not shown))
+ case child.type
+ when Nokogiri::XML::Node::ELEMENT_NODE
+ child.name == other_child.name && attributes_are_equal?(child, other_child)
+ else
+ child.to_s == other_child.to_s
+ end
+ end
+
+ # +attributes_are_equal?+ sorts elements attributes by name and compares
+ # each attribute by calling +equal_attribute?+
+ # If those are +true+ the attributes are considered equal
+ def attributes_are_equal?(element, other_element)
+ first_nodes = element.attribute_nodes.sort_by { |a| a.name }
+ other_nodes = other_element.attribute_nodes.sort_by { |a| a.name }
+
+ return false unless first_nodes.size == other_nodes.size

The condition can be taken above the sorts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...lib/action_view/helpers/sanitize_helper/sanitizers.rb
((1 lines not shown))
+require 'active_support/core_ext/class/attribute'
+require 'active_support/deprecation'
+require 'action_view/helpers/sanitize_helper/scrubbers'
+
+module ActionView
+ XPATHS_TO_REMOVE = %w{.//script .//form comment()}
+
+ class Sanitizer
+ # :nodoc:
+ def sanitize(html, options = {})
+ raise NotImplementedError, "subclasses must implement"
+ end
+
+ def remove_xpaths(html, xpaths)
+ if html.respond_to?(:xpath)
+ html.xpath(*xpaths).remove

nice :smiley:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Paul Nikitochkin pftg referenced this pull request from a commit in jetthoughts/rails
Paul Nikitochkin pftg #11629: Added tests to do not remove non html tags 0d2da10
Robin Dupret robin850 commented on the diff
actionpack/lib/action_dispatch/testing/assertions/tag.rb
((120 lines not shown))
-
- def find_tag(conditions)
- html_document.find(conditions)
- end
-
- def find_all_tag(conditions)
- html_document.find_all(conditions)
- end
-
- def html_document
- xml = @response.content_type =~ /xml$/
- @html_document ||= HTML::Document.new(@response.body, false, xml)
- end
- end
- end
-end
Robin Dupret Collaborator
robin850 added a note

Don't we need to put a deprecation warning as well here ?

Kasper Timm Hansen Collaborator
kaspth added a note

All right, done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionpack/actionpack.gemspec
@@ -20,7 +20,9 @@ Gem::Specification.new do |s|
s.requirements << 'none'
s.add_dependency 'activesupport', version
-
+ s.add_dependency 'actionview', version
Rafael Mendonça França Owner

We can't depend on action view anymore. I think we will have to remove all Dom and selectors assertion from Action Pack.

@josevalim @drogus @strzalek thoughts?

Kasper Timm Hansen Collaborator
kaspth added a note

The Dom, Selector and Tag assertions are removed (well, deprecated) in Action Pack.

Action View isn't required here on master. I think, I might have fudged something up.

Rafael Mendonça França Owner

But ActionView::Assertions is being included in Action Pack. We should remove this too

Kasper Timm Hansen Collaborator
kaspth added a note

We should remove that they're included in Action View? Is that what you're saying?

Rafael Mendonça França Owner

I'm saying we should not include anything from Action View in Action Pack. So https://github.com/rails/rails/pull/11218/files#L2R616 and https://github.com/rails/rails/pull/11218/files#L7R326 have to be removed.

Kasper Timm Hansen Collaborator
kaspth added a note

Ok.
Removing these results in two kinds of undefined methods errors: assert_select and html_document.
Both methods are part of DomAssertions.

What can I do to remove the dependencies?
/cc @strzalek

Guillermo Iguaran Owner

IMO, assert_select should live inside of AV, no AP

Kasper Timm Hansen Collaborator
kaspth added a note

@guilleiguaran assert_select has already been moved to Action View.

We're keeping Action View only as a development dependency, so the tests can use the ActionView::Assertions modules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...lib/action_view/helpers/sanitize_helper/sanitizers.rb
((38 lines not shown))
+ @link_scrubber.tags = %w(a href)
+ end
+
+ def sanitize(html, options = {})
+ Loofah.scrub_fragment(html, @link_scrubber).to_s
+ end
+ end
+
+ class WhiteListSanitizer < Sanitizer
+
+ def initialize
+ @permit_scrubber = PermitScrubber.new
+ end
+
+ def sanitize(html, options = {})
+ return nil unless html
Rafael Mendonça França Owner

return unless html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
.../lib/action_view/helpers/sanitize_helper/scrubbers.rb
((5 lines not shown))
+# PermitScrubber can be subclassed to determine:
+# - When a node should be skipped via +skip_node?+
+# - When a node is allowed via +allowed_node?+
+# - When an attribute should be scrubbed via +scrub_attribute?+
+#
+# Subclasses don't need to worry if tags or attributes are set or not.
+# If tags or attributes are not set, Loofah's behavior will be used.
+# If you override +allowed_node?+ and no tags are set, it will not be called.
+# Instead Loofahs behavior will be used.
+# Likewise for +scrub_attribute?+ and attributes respectively.
+#
+# Text and CDATA nodes are skipped by default.
+# Unallowed elements will be stripped, i.e. element is removed but its subtree kept.
+# Supplied tags and attributes should be Enumerables
+#
+# +tags=+
Rafael Mendonça França Owner

@fxn could you review this documentation? I think it is not following our style but I don't know how to fix it :blush:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...lib/action_view/helpers/sanitize_helper/sanitizers.rb
((41 lines not shown))
+ def sanitize(html, options = {})
+ Loofah.scrub_fragment(html, @link_scrubber).to_s
+ end
+ end
+
+ class WhiteListSanitizer < Sanitizer
+
+ def initialize
+ @permit_scrubber = PermitScrubber.new
+ end
+
+ def sanitize(html, options = {})
+ return nil unless html
+
+ loofah_fragment = Loofah.fragment(html)
+ if scrubber = options[:scrubber]
Rafael Mendonça França Owner

Put a blank line before the if

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...lib/action_view/helpers/sanitize_helper/sanitizers.rb
((51 lines not shown))
+
+ def sanitize(html, options = {})
+ return nil unless html
+
+ loofah_fragment = Loofah.fragment(html)
+ if scrubber = options[:scrubber]
+ # No duck typing, Loofah ensures subclass of Loofah::Scrubber
+ loofah_fragment.scrub!(scrubber)
+ elsif options[:tags] || options[:attributes]
+ @permit_scrubber.tags = options[:tags]
+ @permit_scrubber.attributes = options[:attributes]
+ loofah_fragment.scrub!(@permit_scrubber)
+ else
+ remove_xpaths(loofah_fragment, XPATHS_TO_REMOVE)
+ loofah_fragment.scrub!(:strip)
+ end
Rafael Mendonça França Owner

Put a blank line after the end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...lib/action_view/helpers/sanitize_helper/sanitizers.rb
((32 lines not shown))
+ end
+ end
+
+ class LinkSanitizer < Sanitizer
+ def initialize
+ @link_scrubber = TargetScrubber.new
+ @link_scrubber.tags = %w(a href)
+ end
+
+ def sanitize(html, options = {})
+ Loofah.scrub_fragment(html, @link_scrubber).to_s
+ end
+ end
+
+ class WhiteListSanitizer < Sanitizer
+
Rafael Mendonça França Owner

:scissors: this line

Rafael Mendonça França Owner

I mean, the blank line

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...lib/action_view/helpers/sanitize_helper/sanitizers.rb
((12 lines not shown))
+ end
+
+ def remove_xpaths(html, xpaths)
+ if html.respond_to?(:xpath)
+ html.xpath(*xpaths).remove
+ html
+ else
+ remove_xpaths(Loofah.fragment(html), xpaths).to_s
+ end
+ end
+ end
+
+ class FullSanitizer < Sanitizer
+ def sanitize(html, options = {})
+ return nil unless html
+ return html if html.empty?
Rafael Mendonça França Owner

What do you think?

return html if html.blank?

I know it will return false if you pass false but this is not valid use

Kasper Timm Hansen Collaborator
kaspth added a note

Sounds good to me. Should or can we document that case with false somewhere?

Kasper Timm Hansen Collaborator
kaspth added a note

I've tried to explain it in the commit message.

Rafael Mendonça França Owner

We don't need to document. But you can put in the commit message :+1:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionview/lib/action_view/testing/assertions/dom.rb
((10 lines not shown))
+ message ||= "Expected: #{expected}\nActual: #{actual}"
+ assert compare_doms(expected_dom, actual_dom), message
+ end
+
+ # The negated form of +assert_dom_equal+.
+ #
+ # # assert that the referenced method does not generate the specified HTML string
+ # assert_dom_not_equal '<a href="http://www.example.com">Apples</a>', link_to("Oranges", "http://www.example.com")
+ def assert_dom_not_equal(expected, actual, message = nil)
+ expected_dom, actual_dom = Loofah.fragment(expected), Loofah.fragment(actual)
+ message ||= "Expected: #{expected}\nActual: #{actual}"
+ assert_not compare_doms(expected_dom, actual_dom), message
+ end
+
+ protected
+ def compare_doms(expected, actual)
Rafael Mendonça França Owner

Put a blank line above (this is the guideline)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionview/lib/action_view/testing/assertions/dom.rb
((15 lines not shown))
+ #
+ # # assert that the referenced method does not generate the specified HTML string
+ # assert_dom_not_equal '<a href="http://www.example.com">Apples</a>', link_to("Oranges", "http://www.example.com")
+ def assert_dom_not_equal(expected, actual, message = nil)
+ expected_dom, actual_dom = Loofah.fragment(expected), Loofah.fragment(actual)
+ message ||= "Expected: #{expected}\nActual: #{actual}"
+ assert_not compare_doms(expected_dom, actual_dom), message
+ end
+
+ protected
+ def compare_doms(expected, actual)
+ return false unless expected.children.size == actual.children.size
+
+ expected.children.each_with_index do |child, i|
+ return false unless equal_children?(child, actual.children[i])
+ end
Rafael Mendonça França Owner

put a blank line after the end

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionview/lib/action_view/testing/assertions/dom.rb
((31 lines not shown))
+ true
+ end
+
+ def equal_children?(child, other_child)
+ return false unless child.type == other_child.type
+
+ if child.element?
+ child.name == other_child.name &&
+ equal_attribute_nodes?(child.attribute_nodes, other_child.attribute_nodes)
+ else
+ child.to_s == other_child.to_s
+ end
+ end
+
+ def equal_attribute_nodes?(nodes, other_nodes)
+ return false unless nodes.size == other_nodes.size
Rafael Mendonça França Owner

blank line

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionview/lib/action_view/testing/assertions/dom.rb
((37 lines not shown))
+ if child.element?
+ child.name == other_child.name &&
+ equal_attribute_nodes?(child.attribute_nodes, other_child.attribute_nodes)
+ else
+ child.to_s == other_child.to_s
+ end
+ end
+
+ def equal_attribute_nodes?(nodes, other_nodes)
+ return false unless nodes.size == other_nodes.size
+ nodes = nodes.sort_by(&:name)
+ other_nodes = other_nodes.sort_by(&:name)
+
+ nodes.each_with_index do |attr, i|
+ return false unless equal_attribute?(attr, other_nodes[i])
+ end
Rafael Mendonça França Owner

blank line

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionview/test/template/assert_select_test.rb
@@ -173,11 +173,11 @@ def test_counts
def test_substitution_values
render_html %Q{<div id="1">foo</div><div id="2">foo</div>}
- assert_select "div#?", /\d+/ do |elements|
+ assert_select "div:match('id', ?)", /\d+/ do |elements|
Rafael Mendonça França Owner

We need to document this change in the CHANGELOG since it will be non-backward compatible

Kasper Timm Hansen Collaborator
kaspth added a note

I'll copy the description under notes from this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionpack/actionpack.gemspec
@@ -20,7 +20,9 @@ Gem::Specification.new do |s|
s.requirements << 'none'
s.add_dependency 'activesupport', version
-
+ s.add_dependency 'actionview', version
+
+ s.add_dependency 'loofah', '~> 1.2.1'
Rafael Mendonça França Owner

We can remove this one also.

Kasper Timm Hansen Collaborator
kaspth added a note

Ok, per your earlier comment I'm removing the actionview dependency, too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionview/CHANGELOG.md
@@ -1,3 +1,21 @@
+* Dom and Selector assertions has been moved to Action View.
+
+ Loofah replaces html-scanner in `DomAssertions`:
+ `assert_dom_equal`
+ `assert_dom_not_equal`
+
+ Also in `SelectorAssertions`:
+ `css_select`
+ `assert_select`
+ `assert_select_encoded`
+ `assert_select_email`
+
+ *Kasper Timm Hansen*, *Rafael Mendonça França*
Rafael Mendonça França Owner

The credit is all your bro :clap: You can remove my name.

Kasper Timm Hansen Collaborator
kaspth added a note

O mentor, my mentor! :scream:

Ok, :scissors:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionview/CHANGELOG.md
@@ -1,3 +1,21 @@
+* Dom and Selector assertions has been moved to Action View.
Rafael Mendonça França Owner

We need a similar entry on Action Pack to tell they were removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionview/CHANGELOG.md
@@ -1,3 +1,21 @@
+* Dom and Selector assertions has been moved to Action View.
+
+ Loofah replaces html-scanner in `DomAssertions`:
+ `assert_dom_equal`
+ `assert_dom_not_equal`
+
+ Also in `SelectorAssertions`:
+ `css_select`
+ `assert_select`
+ `assert_select_encoded`
+ `assert_select_email`
+
+ *Kasper Timm Hansen*, *Rafael Mendonça França*
+
+* Loofah replaces html-scanner in sanitize_helper
Rafael Mendonça França Owner

`sanitize_helper`

Rafael Mendonça França Owner

Lets talk about sanitize receiving a scrubber.

Kasper Timm Hansen Collaborator
kaspth added a note

Ok, I'll also add information about PermitScrubber and TargetScrubber.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionview/lib/action_view.rb
@@ -23,6 +23,7 @@
require 'active_support'
require 'active_support/rails'
+require 'loofah'
Rafael Mendonça França Owner

Maybe we should require this only in the entries points. sanitize_helper/sanitizers.rb and testing/assertions.rb. Could you test if this will work?

Kasper Timm Hansen Collaborator
kaspth added a note

hmm...
It works for assertions.rb.
But running the tests for sanitizers.rb gave me 195 failures.
What's weird is, I don't get any errors a la uninitialized constant Loofah, the tests just fail.

Kasper Timm Hansen Collaborator
kaspth added a note

Scratch that.
The failures were from an earlier commit (an unless instead of an if. Ugh).
Requiring only in the entry points works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Rafael Mendonça França rafaelfranca commented on the diff
actionview/lib/action_view/helpers/sanitize_helper.rb
@@ -27,7 +27,29 @@ module SanitizeHelper
#
# <%= sanitize @article.body %>
#
- # Custom Use (only the mentioned tags and attributes are allowed, nothing else)
+ # Custom Use - Custom Scrubber
+ # (supply a Loofah::Scrubber that does the sanitization)
+ #
+ # scrubber can either wrap a block:
+ # scrubber = Loofah::Scrubber.new do |node|
+ # node.text = "dawn of cats"
+ # end
+ #
+ # or be a subclass of Loofah::Scrubber which responds to scrub:
+ # class KittyApocalypse < Loofah::Scrubber
Rafael Mendonça França Owner

lol @ name

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
...ck/test/controller/request_forgery_protection_test.rb
@@ -293,10 +293,10 @@ class RequestForgeryProtectionControllerUsingResetSessionTest < ActionController
end
test 'should emit a csrf-param meta tag and a csrf-token meta tag' do
- SecureRandom.stubs(:base64).returns(@token + '<=?')
+ SecureRandom.stubs(:base64).returns(@token + 'U+003c=U+0022U+003fU+0022') # '<="?"'
Michael Koziarski Owner
NZKoz added a note

It's not clear why you're changing the test code here and then the assertions below. Is this change breaking people's apps if they make similar assertions?

Kasper Timm Hansen Collaborator
kaspth added a note

Nokogiri leaves the < unescaped, so the assert_select looking for &lt; will never work.
We're working on getting the right assertion.
//@rafaelfranca

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionpack/lib/action_controller/test_case.rb
@@ -612,6 +613,7 @@ def build_response
included do
include ActionController::TemplateAssertions
include ActionDispatch::Assertions
+ include ActionView::Assertions
Rafael Mendonça França Owner

We have to remove this or we will make Action Pack dependent on Action View.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionpack/lib/action_dispatch/testing/integration.rb
@@ -322,7 +323,7 @@ def process(method, path, parameters = nil, headers_or_env = nil)
end
module Runner
- include ActionDispatch::Assertions
+ include ActionDispatch::Assertions, ActionView::Assertions
Rafael Mendonça França Owner

We have to remove this or we will make Action Pack dependent on Action View

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
actionpack/lib/action_dispatch/testing/assertions/tag.rb
((122 lines not shown))
- def find_tag(conditions)
- html_document.find(conditions)
- end
-
- def find_all_tag(conditions)
- html_document.find_all(conditions)
- end
-
- def html_document
- xml = @response.content_type =~ /xml$/
- @html_document ||= HTML::Document.new(@response.body, false, xml)
- end
- end
- end
-end
+ActiveSupport::Deprecation.warn("ActionDispatch::Assertions::TagAssertions has been eprecated. Use the assert_select methods from SelectorAssertions in action_view/testing/assertions/selector.rb.")
Rafael Mendonça França Owner

Actually we removed it. So we can't say it is deprecated. Lets only remove it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
guides/source/testing.md
@@ -634,7 +634,7 @@ assert_select "ol" do
end
```
-The `assert_select` assertion is quite powerful. For more advanced usage, refer to its [documentation](http://api.rubyonrails.org/classes/ActionDispatch/Assertions/SelectorAssertions.html).
+The `assert_select` assertion is quite powerful. For more advanced usage, refer to its [documentation](http://api.rubyonrails.org/classes/ActionView/Assertions/SelectorAssertions.html).
Kasper Timm Hansen Collaborator
kaspth added a note

@rafaelfranca where do I point this to now that assert_select is not in rails?

Rafael Mendonça França Owner

Good question. cc @fxn

Xavier Noria Owner
fxn added a note

I believe the gem should be documented, and the Rails guide would maybe just mention the gem and link to it for further details.

Kasper Timm Hansen Collaborator
kaspth added a note
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Robin Dupret robin850 added this to the 4.2.0 milestone
actionmailer/lib/action_mailer/test_case.rb
@@ -20,6 +20,8 @@ module Behavior
class_attribute :_mailer_class
setup :initialize_test_deliveries
setup :set_expected_mail
+
+ teardown :restore_delivery_method
Kasper Timm Hansen Collaborator
kaspth added a note

@rafaelfranca I'm not sure if this is the right move.

The tests on line 112 and 119 in delivery_methods_test.rb we're breaking when all the actionmailer tests we're running (i.e. running just the file the tests passed).

The tests expected the ActionMailer::Base.delivery_method to be :smtp and not :test, which they were on this branch. The only thing different between master and this branch is that assert_select_email_test file. Indeed commenting out the whole class makes the tests pass, while commenting out just the test methods made them break.

I hope I've made the problem clear, it was very confusing to figure out.

Rafael Mendonça França Owner

This change make sense. I think it is an order dependent problem and this is the right way to fix. Maybe we should also ensure assert_select_email_test file doesn't leak any state.

Kasper Timm Hansen Collaborator
kaspth added a note

But with this change it won't leak anything if I'm not mistaken?
Looks like Zuhao fixed this on master: c4f4123

Rafael Mendonça França Owner

You are right.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
kaspth added some commits
Kasper Timm Hansen kaspth Added Loofah as a dependency in actionview.gemspec.
Implemented ActionView: FullSanitizer, LinkSanitizer and WhiteListSanitizer in sanitizers.rb.
Deprecated protocol_separator and bad_tags.
Added new tests in sanitizers_test.rb and reimplemented assert_dom_equal with Loofah.
c94e24f
Kasper Timm Hansen kaspth Removed duplication in the deprecated methods. d4d1392
Kasper Timm Hansen kaspth Added PermitScrubber which allows you to permit elements for sanitiza…
…tion.
2622da1
Kasper Timm Hansen kaspth Reordered form removal with stripping. 3e4ae8e
Kasper Timm Hansen kaspth Removed the contains_bad_protocols? method as well as the tests for i…
…t. Loofah already deals with this.
167e998
Kasper Timm Hansen kaspth Changed expected value from '<b>' to empty string. d3d979e
Kasper Timm Hansen kaspth bad_tags include form since we remove it. Also to prevent a should_al…
…low_form_tag test creation.
91712cc
Kasper Timm Hansen kaspth Added guard clauses to FullSanitizer. 5dfd394
Kasper Timm Hansen kaspth Extracted failing tests in santiizers_test.rb into their own methods …
…and marked them as pending.
2e8c536
Kasper Timm Hansen kaspth Added video poster sanitization testing (from @vipulnsward). 6a05cb6
Kasper Timm Hansen kaspth Renamed the SanitizerTest class to SanitersTest, to remove the confli…
…ct with the old SanitizerTest for html-scanner.
5282518
Kasper Timm Hansen kaspth Added removal of script tags to WhiteListSanitizer. 55b453f
Kasper Timm Hansen kaspth Extracted the xpath removals into some new API that allows users to r…
…emove xpath subtrees.
68f75b9
Kasper Timm Hansen kaspth Added comment removal. Changed definitation of remove_xpaths to not u…
…se a splat operator.
40bbb49
Kasper Timm Hansen kaspth Extracted one highlight test method and marked it as pending. c80da23
Kasper Timm Hansen kaspth Changed the description of some pending tests. Changed the expected o…
…utput of a script test.
4f67398
Kasper Timm Hansen kaspth Added ActionView::Sanitizer and moved remove_xpaths to there. 4fbec83
Kasper Timm Hansen kaspth Added some tests for ActionView::Sanitizer. d631b37
Kasper Timm Hansen kaspth Marked some tests as pending in date_helper_test.rb. 561fbe0
Kasper Timm Hansen kaspth Marked a test in form_helper_test.rb as pending because of unknown en…
…coding ASCII-8BIT output error.
32850b5
Kasper Timm Hansen kaspth Marked tests in sanitize_helper_test.rb as pending. 7e2f7da
Kasper Timm Hansen kaspth Moved requiring of Loofah from sanitizers.rb to action_view.rb. c88d573
Kasper Timm Hansen kaspth Added ability to pass a custom scrubber to sanitize. Includes test co…
…verage.
6241bb8
Kasper Timm Hansen kaspth Marked the private API as not needing code documentation. 8fdf86c
Kasper Timm Hansen kaspth Updated the documentation to reflect the scrubber option. dad96ef
Kasper Timm Hansen kaspth Updated documentation to tell that a custom scrubber takes precedence. 42f0198
Kasper Timm Hansen kaspth Removed whitespace between dependencies. c63b75a
Kasper Timm Hansen kaspth Corrected documentation bug. 8f5547f
Kasper Timm Hansen kaspth Removed tag.rb since it has been deprecated. 2ff60e8
Kasper Timm Hansen kaspth The first attempt at abstracting argument parsing from selection meth…
…ods.
8c0536c
Kasper Timm Hansen kaspth Changed name to selector. And a bunch of other things. a38c759
Kasper Timm Hansen kaspth Removed argument and root variable in assert_select proc. 328512e
Kasper Timm Hansen kaspth Simplified the first delete_if loop in assert_select to use Loofah's …
…text method.
cb215c9
Kasper Timm Hansen kaspth Removed more lines of code in assert_select. 6fa3af8
Kasper Timm Hansen kaspth Removed more redundant lines. Removed fix_content block that circumve…
…nted a bug in html-scanner.
3b942d3
Kasper Timm Hansen kaspth Changed comparison from HTML::Node to Nokogiri::XML::Node in assert_s…
…elect_encoded.
0538d26
Kasper Timm Hansen kaspth Changed css_select and pulled out response_from_page from Selector. 205bfe9
Kasper Timm Hansen kaspth Updated selector to not have reponse_from_page. afa4caf
Kasper Timm Hansen kaspth Removed the custom selected proc. It's no longer needed. 744cba7
Kasper Timm Hansen kaspth Renamed Selector to ArgumentFilter. Put code from HTMLSelector to Arg…
…umentFilter.
63938f6
Kasper Timm Hansen kaspth Added filter_matches to reduce line count in assert_select. ff0939a
Kasper Timm Hansen kaspth Replaced fragment calls with document, since we assume the responses …
…are complete documents.
b4258f8
Kasper Timm Hansen kaspth Changed some documentation for css_select. 95afa79
Kasper Timm Hansen kaspth Added assert_size_match! with the assertions for assert_select. 332ccb3
Kasper Timm Hansen kaspth Removed redundant comments from assert_select.
Cleaned up a comment.
7ef141a
Kasper Timm Hansen kaspth Changed early return for filter_matches as well as reassigning matche…
…s. Meddled with initialize in ArgumentFilter.
9893a28
Kasper Timm Hansen kaspth Fixed Nokogiri::CSS::SyntaxErrors in test file. 240ce95
Kasper Timm Hansen kaspth Initialized @css_selector_is_second_argument in determine_root_from. 11fc26b
Kasper Timm Hansen kaspth Replaced html-scanner with Loofah. 37ac1c4
Kasper Timm Hansen kaspth Fixed Nokogiri::CSS::SyntaxErrors.
Fixed a Nokogiri::CSS::SyntaxError by using its expected format for unicode characters.
37ff080
Kasper Timm Hansen kaspth Changed filter_matches to return a new NodeSet instead of Array. 6aea3bc
Kasper Timm Hansen kaspth Removed selector_test.rb since HTML::Selector will be removed. Soon. ab3a236
Kasper Timm Hansen kaspth Readded html_document method since it is used integration tests. 63fc9f1
Kasper Timm Hansen kaspth Changed tests to assert_kind_of Loofah::HTML::Document. 0a7fac5
Kasper Timm Hansen kaspth Updated documentation to remove mention of HTML::Selector and clarify…
… what is expected of a selector now.
bc1363e
Kasper Timm Hansen kaspth Updated testing guide to reflect changes in ActionDispatch::SelectorA…
…ssertions.
1bc3438
Kasper Timm Hansen kaspth Fixed typo in method name. Fixed Nokogiri::CSS::SyntaxError. e4772cc
Kasper Timm Hansen kaspth Readded some documentation about substitution values. 9eada2d
Kasper Timm Hansen kaspth Added implementation for substitution values via Nokogiri's custom ps…
…eudo classes.
1e4f1ab
Kasper Timm Hansen kaspth Fixed nested assert_select bug. Trying to create a full document for …
…a nested call that already had a document.
8d6c92c
Kasper Timm Hansen kaspth Added a proper substitution context class. Changed ArgumentFilter to …
…be a selector. It is now called HTMLSelector.
5f756e3
Kasper Timm Hansen kaspth Changed css_select to not count on multiple selectors. Fixed bug in d…
…etermine_root_from where @selected was an Array. Changed assert_select_encoded to use a fragment instead of a document.
1b0a4b9
Kasper Timm Hansen kaspth Changed test methods to use new substitution syntax more in line with…
… css selectors.
6f61cf6
Kasper Timm Hansen kaspth Fixed bug by switching to Loofah fragment instead of document. 5670bbe
Kasper Timm Hansen kaspth Changed xml_namespace test to correct syntax. However, Nokogiri won't…
… recognize the namespace.
a5eeec4
Kasper Timm Hansen kaspth Changed html_document to use fragments. Changed response_from_page to…
… be an alias of html_document.
2cc4f42
Kasper Timm Hansen kaspth Added NodeSet comparison to possible root element in determine_root_f…
…rom.
7e7e191
Kasper Timm Hansen kaspth Fixed: now only compares html of children in filter_matches. 4ed7841
Kasper Timm Hansen kaspth Simplified assert_select further by moving match filtering into HTMLS…
…elector select.
99ac0cd
Kasper Timm Hansen kaspth Cleaned up SubstitutionContext class. 22aa73b
Kasper Timm Hansen kaspth Fixed: inadvertently called message method in MiniTest instead of sel…
…ector.message.
3c6ce74
Kasper Timm Hansen kaspth Fixed: test_nested_assert_select selects from elements instead of ele…
…ments[0] and elements[1].
4f19614
Kasper Timm Hansen kaspth Added test case for non-comment. Removed pending assertion that passed. 3ac4c62
Kasper Timm Hansen kaspth add_regex returns inspected value for non Regexp objects. Workaround,…
… so users don't have to care about enclosing values in double quotes.
a45ee9e
Kasper Timm Hansen kaspth Reverted to using documents instead of document fragments, since sear…
…ching via default xml namespaces didn't work.
f103122
Kasper Timm Hansen kaspth Fixed: no longer wrapped @selected in fragment, since .css works fine…
… without it.
91650dd
Kasper Timm Hansen kaspth Returning from filter if matches are empty. f2c9734
Kasper Timm Hansen kaspth Moved around alias line. ea5f3ba
Kasper Timm Hansen kaspth Wrapped element to search in NodeSet. Changed selectors to selector. 77d0333
Kasper Timm Hansen kaspth Reworked the wrapping root in NodeSet implementation in css_select. 9020abe
Kasper Timm Hansen kaspth Updated documentation to state more things about css selectors with s…
…ubstitution values.
6fd74dc
Kasper Timm Hansen kaspth Removed mention of css_select supporting substitution values. It is n…
…ot tested anywhere.
1bc0bec
Kasper Timm Hansen kaspth Fixed: assert_select_encoded finds the right content. No longer uses …
…a <encoded> wrapper. Updated tests to reflect this.
9f73f9f
Kasper Timm Hansen kaspth Moved Dom and Selector assertions from ActionDispatch to ActionView. 95c517b
Kasper Timm Hansen kaspth Removed require for active_support/core_ext/object/inclusion since in…
…? isn't used anywhere.
09454dc
Kasper Timm Hansen kaspth Removed copyright notice since we aren't relying on html-scanner anym…
…ore.
dea8ddb
Kasper Timm Hansen kaspth Marked test_feed_xhtml as pending. See description in the test. 7b2e753
Kasper Timm Hansen kaspth Require ActionView::Assertions in ActionController test_case.rb. 748f281
Kasper Timm Hansen kaspth Added correct requires in html-scanner tests. Sanitizers are not base…
…d on html-scanner anymore, so sanitizer_test.rb is removed.
3ca1061
Kasper Timm Hansen kaspth Changed explanation for no duck typing of custom scrubbers. f428aea
Kasper Timm Hansen kaspth Refactored remove_xpaths to use duck typing and read better. 945e7f5
Kasper Timm Hansen kaspth Extracted the common xpaths to remove into XPATHS_TO_REMOVE. d1de087
Kasper Timm Hansen kaspth Changed FullSanitizer sanitize to use tap method instead of temporary…
… variable.
739ecdf
Kasper Timm Hansen kaspth Added LinkScrubber to remove duplication in LinkSanitizer. As such ma…
…de PermitScrubber easier to subclass.
1cdc511
Kasper Timm Hansen kaspth Already killed off LinkScrubber. Changed it instead to be TargetScrub…
…ber, which is more general, while still allowing maximum code reuse.
ac0d778
Kasper Timm Hansen kaspth Changed PermitScrubbers documentation to list override points for sub…
…classes. Renamed should_remove_attributes? to should_scrub_attributes?.
ea57c7c
Kasper Timm Hansen kaspth Changed PermitScrubber to be even more extensible. Updated TargetScru…
…bber to be compliant. Updated documentation for PermitScrubber and TargetScrubber for clarity.
557806f
Kasper Timm Hansen kaspth Refactored scrub to keep_node? instead of scrub_node calling it. Also…
… added ability to stop traversing by returning STOP from scrub_node.
39df402
Kasper Timm Hansen kaspth Initialized tags and attributes to nil. b13d22b
Kasper Timm Hansen kaspth Fixed: spelling error. 349230e
Kasper Timm Hansen kaspth Reworked documentation for PermitScrubber and TargetScrubber. d6a6d42
Kasper Timm Hansen kaspth Removed :nodoc: from PermitScrubber. 53f25ae
Kasper Timm Hansen kaspth Trimmed deprecation message for ActionDispatch::Assertions::SelectorA…
…ssertions.
dddf86a
Kasper Timm Hansen kaspth Moved Action Pack changelog message to Action View. Clarified Dom and…
… Selector assertions changes in there.
63e0fa7
Kasper Timm Hansen kaspth Added related Nokogiri issue link to tests that fail with unknown enc…
…oding ASCII-8BIT.
5a14dbf
Kasper Timm Hansen kaspth Reworked root and selector conditional assignment in css_select. bffa646
Kasper Timm Hansen kaspth Changed wording of missing selector argument exception message in css…
…_select.
86c6f5b
Kasper Timm Hansen kaspth Removed duplication in assert_dom_equal and assert_dom_not_equal. 9dac1e8
Kasper Timm Hansen kaspth Fixed: spelling mistake in SanitizeHelperTest. 9a3a59e
Kasper Timm Hansen kaspth Changed: remove_xpaths called with String returns String, while calle…
…d with Loofah fragment returns Loofah fragment. Added tests for this.
97c5e6f
Kasper Timm Hansen kaspth Renamed: remove_xpaths tests no longer prefixed with sanitizer. 1825edc
Kasper Timm Hansen kaspth Changed: return early from compare_doms if the two doms don't have th…
…e same number of children.
75789d5
Kasper Timm Hansen kaspth Changed: removed @selected and @page variables from HTMLSelector sinc…
…e one method used them. Passed the values directly to there instead.
71aaddb
Kasper Timm Hansen kaspth Changed: HTMLSelector comparisons renamed to equality_tests. 20615ec
Kasper Timm Hansen kaspth Changed: put selector extraction into selector_from, which is renamed…
… to extract_selector.
ce4396b
Kasper Timm Hansen kaspth Removed unnecessary lines from HTMLSelector initialize. 9a536bc
Kasper Timm Hansen kaspth Renamed: HTMLSelector css_selector to selector. 65ed2b6
Kasper Timm Hansen kaspth Changed: using duck typing instead of requiring subclasses of Node an…
…d NodeSet.
cabef14
Kasper Timm Hansen kaspth Moved: initial assignment of @selector_is_second_argument is now in i…
…nitialize.
4b55c0a
Kasper Timm Hansen kaspth Changed conditional check in filter. Removed weird comments. e600b3a
Kasper Timm Hansen kaspth Extracted: create Regexp from match_with and use =~ to compare instea…
…d of checking .is_a? Regexp every time through the loop.
5169b00
Kasper Timm Hansen kaspth Fixed: added apostrophe to possessive noun. c1a7864
Kasper Timm Hansen kaspth Simplified the removal of xpaths in remove_xpaths. Added more tests f…
…or remove_xpaths.
6217178
Kasper Timm Hansen kaspth Changed back to =~ or == comparison in HTMLSelector filter. bab54e4
Kasper Timm Hansen kaspth Removed html_strings variable, no splat operator needed. 73c690d
Kasper Timm Hansen kaspth Changed attributes_are_equal? to equal_attribute_nodes? which takes a…
…ttribute_nodes instead of nodes.
d6067e8
Kasper Timm Hansen kaspth Reworked some internal documentation for equal_attribute_nodes?. 905d2bc
Kasper Timm Hansen kaspth Removed case statement in equal_children? used child.element? instead. 97d20b1
Kasper Timm Hansen kaspth Removed unnecessary documentation in DomAssertions. 7f7a1b5
Kasper Timm Hansen kaspth Removed a bunch of duplicated tests in SanitizeHelperTest. 2563c2c
Kasper Timm Hansen kaspth Fixed uninitialized constant ActionView::HTML error entered after rec…
…ent git rebase.
01e6e1d
Kasper Timm Hansen kaspth Removed dom_assertion method since it created bugs. cb865e1
Kasper Timm Hansen kaspth Added deprecation warning to ActionDispatch::Assertions::TagAssertions. 4e97d75
Kasper Timm Hansen kaspth Changed test expectation from '<<' to '' with string to sanitize '<<<…
…bad html>' in sanitizers_test.
229092f
Kasper Timm Hansen kaspth Removed require's for html-scanner. 170f414
Kasper Timm Hansen kaspth Stylistic improvements. Some light documentation for remove_xpaths. 5430487
Kasper Timm Hansen kaspth Now returning html if html is blank? in FullSanitizer and WhiteListSa…
…nitizer. This means it'll return false if called with false, however that is not a valid use case.
0a0d151
Kasper Timm Hansen kaspth Stylistic improvements in ActionView::Assertions::DomAssertions. dd19557
Kasper Timm Hansen kaspth Updated Action View changelog entries with more information about the…
… changes in the API. Removed mention of mentor (at his request).
240ac66
Kasper Timm Hansen kaspth Added deprecation notice to actionpack changelog. 7cd2eb5
Kasper Timm Hansen kaspth Minor rewording in TargetScrubber documentation. 19406da
Kasper Timm Hansen kaspth Now only requiring Loofah in the places where it is needed. 7f9106d
Kasper Timm Hansen kaspth Changed PermitScrubber's direction to bottom up to align better with …
…Loofah's strip scrubber.
ddc24fd
Kasper Timm Hansen kaspth Added some test coverage for PermitScrubber. facc4f3
Kasper Timm Hansen kaspth Moved some tests to scrubbers_test.rb. Added better testing of access…
…or validation.
b4cfb59
Kasper Timm Hansen kaspth Rounded out PermitScrubber tests. Extracted helper methods to a Scrub…
…berTest class.
15382e9
Kasper Timm Hansen kaspth Added tests for TargetScrubber. af05b01
Kasper Timm Hansen kaspth Nokogiri leaves '<' unescaped, so the assert_select looking for '&lt;…
…' will never work. Switched to assert_matching the reponse body.
535a3b6
Kasper Timm Hansen kaspth Added deprecation warning for invalid selectors and skipping assertions. 9ef95a7
Kasper Timm Hansen kaspth Silenced deprecation warnings in the tests. Documentation uses presen…
…t tense. Changed deprecation message to not use you. Also returning from rescue block in catch_invalid_selector to abort reraising the exception.
68e08fe
Kasper Timm Hansen kaspth Changed ActiveSupport::Derprecation.silence to assert_deprecated. 2e81324
Kasper Timm Hansen kaspth Moved ActionView::Assertions dependency from Action Pack's lib to abs…
…tract_unit.rb.
a766a02
Kasper Timm Hansen kaspth Removed tag.rb, since it is actually removed, not just deprecated. [c…
…i skip]
fa916af
Kasper Timm Hansen kaspth Added rails-dom-testing and rails-html-sanitizer to Gemfile. Added te…
…sts for assert_select_email.
94ca27b
Kasper Timm Hansen kaspth Removed ActionView::Assertions. Getting ready to exchange with Rails:…
…:Dom::Testing::Assertions.
c287572
Kasper Timm Hansen kaspth Exchanged requiring of action view assertions with rails dom testing …
…assertions.
5ac99b0
Kasper Timm Hansen kaspth Changed deprecation message in dom and selector assertions in Action …
…Dispatch.
6061472
Kasper Timm Hansen kaspth Required rails-dom-testing in test_case.rb 5dc57db
Kasper Timm Hansen kaspth Removed assert_select test file, since it has been moved to rails-dom…
…-testing.
72ce9a4
Kasper Timm Hansen kaspth Included DomAssertions in url_helper- and atom_feed_helper_test.rb. 93f2cd8
Kasper Timm Hansen kaspth Removed sanitizers- and scrubbers_test.rb. They are in rails-html-san…
…itizer.
82e0705
Kasper Timm Hansen kaspth Fixed deprecated selector in form_collections_helper_test.rb with fro…
…m catch_invalid_selector. Sweet.
648f748
Kasper Timm Hansen kaspth Included rails-dom-testing in rails_info_controller_test.rb 28fd5eb
Kasper Timm Hansen kaspth Support for changes in SelectorAssertions. 83f1563
Kasper Timm Hansen kaspth Updated html-scanner deprecation message. 50347b1
Kasper Timm Hansen kaspth Moved html_document to ActionDispatch::Assertions. Included the Rails…
…::Dom::Testing::Assertions there as well.
9efdffe
Kasper Timm Hansen kaspth Completed integration of rails-html-sanitizer in SanitizeHelper. Depr…
…ecated protocol_separator accessors and bad_tags=.
38620e1
Kasper Timm Hansen kaspth Updated CHANGELOG message about rails-dom-testing. 51c1986
Kasper Timm Hansen kaspth Updated CHANGELOG message to include info about rails-html-sanitizer. 0926fa5
Kasper Timm Hansen kaspth Remove some whitespace in actionpack.gemspec. 5913a09
Kasper Timm Hansen kaspth Remove unneeded comment in test. 65d0443
Kasper Timm Hansen kaspth Made deprecation messages in sanitize_helper more clear. 2a7f13e
Kasper Timm Hansen kaspth Migrated test away from escaped quotes. dd48b0a
Kasper Timm Hansen kaspth Remove include of rails-dom-testing in rails_info_controller_test.rb …
…as it is included in ActionController::TestCase.
c0e1b20
Kasper Timm Hansen kaspth Deprecate configurations and use allowed_tags and allowed_attributes …
…on WhiteListSanitizer.
13da278
Kasper Timm Hansen kaspth Changed configuration documentation to no longer state it replaces a …
…Set.
7587632
Kasper Timm Hansen kaspth Delegate allowed tags and attributes setting to HTML::WhiteListSaniti…
…zer.
5d3a292
Kasper Timm Hansen kaspth Add a layer of indirection making sanitizers pluggable. 427f3f9
Kasper Timm Hansen kaspth Remove deprecation notice. 017ddc6
Kasper Timm Hansen kaspth Remove html-scanner and its tests. 33019a3
Kasper Timm Hansen kaspth Point gems to all the right places. 33c8bfc
Kasper Timm Hansen kaspth Revert some stuff to use the new sanitizers. d4cd7e2
Kasper Timm Hansen kaspth Change sanitizer_vendor to just be a method and reword documentation. e438c09
Kasper Timm Hansen kaspth Don't splat arguments to allowed tags or attributes. bcd71b4
Kasper Timm Hansen kaspth Fix invalid css selectors in form_collections_helper_test.rb. 01ff0f3
Kasper Timm Hansen kaspth Change date helper tests to expect attributes with double quoted stri…
…ngs.
cdf2f28
Kasper Timm Hansen kaspth Make output_buffers used in tests be utf-8 encoded. Fixing unknown en…
…coding ASCII-8BIT test errors.
6cb6290
Kasper Timm Hansen kaspth Add document_root_element to ActionDispatch::IntegrationTest so asser…
…t_select can be called without specifying a root.
5ffc36d
Kasper Timm Hansen kaspth Use 1.9 syntax. 1a8ca9f
Kasper Timm Hansen kaspth Remove response faking. b276108
Kasper Timm Hansen kaspth Inline Assertion reference. 9c9875b
Kasper Timm Hansen kaspth Restore delivery method on teardowns. ff1b7e7
Rafael Mendonça França

I merged this branch on the rails/rails loofah branch. I'll test this branch with some applications I have and will test the deprecated gems too. If you have something to change on this branch please open PRs against this new branch.

@kaspth awesome work! :heart:

Richard Schneeman
Collaborator

epic!

Kasper Timm Hansen
Collaborator
Godfrey Chan
Owner

:metal:

Vipul A M

Awesome! Great work @kaspth !

Kasper Timm Hansen kaspth deleted the branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.