blacklist not blacklisting API calls

[PhilCoggins]

I am trying to block all API requests to google from my selenium browser session using browsermob. Below is my code:

Capybara.register_driver :selenium_without_google do |app|
  filename = File.join(Rails.root, "lib", "browsermob", "bin", "browsermob-proxy")

  server = BrowserMob::Proxy::Server.new(filename) #=> #<BrowserMob::Proxy::Server:0x000001022c6ea8 ...>
  server.start

  proxy = server.create_proxy #=> #<BrowserMob::Proxy::Client:0x0000010224bdc0 ...>
  proxy.blacklist(/google/, '200')

  profile = Selenium::WebDriver::Firefox::Profile.new #=> #<Selenium::WebDriver::Firefox::Profile:0x000001022bf748 ...>
  profile.proxy = proxy.selenium_proxy
  Capybara::Selenium::Driver.new(app, :browser => :firefox, :profile => profile)
end

It is still sending out the API calls. Any direction you can give me on this?

[jarib]

Does it work if you pass the status code as an integer instead of a string?

[PhilCoggins]

Nope.

[abotalov]

If API calls are sent via HTTPS you should use:

proxy.selenium_proxy(:http, :ssl)

[PhilCoggins]

@abotalov changed, still making the API calls...

[jarib]

How do you check?

[PhilCoggins]

@jarib using both charles for OSX and tcpdump -v | grep google

[jarib]

Indeed, I can reproduce this. Even navigating to google.com works well. Not sure what's up here since our specs are passing.

[PhilCoggins]

I would be happy to help with this if at all possible, but I really don't know the first thing about proxy implementations, or even selenium for that matter. Any recommendations for a starting point?

[jarib]

First thing would be to figure out how /google/ is different from what's happening in our end-to-end tests for blacklisting - e.g. is it because the specs are testing against a localhost server, no SSL, something else?

The specs are just Ruby - feel free to dig into it.

[PhilCoggins]

Which version of Ruby are you guys building off of? I get 17 failures (including the blacklist specs) in Ruby 2.1.0

EDIT Scratch that, was not setting BROWSERMOB_PROXY_HOME, all pass now.

[PhilCoggins]

Yeah, the example that disallows access to URLs from blacklist is bunk.

The below passes:

    it "disallows access to urls in blacklist" do
      proxy.new_har('blacklist')
      dest = url_for('https://www.ruby-lang.org/en/')
      proxy.blacklist(Regexp.quote('google.com'), 404)
      driver.get dest
      proxy.har.entries.first.response.status.should == 404
    end

Will see what I can find out tonight.

[jarib]

That fails because because url_for takes a path on the local server, not a full URL. Your example attempts to open "http://0.0.0.0:3001/https://www.ruby-lang.org/en/", which of course 404s.

Anyway, I've changed the relevant specs to make sure that they pull the correct entry from the HAR, and they still pass.