http_proxy does not work in some cases like remote_file

[esciara]

Hi there.

Looks like there are scenarios where http_proxy does not quiet work as mentioned in #6 ...

The cookbook I am working on is downloading a large binary file (200+ megs) and remote_file is used to download it from the website (sample url: http://www.atlassian.com/software/confluence/downloads/binary/atlassian-confluence-5.4.4-x64.bin). The code is as follows:

remote_file "#{Chef::Config[:file_cache_path]}/atlassian-confluence-#{node['confluence']['version']}-#{node['confluence']['arch']}.bin" do
  source node['confluence']['url']
  checksum node['confluence']['checksum']
  mode '0755'
  action :create_if_missing
end

I am using polipo and the Auto HTTP Caching Spike from @fnichol (nice work :) ) to run test-kitchen on an ubuntu-1204, but polipo's cache is not used for the file. It is used by apt installs in cookbooks les mysql or apache2, or other types of downloads like in the java cookbook, but not for the remote_file.

I tried with Vagrant. I adapted the spike to use in on the Vagrantfile in conjonction with vagrant-proxyconf:

  if proxy_running?
    if Vagrant.has_plugin?("vagrant-proxyconf")
      puts http_proxy_url
      puts "from Vagrantfile"
      config.proxy.http     = "#{http_proxy_url}/"
      config.proxy.https     = "#{http_proxy_url}/"
      config.proxy.no_proxy = "localhost,127.0.0.1"
    end
  end

There, caching works and the file is downloaded in no time.

I have also tried to port this for use with test-kitchen using the vagrantfile_erb config to have the same vagrant file used. Caching still does not take place for remote_file.

Any clues?

[esciara]

Thought I found my saviour in test-kitchen/test-kitchen#192, but this was overridden by test-kitchen/test-kitchen@6e30431 . Not quiet sure how it is supposed to work now.

[esciara]

Managed to get it to work.

Needed to add to Auto HTTP Caching Spike the following in the .kitchen.local.yml (ref: test-kitchen/test-kitchen@6e30431):

provisioner:
  chef_omnibus_url: http://www.getchef.com/chef/install.sh
  solo_rb:
    http_proxy: <%= http_proxy_url %>
    https_proxy: <%= http_proxy_url %>

After a few experiments, it looks like a combination of this and the use of the vagrant-cachier in the Vagrantfile.erb is the fastest solution, even though it looks like some apt calls to security.ubuntu.com:

  if Vagrant.has_plugin?("vagrant-cachier")
    # Configure cached packages to be shared between instances of the same base box.
    # More info on http://fgrehm.viewdocs.io/vagrant-cachier/usage
    c.cache.scope = :box
  end

Don't forget to add vagrantfile_erb in your kitchen.yml, or even better to your .kitchen.local.yml:

driver:
  [...]
  vagrantfile_erb: Vagrantfile.erb

Results of my tests:

Without any Vagrantfile.erb

-----> Kitchen is finished. (5m39.62s)

real    5m40.473s
user    0m8.651s
sys 0m2.358s

With vagrant-proxyconf in Vagrantfile.erb

-----> Kitchen is finished. (5m48.12s)

real    5m48.943s
user    0m9.268s
sys 0m2.608s

With vagrant-cachier in Vagrantfile.erb (after first run to have the files cached)

-----> Kitchen is finished. (3m46.02s)

real    3m46.860s
user    0m9.049s
sys 0m2.531s