Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

updated dd-hander chef-handler to use the web_proxy from agent config #208

Closed
wants to merge 2 commits into from

Conversation

Projects
None yet
2 participants
@datwiz
Copy link
Contributor

commented May 29, 2015

Updated dd-handler chef-handler to use web_proxy attributes, if set.
Enables the datadog handler to post via the configured proxy without requiring http_proxy env var's to be set prior to the chef-client run.

@miketheman

This comment has been minimized.

Copy link
Collaborator

commented May 29, 2015

Awesome stuff! Thanks for the contribution.

Do you know if using this pattern, the environment variable persists for other applications/services that might look for this variable during a Chef run? Env cars can be considered 'global' state.

@datwiz

This comment has been minimized.

Copy link
Contributor Author

commented May 29, 2015

I suspect it does as I had to place the web_proxy settings after the chef_gem section. Otherwise the gem install also tries to use the http_proxy setting and fails to connect to our local gem server.

@miketheman

This comment has been minimized.

Copy link
Collaborator

commented May 29, 2015

Ok, so I can confirm that setting the envvar during a Chef run will persist only for the duration of the Chef run - not beyond, so any application running outside of Chef's environment won't have this setting.

However, this has the unintended behavior of then using the same environment variable for resources like remote_file, one resource that uses HTTP_PROXY (even though it recommends against it).

$ chef-shell
...
 Chef Version: 11.16.4
...
chef > recipe_mode
chef:recipe > ENV['HTTP_PROXY'] = 'https://foobar.com'
 => "https://foobar.com"
chef:recipe > remote_file '/tmp/.rspec' do
chef:recipe >   source 'https://raw.githubusercontent.com/DataDog/chef-datadog/master/.rspec'
chef:recipe ?> end
 => <remote_file[/tmp/.rspec] @name: "/tmp/.rspec" @noop: nil @before: nil @params: {} @provider: Chef::Provider::RemoteFile @allowed_actions: [:nothing, :create, :delete, :touch, :create_if_missing] @action: "create" @updated: false @updated_by_last_action: false @supports: {} @ignore_failure: false @retries: 0 @retry_delay: 2 @source_line: "(irb#1):2:in `irb_binding'" @guard_interpreter: :default @elapsed_time: 0 @sensitive: false @resource_name: :remote_file @path: "/tmp/.rspec" @backup: 5 @atomic_update: true @force_unlink: false @manage_symlink_source: nil @diff: nil @source: ["https://raw.githubusercontent.com/DataDog/chef-datadog/master/.rspec"] @use_etag: true @use_last_modified: true @ftp_active_mode: false @headers: {} @cookbook_name: nil @recipe_name: nil>
chef:recipe > run_chef
[2015-05-29T13:32:05+00:00] INFO: Processing remote_file[/tmp/.rspec] action create ((irb#1) line 2)
[2015-05-29T13:32:05+00:00] DEBUG: touching /tmp/.rspec to create it
[2015-05-29T13:32:05+00:00] INFO: remote_file[/tmp/.rspec] created file /tmp/.rspec
[2015-05-29T13:32:05+00:00] DEBUG: remote_file[/tmp/.rspec] checking for changes
[2015-05-29T13:32:05+00:00] INFO: Unable to access cache at /var/chef. Switching cache to /home/michael/.chef
[2015-05-29T13:32:05+00:00] DEBUG: Cache control headers: {}
[2015-05-29T13:32:05+00:00] DEBUG: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request
[2015-05-29T13:32:05+00:00] DEBUG: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request
[2015-05-29T13:32:05+00:00] DEBUG: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request
[2015-05-29T13:32:05+00:00] DEBUG: Initiating GET to https://raw.githubusercontent.com/DataDog/chef-datadog/master/.rspec
[2015-05-29T13:32:05+00:00] DEBUG: ---- HTTP Request Header Data: ----
[2015-05-29T13:32:05+00:00] DEBUG: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
[2015-05-29T13:32:05+00:00] DEBUG: ---- End HTTP Request Header Data ----
The environment variable HTTP_PROXY is discouraged.  Use http_proxy.
[2015-05-29T13:32:05+00:00] INFO: Unable to access cache at /var/chef. Switching cache to /home/michael/.chef
[2015-05-29T13:32:05+00:00] INFO: Unable to access cache at /var/chef. Switching cache to /home/michael/.chef
[2015-05-29T13:32:05+00:00] INFO: Unable to access cache at /var/chef. Switching cache to /home/michael/.chef
[2015-05-29T13:32:05+00:00] INFO: Unable to access cache at /var/chef. Switching cache to /home/michael/.chef

================================================================================
Error executing action `create` on resource 'remote_file[/tmp/.rspec]'
================================================================================

EOFError
--------
end of file reached

Resource Declaration:
---------------------


Compiled Resource:
------------------
# Declared in (irb#1):2:in `irb_binding'

remote_file("/tmp/.rspec") do
  provider Chef::Provider::RemoteFile
  action "create"
  retries 0
  retry_delay 2
  guard_interpreter :default
  path "/tmp/.rspec"
  backup 5
  atomic_update true
  source ["https://raw.githubusercontent.com/DataDog/chef-datadog/master/.rspec"]
  use_etag true
  use_last_modified true
end

[2015-05-29T13:32:05+00:00] INFO: Running queued delayed notifications before re-raising exception
EOFError: remote_file[/tmp/.rspec] ((irb#1) line 2) had an error: EOFError: end of file reached
    from /usr/lib/ruby/2.1.0/openssl/buffering.rb:182:in `read_nonblock'
    from /usr/lib/ruby/2.1.0/openssl/buffering.rb:182:in `sysread_nonblock'
    from /usr/lib/ruby/2.1.0/openssl/buffering.rb:182:in `read_nonblock'
    from /usr/lib/ruby/2.1.0/net/protocol.rb:153:in `rbuf_fill'
    from /usr/lib/ruby/2.1.0/net/protocol.rb:134:in `readuntil'
    from /usr/lib/ruby/2.1.0/net/protocol.rb:144:in `readline'
    from /usr/lib/ruby/2.1.0/net/http/response.rb:39:in `read_status_line'
    from /usr/lib/ruby/2.1.0/net/http/response.rb:28:in `read_new'
    from /usr/lib/ruby/2.1.0/net/http.rb:915:in `connect'
    from /usr/lib/ruby/2.1.0/net/http.rb:863:in `do_start'
    from /usr/lib/ruby/2.1.0/net/http.rb:852:in `start'
    from /usr/lib/ruby/2.1.0/net/http.rb:1375:in `request'
    from /var/lib/gems/2.1.0/gems/rest-client-1.6.7/lib/restclient/net_http_ext.rb:51:in `request'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/http/basic_client.rb:65:in `request'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/http.rb:260:in `block in send_http_request'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/http.rb:294:in `block in retrying_http_errors'
... 12 levels...
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/provider.rb:121:in `run_action'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/resource.rb:648:in `run_action'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/runner.rb:49:in `run_action'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/runner.rb:81:in `block (2 levels) in converge'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/runner.rb:81:in `each'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/runner.rb:81:in `block in converge'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/resource_collection.rb:98:in `block in execute_each_resource'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/resource_collection.rb:96:in `execute_each_resource'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/runner.rb:80:in `converge'
    from /var/lib/gems/2.1.0/gems/chef-11.16.4/lib/chef/shell/ext.rb:236:in `run_chef'
    from (irb#1):5chef:recipe >
 => :recipe

Now, in this test, there's no actual proxy server, so real behavior might differ. But the point remains that the use of the environment variable will likely affect other resources.

Can you think of another way to implement this change with something that doesn't affect global state of a Chef run

@datwiz

This comment has been minimized.

Copy link
Contributor Author

commented May 29, 2015

I did consider a safer alternate method, which is setting a new env var specifically for the DD web proxy, something like DD_WEB_PROXY.
I had thought the alternate method would require a mod to the dogapi-rb connection code, but digging a bit more, it may be possible make a mod in chef-handler-datadog to wrap the emit calls with a set/revert of an HTTP_PROXY env var that gets passed through into the dogapi methods.
I should probably also confirm the precedence of the ruby Net::HTTP/URI::Generic find_proxy method to make the first var gets picked. HTTPS_PROXY may get picked first.
I'll try out the chef-handler-datadog approach.

modified proxy settings for a handler specific env variable over the …
…more general HTTP_PROXY, which has side effects
@datwiz

This comment has been minimized.

Copy link
Contributor Author

commented Jul 26, 2015

Change has been modified to pass the datadog proxy information to the report handler as a new ENV variable 'DATADOG_PROXY'. A corresponding pull request has also been added to the chef-handler-datadog project to use the new proxy variable on chef handler report runs. The second change also preserves the pre-call values of the environment proxy settings.
Refer to issue DataDog/chef-handler-datadog#64.

@miketheman miketheman added the feature label Aug 19, 2015

@miketheman miketheman added this to the Next minor milestone Oct 30, 2015

@miketheman

This comment has been minimized.

Copy link
Collaborator

commented Apr 15, 2016

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.