https issues

[bHoskins07]

Hello,

I am attempting to recreate HTTPS request. When we issue curl request it looks like this...
curl --tlsv1.2 -n -k https://hostname:port/restendpoint

I configured httpbeat.yml, see below

   basic_auth:
     username: username
     password: password
#Optional ssl configuration. By default is off.
   ssl:
     #List of root certificates for HTTPS server verifications
     #certificate_authorities: ["/etc/pki/root/ca.pem"]

     #Certificate for TLS client authentication
     #certificate: "/etc/pki/client/cert.pem"

     # Client Certificate Key
     #key: "/etc/pki/client/cert.key"

     #This option controls whether the client verifies server certificates and host
     #names. The values `none` and `full` can be used. If `verification_mode` is set
     #to `none`, all server host names and certificates are accepted. In this mode,
     #TLS-based connections are susceptible to man-in-the-middle attacks. Use this
     #option for testing only.

     #The default is `full`.
     verification_mode: none

     #Configure cipher suites to be used for TLS connections
     #cipher_suites: []

     #Configure curve types for ECDHE based cipher suites
     #curve_types: []

     #List of allowed SSL/TLS versions. If SSL/TLS server decides for protocol versions
     #not configured, the connection will be dropped during or after the handshake. The
     #setting is a list of allowed protocol versions:
     #`SSLv3`, `TLSv1` for TLS version 1.0, `TLSv1.0`, `TLSv1.1` and `TLSv1.2`.

     #The default value is `[TLSv1.0, TLSv1.1, TLSv1.2]`.
     versions: [TLSv1.2]

I am sending directly to elasticsearch and also output to file but no data has being written to either.

I only get this INFO message in log.
2017/01/06 19:56:03.162631 logp.go:232: INFO No non-zero metrics in the last 30s

Any help would be appreciated!
Thanks

[bHoskins07]

Update on this issue. The error I am getting now is below:

2017/01/09 19:32:35.127750 poller.go:140: ERR An error occurred while executing HTTP request: [Get https://hostname:port/ws/v1/cluster/metrics: x509: certificate signed by unknown authority]

[christiangalsterer]

I will try to look into it on the next days/weekend.

[christiangalsterer]

After a very first glance, I think you need to configure either "certificate_authorities" to include the correct certificates or set "verification_mode" to "none". Setting "vverification_mode" to "none" is the same as using -k switch in curl.

[christiangalsterer]

@bHoskins07: Which version do you use?

[bHoskins07]

Thanks for the quick response.

elasticsearch version 5.1.1
httpbeat version 3.1.1

[bHoskins07]

Below is what I have set in httpbeat.yml file for input.

httpbeat:

  urls:
    # Each - URL endpoints to call. Below are the URL endpoint specific configurations
    -
      # Optional cron expression, defines when to poll the URL endpoint.
      # Default is every 1 minute.
      cron: "@every 5s"

      # The URL endpoint to call by Httpbeat
      url: https://hostname:password/ws/v1/cluster/metrics

      # HTTP method to use.
      # Possible options are:
      # * get
      # * delete
      # * head
      # * patch
      # * post
      # * put
      method: get

      # Optional basic authentication
      basic_auth:
        #Basic authentication username
        username: username
        #Basic authentication password
        password: password

      # Type to be published in the 'type' field. For Elasticsearch output,
      # the type defines the document type these entries should be stored
      # in. Default: httpbeat
      document_type: httpbeat

      # Optional output format for the response body.
      # Possible options are:
      # * string
      # * json
      # Default output format is 'string'
      #output_format: json

      # Optional additional headers to send to the endpoint
      #headers:
      #  Accept: application/json

  ssl:

      # Configure SSL verification mode. If `none` is configured, all server hosts
      # and certificates will be accepted. In this mode, SSL based connections are
      # susceptible to man-in-the-middle attacks. Use only for testing. Default is
      # `full`.
      verification_mode: none

[Mrc0113]

I'm having a similar issue. Is ssl even supported on the polling piece of this? The ssl configurations seem to be only available in the output sections of the full example config: https://github.com/christiangalsterer/httpbeat/blob/master/httpbeat.full.yml

[MarkSonghurst]

FYI Some comments in #15 from me about using SSL.

[Mrc0113]

As a quick test on my local machine I set InsecureSkipVerify = true in func convertTLSConfig(config *transport.TLSConfig) and it acted as I expected "verification_mode: none" to act. Not sure if you'd want to expose that configuration option instead? I'll leave it up to you.

https://golang.org/src/crypto/tls/common.go

// InsecureSkipVerify controls whether a client verifies the

331 // server's certificate chain and host name.
332 // If InsecureSkipVerify is true, TLS accepts any certificate
333 // presented by the server and any host name in that certificate.
334 // In this mode, TLS is susceptible to man-in-the-middle attacks.
335 // This should be used only for testing.
336 InsecureSkipVerify bool

[christiangalsterer]

I fixed the issue and will cut a release soon.

[christiangalsterer]

Release 3.2.0 now available

[Mrc0113]

Thanks for the timely fix & release. Works for me

[bHoskins07]

@christiangalsterer Yes, Thank You For the Fix. This worked for me as well. You are the best! 🥇 💯

[christiangalsterer]

Always glad to help...