Framework to support exporters from configuration file

[ading1977]

This pull request is the initial implementation of a general framework to support Prometheus exporters from configuration file. The appMetric component will now construct EntityMetric based on the following information defined in the configuration:

• The exporter name
• The entity type
• The metric type
• The metric kind (used or capacity) and its corresponding prometheus query
• The entity attributes

The following is an example of the configuration for mysql exporter:

  - name: mysql
    entities:
      - type: APPLICATION
        metrics:
          DB_MEM:
            queries:
              used: 'mysql_global_status_innodb_buffer_pool_bytes_data{job="mysql"}'
              capacity: 'mysql_global_variables_innodb_buffer_pool_size{job="mysql"}'
          DB_CACHE_HIT_RATE:
            queries:
              used: '1/(1 + delta(mysql_global_status_innodb_buffer_pool_reads{job="mysql"}[10m])/rate(mysql_global_status_innodb_buffer_pool_read_requests[10m]))'
          CONNECTION:
            queries:
              used: 'mysql_global_status_threads_connected{job="mysql"}'
              capacity: 'mysql_global_variables_max_connections{job="mysql"}'
          TRANSACTION:
            queries:
              used: 'sum(rate(mysql_global_status_commands_total{command=~"(commit|rollback)"}[5m])) without (command)'
        attributes:
          ip:
            label: instance
            matches: \d{1,3}(?:\.\d{1,3}){3}(?::\d{1,5})??
            isIdentifier: true
          turboVersion:
            label: turbo_version
          node:
            label: node
          target:
            label: job

The following is the sample output:

{
    uid: "10.10.174.140", 
    type: 33,
    labels: {
        ip: "10.10.174.140",
        node: "ux1.foundation.eng.turbonomic.com",
        target: "mysql",
        turboVersion: "6.2.1-20180928133442000"
    },
    metrics: {
        44: {
            capacity: 151,
            used: 1
        },
        49: {
            used: 0
        },
        67: {
            capacity: 1073741824,
            used: 1073692672
        },
        69: {
            used: 0.04033115230470693
        }
    }
},

TODO:

• Add detailed documentation/comments to the public data structures and methods
• Parallelize prometheus queries with go-routines
• Add unit tests
• More extensive tests

This pull request also adds turbo custom metrics definition for istio 1.4.

[dblinn]

Nice work! Looks like there will be a good amount of integration effort on my part though haha. It would be good to increase our test coverage at some point though.

[dblinn]

Maybe document somewhere that the "used" query is mandatory?

[ading1977]

Will do

[dblinn]

Maybe document that every attributes section must include exactly one identifier?

[ading1977]

Will do

[ading1977]

Thanks for the review @dblinn. As a general comment, it is in my plan to add complete documentation and test cases in separate PRs.