Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add azure module/monitor metricset #13196

Merged
merged 37 commits into from
Sep 23, 2019
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
d5bbbc4
Progress commit
narph May 22, 2019
52f709e
Progress on azure module
narph Jun 12, 2019
774b609
Staging
narph Jun 19, 2019
cbf3f77
Progres on POC
narph Jun 24, 2019
bb47fd2
Work on client
narph Jun 24, 2019
d5936c3
Update event format, implement refresh interval
narph Jul 4, 2019
c2e79b7
Merge branch 'master' into azure-module
narph Aug 2, 2019
6dc4433
Work on importing dependencies
narph Aug 8, 2019
6a081f7
Fix houndbot notifications
narph Aug 8, 2019
b6514c5
Start work on unittests
narph Aug 14, 2019
45dde4d
Work on unittests
narph Aug 14, 2019
b257446
Work on unittests
narph Aug 14, 2019
9fdd3dd
Adding unittests
narph Aug 15, 2019
5b686b0
Fixing issues
narph Aug 20, 2019
1b32171
Generate fields
narph Aug 20, 2019
2bed2a2
Small changes
narph Aug 21, 2019
69bdb22
Merge branch 'master' into azure-module
narph Aug 21, 2019
c6b8e23
Work on removing double records
narph Aug 23, 2019
ddd3b27
Work on error handling and regenrate files
narph Aug 28, 2019
0f1b733
Rename system.yml
narph Aug 28, 2019
3e8662c
Working on build
narph Aug 28, 2019
7cfb9b9
Work on build
narph Aug 28, 2019
587a3f6
Fmt
narph Aug 29, 2019
c712eba
generate notice
narph Aug 29, 2019
5676fa9
Work on unittests
narph Sep 2, 2019
3b26aaf
Adding 2 new metricsets
narph Sep 16, 2019
dbba93a
Work on grouping metric values per event
narph Sep 17, 2019
1294bb2
Work on handling errors
narph Sep 17, 2019
426cfe8
Work on tests
narph Sep 18, 2019
0da8ac7
run fmt update
narph Sep 18, 2019
e746103
Merge branch 'master' into azure-module
narph Sep 18, 2019
6a075e3
Generate notice
narph Sep 18, 2019
a17d73e
Work on monitor service
narph Sep 18, 2019
d001e84
Work on failing test
narph Sep 19, 2019
e1f19f2
Wotk on test
narph Sep 19, 2019
bd47d97
Work on feedback
narph Sep 20, 2019
beac3e6
Work on event format
narph Sep 20, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 129 additions & 2 deletions NOTICE.txt
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,15 @@ License type (autodetected): Apache-2.0
Apache License 2.0


--------------------------------------------------------------------
Dependency: contrib.go.opencensus.io/exporter/ocagent
Revision: 8110e6c0236bb231b19119275a6be6ec666d05c8
License type (autodetected): Apache-2.0
./vendor/contrib.go.opencensus.io/exporter/ocagent/LICENSE:
--------------------------------------------------------------------
Apache License 2.0


--------------------------------------------------------------------
Dependency: github.com/aerospike/aerospike-client-go
Revision: 0f3b54da6bdc2c31c505f9afbc5f434dd2089658
Expand Down Expand Up @@ -244,6 +253,30 @@ Apache License 2.0
GoFormation
Copyright 2011-2017 Amazon.com, Inc. or its affiliates. All Rights Reserved.

--------------------------------------------------------------------
Dependency: github.com/Azure/azure-sdk-for-go
Revision: 7a9d2769e4a581b0b1bc609c71b59af043e05c98
License type (autodetected): Apache-2.0
./vendor/github.com/Azure/azure-sdk-for-go/LICENSE:
--------------------------------------------------------------------
Apache License 2.0

-------NOTICE-----
Microsoft Azure-SDK-for-Go
Copyright 2014-2017 Microsoft

This product includes software developed at
the Microsoft Corporation (https://www.microsoft.com).

--------------------------------------------------------------------
Dependency: github.com/Azure/go-autorest
Revision: ba1147dc57f993013ef255c128ca1cac8a557409
License type (autodetected): Apache-2.0
./vendor/github.com/Azure/go-autorest/LICENSE:
--------------------------------------------------------------------
Apache License 2.0


--------------------------------------------------------------------
Dependency: github.com/bsm/sarama-cluster
Revision: 7e67d87a6b3f83fe08c096fd084691bd9dca112f
Expand Down Expand Up @@ -273,6 +306,15 @@ CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

--------------------------------------------------------------------
Dependency: github.com/census-instrumentation/opencensus-proto
Revision: 26aa36c099c2041b432cf3cc8a26c5fb858d218b
License type (autodetected): Apache-2.0
./vendor/github.com/census-instrumentation/opencensus-proto/LICENSE:
--------------------------------------------------------------------
Apache License 2.0


--------------------------------------------------------------------
Dependency: github.com/coreos/bbolt
Revision: af9db2027c98c61ecd8e17caa5bd265792b9b9a2
Expand Down Expand Up @@ -441,6 +483,21 @@ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

--------------------------------------------------------------------
Dependency: github.com/dgrijalva/jwt-go
Revision: 5e25c22bd5d6de03265bbe5462dcd162f85046f6
License type (autodetected): MIT
./vendor/github.com/dgrijalva/jwt-go/LICENSE:
--------------------------------------------------------------------
Copyright (c) 2012 Dave Grijalva

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


--------------------------------------------------------------------
Dependency: github.com/digitalocean/go-libvirt
Revision: 59d541f19311883ad82708651353009fb207d8a9
Expand All @@ -450,6 +507,15 @@ License type (autodetected): Apache-2.0
Apache License 2.0


--------------------------------------------------------------------
Dependency: github.com/dimchansky/utfbom
Revision: d2133a1ce379ef6fa992b0514a77146c60db9d1c
License type (autodetected): Apache-2.0
./vendor/github.com/dimchansky/utfbom/LICENSE:
--------------------------------------------------------------------
Apache License 2.0


--------------------------------------------------------------------
Dependency: github.com/dlclark/regexp2
Revision: 7632a260cbaf5e7594fc1544a503456ecd0827f1
Expand Down Expand Up @@ -1594,8 +1660,7 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

--------------------------------------------------------------------
Dependency: github.com/golang/protobuf
Version: v1.3.1
Revision: b5d812f8a3706043e23a9cd5babf2e5423744d30
Revision: 6c65a5562fc06764971b7c5d05c76c75e84bdbf7
License type (autodetected): BSD-3-Clause
./vendor/github.com/golang/protobuf/LICENSE:
--------------------------------------------------------------------
Expand Down Expand Up @@ -1839,6 +1904,40 @@ License type (autodetected): Apache-2.0
Apache License 2.0


--------------------------------------------------------------------
Dependency: github.com/grpc-ecosystem/grpc-gateway
Revision: d63917fcb0d53f39184485b9b6a0893af18a5668
License type (autodetected): BSD-3-Clause
./vendor/github.com/grpc-ecosystem/grpc-gateway/LICENSE.txt:
--------------------------------------------------------------------
Copyright (c) 2015, Gengo, Inc.
All rights reserved.

Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.

* Neither the name of Gengo, Inc. nor the names of its
contributors may be used to endorse or promote products derived from this
software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

--------------------------------------------------------------------
Dependency: github.com/hashicorp/go-uuid
Revision: 4f571afc59f3043a65f8fe6bf46d887b10a01d43
Expand Down Expand Up @@ -3058,6 +3157,34 @@ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


--------------------------------------------------------------------
Dependency: github.com/mitchellh/go-homedir
Revision: af06845cf3004701891bf4fdb884bfe4920b3727
License type (autodetected): MIT
./vendor/github.com/mitchellh/go-homedir/LICENSE:
--------------------------------------------------------------------
The MIT License (MIT)

Copyright (c) 2013 Mitchell Hashimoto

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

--------------------------------------------------------------------
Dependency: github.com/mitchellh/hashstructure
Revision: ab25296c0f51f1022f01cd99dfb45f1775de8799
Expand Down
108 changes: 108 additions & 0 deletions metricbeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ grouped in the following categories:
* <<exported-fields-aerospike>>
* <<exported-fields-apache>>
* <<exported-fields-aws>>
* <<exported-fields-azure>>
* <<exported-fields-beat-common>>
* <<exported-fields-beat>>
* <<exported-fields-ceph>>
Expand Down Expand Up @@ -2116,6 +2117,113 @@ type: keyword

--

[[exported-fields-azure]]
== azure fields

azure module



[float]
=== azure




[float]
=== monitor

monitor



[float]
=== resource

The resource specified



*`azure.monitor.resource.name`*::
+
--
The name of the resource


type: keyword

--

*`azure.monitor.resource.type`*::
+
--
The type of the resource


type: keyword

--

*`azure.monitor.resource.group`*::
+
--
The resource group


type: keyword

--

*`azure.monitor.resource.tags.*`*::
+
--
Azure resource tags.


type: object

--

*`azure.monitor.namespace`*::
+
--
The namespace selected


type: keyword

--

*`azure.monitor.subscription_id`*::
+
--
The subscription ID


type: keyword

--

*`azure.monitor.metrics.*.*`*::
+
--
Metrics returned.


type: object

--

*`azure.monitor.dimensions.*`*::
+
--
Azure metric dimensions.


type: object

--

[[exported-fields-beat-common]]
== Beat fields

Expand Down
42 changes: 42 additions & 0 deletions metricbeat/docs/modules/azure.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
////
This file is generated! See scripts/mage/docs_collector.go
////

[[metricbeat-module-azure]]
[role="xpack"]
== azure module

beta[]

This is the azure module.



[float]
=== Example configuration

The azure module supports the standard configuration options that are described
in <<configuration-metricbeat>>. Here is an example configuration:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More than a big example it may be good to have smaller examples with more explanations in the docs. We may revisit docs in follow-ups in any case.


[source,yaml]
----
metricbeat.modules:
- module: azure
metricsets: ["monitor"]
enabled: true
period: 60s
client_id: '${AZURE_CLIENT_ID:""}'
client_secret: '${AZURE_CLIENT_SECRET:""}'
tenant_id: '${AZURE_TENANT_ID:""}'
subscription_id: '${AZURE_SUBSCRIPTION_ID:""}'
----

[float]
=== Metricsets

The following metricsets are available:

* <<metricbeat-metricset-azure-monitor,monitor>>

include::azure/monitor.asciidoc[]

24 changes: 24 additions & 0 deletions metricbeat/docs/modules/azure/monitor.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
////
This file is generated! See scripts/mage/docs_collector.go
////

[[metricbeat-metricset-azure-monitor]]
=== azure monitor metricset

beta[]

include::../../../../x-pack/metricbeat/module/azure/monitor/_meta/docs.asciidoc[]

This is a default metricset. If the host module is unconfigured, this metricset is enabled by default.

==== Fields

For a description of each field in the metricset, see the
<<exported-fields-azure,exported fields>> section.

Here is an example document generated by this metricset:

[source,json]
----
include::../../../../x-pack/metricbeat/module/azure/monitor/_meta/data.json[]
----
3 changes: 3 additions & 0 deletions metricbeat/docs/modules_list.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ This file is generated! See scripts/mage/docs_collector.go
|<<metricbeat-metricset-aws-s3_daily_storage,s3_daily_storage>> beta[]
|<<metricbeat-metricset-aws-s3_request,s3_request>> beta[]
|<<metricbeat-metricset-aws-sqs,sqs>> beta[]
|<<metricbeat-module-azure,azure>> beta[] |image:./images/icon-no.png[No prebuilt dashboards] |
.1+| .1+| |<<metricbeat-metricset-azure-monitor,monitor>> beta[]
|<<metricbeat-module-beat,Beat>> |image:./images/icon-no.png[No prebuilt dashboards] |
.2+| .2+| |<<metricbeat-metricset-beat-state,state>>
|<<metricbeat-metricset-beat-stats,stats>>
Expand Down Expand Up @@ -197,6 +199,7 @@ This file is generated! See scripts/mage/docs_collector.go
include::modules/aerospike.asciidoc[]
include::modules/apache.asciidoc[]
include::modules/aws.asciidoc[]
include::modules/azure.asciidoc[]
include::modules/beat.asciidoc[]
include::modules/ceph.asciidoc[]
include::modules/cockroachdb.asciidoc[]
Expand Down
Loading