Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Silk integration #10783

Merged
merged 82 commits into from
Feb 17, 2022
Merged

Add Silk integration #10783

merged 82 commits into from
Feb 17, 2022

Conversation

ChristineTChen
Copy link
Contributor

@ChristineTChen ChristineTChen commented Dec 2, 2021 •
edited by yzhan289
Loading

What does this PR do?

This PR adds the Silk integration

Motivation

Additional Notes

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • PR title must be written as a CHANGELOG entry (see why)
  • Files changes must correspond to the primary purpose of the PR as described in the title (small unrelated changes should have their own PR)
  • PR must have changelog/ and integration/ labels attached

@ChristineTChen ChristineTChen requested review from a team as code owners December 2, 2021 19:02
@ghost ghost added the documentation label Dec 2, 2021
@github-actions
Copy link

github-actions bot commented Dec 2, 2021

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

1 similar comment
@github-actions
Copy link

github-actions bot commented Dec 2, 2021

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@ghost ghost added the dev/testing label Dec 2, 2021
@github-actions
Copy link

github-actions bot commented Dec 2, 2021

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

3 similar comments
@github-actions
Copy link

github-actions bot commented Dec 2, 2021

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@github-actions
Copy link

github-actions bot commented Dec 2, 2021

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@github-actions
Copy link

github-actions bot commented Dec 3, 2021

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@codecov
Copy link

codecov bot commented Dec 3, 2021 •
edited
Loading

Codecov Report

Merging #10783 (11e79e6) into master (4cd0620) will decrease coverage by 6.23%.
The diff coverage is 93.18%.

Flag Coverage Δ
active_directory ?
activemq_xml ?
aerospike ?
airflow ?
amazon_msk 88.83% <ø> (ø)
ambari ?
apache ?
aspdotnet ?
avi_vantage ?
azure_iot_edge ?
btrfs ?
cacti ?
cassandra_nodetool ?
ceph ?
cilium ?
cisco_aci ?
citrix_hypervisor ?
clickhouse ?
cloud_foundry_api ?
cockroachdb ?
consul ?
coredns ?
couch ?
couchbase ?
crio ?
datadog_checks_base ?
datadog_checks_dev ?
datadog_checks_downloader ?
datadog_cluster_agent ?
directory ?
disk ?
dns_check ?
dotnetclr ?
druid ?
ecs_fargate ?
eks_fargate ?
elastic ?
envoy ?
etcd ?
exchange_server ?
external_dns ?
fluentd ?
gearmand ?
gitlab ?
gitlab_runner ?
glusterfs ?
go_expvar ?
gunicorn ?
haproxy ?
harbor ?
hazelcast ?
hdfs_datanode ?
hdfs_namenode ?
http_check ?
ibm_db2 ?
ibm_i ?
ibm_mq ?
ibm_was ?
iis ?
istio ?
kafka_consumer ?
kong ?
kube_apiserver_metrics ?
kube_controller_manager ?
kube_dns ?
kube_metrics_server ?
kube_proxy ?
kube_scheduler ?
kubelet ?
kubernetes_state ?
kyototycoon ?
lighttpd ?
linkerd ?
linux_proc_extras ?
mapr ?
mapreduce ?
marathon ?
marklogic ?
mcache ?
mesos_master ?
mesos_slave 93.63% <ø> (ø)
mongo ?
mysql ?
nagios ?
network ?
nfsstat ?
nginx ?
nginx_ingress_controller 98.36% <ø> (ø)
openldap ?
openmetrics 97.14% <ø> (ø)
openstack 51.45% <ø> (ø)
openstack_controller 90.74% <ø> (ø)
oracle ?
pdh_check 95.65% <ø> (ø)
pgbouncer ?
php_fpm ?
postfix ?
postgres ?
powerdns_recursor ?
process ?
prometheus ?
proxysql ?
rabbitmq ?
redisdb ?
rethinkdb ?
riak ?
riakcs ?
sap_hana ?
scylla ?
silk 93.18% <93.18%> (?)
singlestore ?
snmp ?
snowflake ?
sonarqube ?
spark ?
sqlserver ?
squid ?
ssh_check ?
statsd ?
supervisord ?
system_core ?
system_swap ?
tcp_check ?
teamcity ?
tls ?
twemproxy ?
twistlock ?
varnish ?
vault ?
vertica ?
voltdb ?
vsphere ?
win32_event_log ?
windows_performance_counters ?
windows_service ?
wmi_check ?
yarn ?
zk ?

Flags with carried forward coverage won't be shown. Click here to find out more.

@github-actions
Copy link

github-actions bot commented Dec 3, 2021

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@ChristineTChen
Copy link
Contributor Author

/azp run PR changes

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@github-actions
Copy link

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

11 similar comments
@github-actions
Copy link

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@github-actions
Copy link

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@github-actions
Copy link

github-actions bot commented Jan 6, 2022

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@github-actions
Copy link

github-actions bot commented Jan 6, 2022

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@github-actions
Copy link

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@github-actions
Copy link

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@github-actions
Copy link

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@github-actions
Copy link

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@github-actions
Copy link

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@github-actions
Copy link

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

@github-actions
Copy link

The validations job has failed; please review the Files changed tab for possible suggestions to resolve.

sarah-witt
sarah-witt previously approved these changes Feb 15, 2022
View reviewed changes
ofek
ofek requested changes Feb 16, 2022
View reviewed changes
silk/assets/configuration/spec.yaml Outdated Show resolved Hide resolved
silk/assets/configuration/spec.yaml Outdated Show resolved Hide resolved
silk/datadog_checks/silk/check.py Outdated Show resolved Hide resolved
silk/datadog_checks/silk/check.py Outdated Show resolved Hide resolved
silk/datadog_checks/silk/check.py Outdated Show resolved Hide resolved
silk/datadog_checks/silk/metrics.py Outdated Show resolved Hide resolved
silk/datadog_checks/silk/events.py Outdated Show resolved Hide resolved
silk/datadog_checks/silk/events.py Outdated Show resolved Hide resolved
@yzhan289 yzhan289 requested a review from ofek February 16, 2022 20:41
ofek
ofek requested changes Feb 16, 2022
View reviewed changes
silk/datadog_checks/silk/constants.py Outdated Show resolved Hide resolved
silk/datadog_checks/silk/constants.py Outdated Show resolved Hide resolved
silk/datadog_checks/silk/events.py Outdated Show resolved Hide resolved
silk/datadog_checks/silk/check.py Outdated Show resolved Hide resolved
ofek
ofek approved these changes Feb 16, 2022
View reviewed changes
Copy link
Contributor

@ofek ofek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

fanny-jiang
fanny-jiang approved these changes Feb 16, 2022
View reviewed changes
Copy link
Contributor

@fanny-jiang fanny-jiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

@yzhan289 yzhan289 merged commit bd217c5 into master Feb 17, 2022
@yzhan289 yzhan289 deleted the cc/silk branch February 17, 2022 17:35
github-actions bot pushed a commit that referenced this pull request Feb 17, 2022
@yzhan289
Add Silk integration (#10783)
87dc6b1 
* New integration: Silk

* Add e2e environment with mocked endpoints

* Fix docker e2e env

* Add configuration options

* Add to CI

* Add check logic

* User server instead

* Fix api format

* Create Metrics class and parse metrics

* Fix state

* Comment

* Add short desc

* Update

* Avoid redundant dictionary modification

* Add additional metrics

* Comment test out

* Add additional system metrics and test

* Add events

* Make service checks more verbose, remove port and use host_address

* Add metrics metadata

* Fix url in test env, clean up sc

* Document service checks

* Update e2e mock responses

* Add metrics to assert

* Add e2e test

* FIx style

* Add event support for Silk (#10818)

* refactor service check

* Move test fixture to fixture dir

* Clarify connect sc from state sc

* Rename get_metrics to get_data

* Support version metadata

* Change from datetime to timestamp (#10877)

* Add silk host tag to service checks

* Add bs and rw breakdown

* Support replication stats

* Fix test

* Fix metadata

* Add units

* Clean up

* Refactor error msg and server state service checks

* Refactor

* Remove resolution metric

* Fix tests

* Fix test from get_data refactor

* Fix format

* clean up

* Add new blocksize metrics

* document block size metrics

* Extract field to name

* Add recommended monitor and fix rw volumes endpoint

* Add config options for blocksize and read write stats

* Update README

* Move recommended monitors

* Fix blocksize and readwrite config option

* Update description for service check

* Move blocksize and readwrite options up

* Fix silk tests to account for blocksize and read/write metrics

* Fix tests for CI

* Update e2e tests

* Import metrics

* Fix CI

* Validate config

* Rename events.json fixture to fix CI

* Update silk/README.md

Co-authored-by: Austin Lai <76412946+alai97@users.noreply.github.com>

* Apply suggestions

* Remove empty silk dashboard until dashboard is ready

* Add note about e2e testing

* Apply suggestions

* Apply suggestions

* Sync validations

* Add test for optional metrics

* Fix test

* Address comments

* Add  file

* Add curated_metric section

* Remove requirements.in

* Address comments

* Fix style

* Address comments

* Fix style

* Address comments

Co-authored-by: Andrew Zhang <31313038+yzhan289@users.noreply.github.com>
Co-authored-by: Andrew Zhang <yzhan289@jhu.edu>
Co-authored-by: Andrew Zhang <andrew.zhang@datadoghq.com>
Co-authored-by: Austin Lai <76412946+alai97@users.noreply.github.com> bd217c5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ofek ofek ofek approved these changes

@fanny-jiang fanny-jiang fanny-jiang approved these changes

@sarah-witt sarah-witt sarah-witt approved these changes

@alai97 alai97 alai97 left review comments

@yzhan289 yzhan289 Awaiting requested review from yzhan289

Assignees
No one assigned
Labels
dev/testing documentation integration/silk
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@ChristineTChen @ofek @fanny-jiang @yzhan289 @sarah-witt @alai97