feat: Azure Event Hubs output plugin #9346

tomconte · 2021-06-09T08:46:47Z

Required for all PRs:

Updated associated README.md.
Wrote appropriate unit tests.

This is a new output plugin for the Azure Event Hubs service. Event Hubs is a popular ingestion service and can be used to pipe data into various Azure streaming and batch analytics services.

The unit tests are minimal since the plugin is mostly a wrapper for the Event Hubs Go SDK. I have included an integration test, but it requires a real Event Hubs instance.

telegraf-tiger · 2021-06-09T08:46:51Z

Thanks so much for the pull request!
🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

telegraf-tiger · 2021-06-09T09:06:02Z

Thanks so much for the pull request!
🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

telegraf-tiger · 2021-06-09T10:01:28Z

Thanks so much for the pull request!
🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

tomconte · 2021-06-09T10:03:41Z

!signed-cla

tomconte · 2021-06-30T08:29:51Z

Hi folks, I think this PR is ready for review, I will be happy to address comments. Thanks!

sspaink

Thanks for making this new output plugin! I noticed there is a binary file called "mockery" that probably shouldn't have been committed? Minor code suggestion in the README.md but besides that looks good.

plugins/outputs/event_hubs/README.md

Co-authored-by: Sebastian Spaink <3441183+sspaink@users.noreply.github.com>

tomconte · 2021-09-01T11:04:03Z

Thanks @sspaink ! I made the changes, but not sure why the Windows test fails (I/O error).

sspaink · 2021-09-07T18:59:19Z

@tomconte thanks for addressing the requested changes, it looks like the windows test failed for unrelated reasons I re-triggered the build and it passed. There is a minor linter error that will need to get resolved before this can be merged.

telegraf-tiger · 2021-09-08T08:26:48Z

Looks like new artifacts were built from this PR. Get them here!

Artifact URLs

tomconte · 2021-09-09T09:08:31Z

Thanks @sspaink ! Looks like all green now :)

tomconte · 2021-09-21T08:05:58Z

Hi @sspaink, hope you are well! Any chance you could take another look? Thanks!

sspaink · 2021-10-14T20:32:49Z

@tomconte sorry for the delay in response, I think everything looks good and is ready to merge. If you can resolve the merge conflicts, then I will merge it. Thanks for working on this!

telegraf-tiger · 2021-10-18T09:47:42Z

📦 Looks like new artifacts were built from this PR.

Expand this list to get them here! 🐯

Artifact URLs

DEB	RPM	TAR GZ	ZIP
amd64.deb	aarch64.rpm	darwin_amd64.tar.gz	windows_amd64.zip
arm64.deb	armel.rpm	freebsd_amd64.tar.gz	windows_i386.zip
armel.deb	armv6hl.rpm	freebsd_armv7.tar.gz
armhf.deb	i386.rpm	freebsd_i386.tar.gz
i386.deb	ppc64le.rpm	linux_amd64.tar.gz
mips.deb	s390x.rpm	linux_arm64.tar.gz
mipsel.deb	x86_64.rpm	linux_armel.tar.gz
ppc64el.deb		linux_armhf.tar.gz
s390x.deb		linux_i386.tar.gz
		linux_mips.tar.gz
		linux_mipsel.tar.gz
		linux_ppc64le.tar.gz
		linux_s390x.tar.gz
		static_linux_amd64.tar.gz

tomconte · 2021-10-18T11:49:54Z

Thanks @sspaink, all done!

* origin/master: (176 commits) fix: Linter fixes for plugins/inputs/[h-j]* (influxdata#9986) fix(inputs/kube_inventory): don't skip resources with zero s/ns timestamps (influxdata#9978) fix: update gjson to v1.10.2 (influxdata#9998) fix: procstat tags were not getting generated correctly (influxdata#9973) chore: create bug report form (influxdata#9976) fix: Allow for non x86 macs in Go install script (influxdata#9982) test: add sqlserver plugin integration tests (influxdata#9943) feat: plugins/common/tls/config.go: Filter client certificates by DNS names (influxdata#9910) feat: add option to skip table creation in azure data explorer output (influxdata#9942) docs: update nightlies links (influxdata#9989) fix: add s390x to nightlies (influxdata#9990) feat: Add more details to processors.ifname logmessages (influxdata#9984) docs: Create SECURITY.md (influxdata#9951) fix: set NIGHTLY=1 for correctly named nightly artifacts (influxdata#9987) feat: Kafka Add metadata full to config (influxdata#9833) chore: Update to AWS SDK v2 (influxdata#9647) chore: lint ignore fmt.Printf unhandled error (influxdata#9967) fix: starlark pop operation for non-existing keys (influxdata#9954) feat: update etc/telegraf.conf and etc/telegraf_windows.conf (influxdata#9876) fix: Check return code of zfs command for FreeBSD. (influxdata#9956) chore: update go to 1.17.2 (influxdata#9873) fix: Graylog plugin TLS support and message format (influxdata#9862) docs: update README with info on package repos (influxdata#9964) feat: Modbus connection settings (serial) (influxdata#9256) fix: segfault in ingress, persistentvolumeclaim, statefulset in kube_inventory (influxdata#9585) fix: add normalization of tags for ethtool input plugin (influxdata#9901) chore: remove empty build.py (influxdata#9958) fix: internet_speed input plugin not collecting/reporting latency (influxdata#9957) chore: reference db2 external plugin (influxdata#9952) chore: update readme go version from 1.14 to 1.17 (influxdata#9944) fix: decode Prometheus scrape path from Kuberentes labels (influxdata#9662) docs: fix broken link (influxdata#9812) fix: Correct conversion of int with specific bit size (influxdata#9933) fix: update golanci-lint to v1.42.1 (influxdata#9932) feat: Azure Event Hubs output plugin (influxdata#9346) feat: more fields for papertrail event webhook (influxdata#9940) fix: solve compatibility issue for mongodb inputs when using 5.x relicaset (influxdata#9892) docs: Add symlink to command documentation (influxdata#9926) docs: update contributing.md (influxdata#9914) chore: reference oracle external plugin (influxdata#9934) ...

telegraf-tiger bot added new plugin plugin/output 1. Request for new output plugins 2. Issues/PRs that are related to out plugins labels Jun 9, 2021

tomconte force-pushed the output/event-hubs branch from 0d4c213 to 1775fb3 Compare June 11, 2021 07:18

tomconte added 2 commits June 24, 2021 11:04

Azure Event Hubs output plugin

b543312

Fix Azure Monitor tests

7fe7245

tomconte force-pushed the output/event-hubs branch 2 times, most recently from 940d92c to c3cd9b0 Compare June 25, 2021 16:00

Fix linting errors

55fa40a

tomconte force-pushed the output/event-hubs branch from c3cd9b0 to 55fa40a Compare June 28, 2021 11:43

sspaink suggested changes Aug 31, 2021

View reviewed changes

plugins/outputs/event_hubs/README.md Show resolved Hide resolved

tomconte and others added 2 commits September 1, 2021 11:32

Update plugins/outputs/event_hubs/README.md

b61fabb

Co-authored-by: Sebastian Spaink <3441183+sspaink@users.noreply.github.com>

Delete binary file

c1b3b7a

tomconte changed the title ~~Azure Event Hubs output plugin~~ feat: Azure Event Hubs output plugin Sep 1, 2021

tomconte added 3 commits September 1, 2021 12:07

Merge remote-tracking branch 'origin/master' into output/event-hubs

eb906b8

go mod tidy

4cdd477

make fmt

83d84ee

tomconte requested a review from sspaink September 3, 2021 08:41

sspaink approved these changes Sep 7, 2021

View reviewed changes

tomconte added 2 commits September 8, 2021 09:51

Error handling

8094eae

Re-trigger CI

f2de694

sjwang90 added area/azure Azure plugins including eventhub_consumer, azure_storage_queue, azure_monitor cloud Issues or requests around cloud environments ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. labels Oct 12, 2021

tomconte added 3 commits October 18, 2021 10:50

Merge remote-tracking branch 'origin/master' into output/event-hubs

1a50f9b

go mod tidy

50cd22d

Fix call to json.NewSerializer

57f9908

tomconte force-pushed the output/event-hubs branch from 4fcc66a to 57f9908 Compare October 18, 2021 09:29

MyaLongmire approved these changes Oct 18, 2021

View reviewed changes

sspaink merged commit e324ef1 into influxdata:master Oct 18, 2021

tomconte deleted the output/event-hubs branch October 19, 2021 07:36

VladislavSenkevich pushed a commit to gwos/telegraf that referenced this pull request Nov 23, 2021

feat: Azure Event Hubs output plugin (influxdata#9346)

3ac36ff

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Azure Event Hubs output plugin #9346

feat: Azure Event Hubs output plugin #9346

tomconte commented Jun 9, 2021 •

edited

Loading

telegraf-tiger bot commented Jun 9, 2021

telegraf-tiger bot commented Jun 9, 2021

telegraf-tiger bot commented Jun 9, 2021

tomconte commented Jun 9, 2021

tomconte commented Jun 30, 2021

sspaink left a comment

tomconte commented Sep 1, 2021

sspaink commented Sep 7, 2021

telegraf-tiger bot commented Sep 8, 2021

Artifact URLs

tomconte commented Sep 9, 2021

tomconte commented Sep 21, 2021

sspaink commented Oct 14, 2021

telegraf-tiger bot commented Oct 18, 2021

Artifact URLs

tomconte commented Oct 18, 2021

feat: Azure Event Hubs output plugin #9346

feat: Azure Event Hubs output plugin #9346

Conversation

tomconte commented Jun 9, 2021 • edited Loading

Required for all PRs:

telegraf-tiger bot commented Jun 9, 2021

telegraf-tiger bot commented Jun 9, 2021

telegraf-tiger bot commented Jun 9, 2021

tomconte commented Jun 9, 2021

tomconte commented Jun 30, 2021

sspaink left a comment

Choose a reason for hiding this comment

tomconte commented Sep 1, 2021

sspaink commented Sep 7, 2021

telegraf-tiger bot commented Sep 8, 2021

Artifact URLs

tomconte commented Sep 9, 2021

tomconte commented Sep 21, 2021

sspaink commented Oct 14, 2021

telegraf-tiger bot commented Oct 18, 2021

Artifact URLs

tomconte commented Oct 18, 2021

tomconte commented Jun 9, 2021 •

edited

Loading