Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding a new directory monitor input plugin. #8751

Merged
merged 9 commits into from
Mar 2, 2021
Merged

Conversation

ivorybilled
Copy link
Contributor

@ivorybilled ivorybilled commented Jan 26, 2021

This will monitor and parse all files coming into a single configured directory.

Closes #7548

Copy link
Contributor

@telegraf-tiger telegraf-tiger bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks so much for the pull request!
🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

Copy link
Contributor

@telegraf-tiger telegraf-tiger bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

Copy link
Contributor

@telegraf-tiger telegraf-tiger bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

@ivorybilled ivorybilled dismissed telegraf-tiger’s stale review January 26, 2021 20:02

I'm being attacked by a tiger

Copy link
Contributor

@telegraf-tiger telegraf-tiger bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤝 ✒️ Just a reminder that the CLA has not yet been signed, and we'll need it before merging. Please sign the CLA when you get a chance, then post a comment here saying !signed-cla

Copy link
Contributor

@telegraf-tiger telegraf-tiger bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤝 ✅ CLA has been signed. Thank you!

Copy link
Contributor

@reimda reimda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's coming along!

plugins/inputs/directory_monitor/directory_monitor.go Outdated Show resolved Hide resolved
plugins/parsers/csv/parser.go Outdated Show resolved Hide resolved
plugins/inputs/directory_monitor/directory_monitor.go Outdated Show resolved Hide resolved
plugins/inputs/directory_monitor/directory_monitor.go Outdated Show resolved Hide resolved
@sjwang90 sjwang90 mentioned this pull request Jan 28, 2021
3 tasks
@sjwang90 sjwang90 added new plugin plugin/input 1. Request for new input plugins 2. Issues/PRs that are related to input plugins labels Jan 28, 2021
@influxdata influxdata deleted a comment from ivorybilled Jan 29, 2021
@influxdata influxdata deleted a comment from ivorybilled Jan 29, 2021
@influxdata influxdata deleted a comment from ivorybilled Jan 29, 2021
@influxdata influxdata deleted a comment from ivorybilled Jan 29, 2021
Copy link
Contributor

@ssoroka ssoroka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks well thought through. I think we can clean it up further. Mostly minor feedback, but I think I'd like the parallelism to go away. It complicates the code and we're not going to get any better throughput for it. Keep in mind the limiting factor is going to be the TrackingMetrics by a long shot.

plugins/inputs/directory_monitor/README.md Outdated Show resolved Hide resolved
plugins/inputs/directory_monitor/README.md Outdated Show resolved Hide resolved
plugins/inputs/directory_monitor/README.md Outdated Show resolved Hide resolved
plugins/inputs/directory_monitor/README.md Outdated Show resolved Hide resolved
plugins/inputs/directory_monitor/README.md Outdated Show resolved Hide resolved
plugins/inputs/directory_monitor/directory_monitor.go Outdated Show resolved Hide resolved
plugins/inputs/directory_monitor/directory_monitor.go Outdated Show resolved Hide resolved
plugins/inputs/directory_monitor/directory_monitor.go Outdated Show resolved Hide resolved
plugins/inputs/directory_monitor/directory_monitor.go Outdated Show resolved Hide resolved
plugins/inputs/directory_monitor/directory_monitor.go Outdated Show resolved Hide resolved
@ivorybilled
Copy link
Contributor Author

!retry-checks

Copy link
Contributor

@ssoroka ssoroka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider removing the parallelism as I don't think it's adding anything here.

plugins/inputs/directory_monitor/directory_monitor.go Outdated Show resolved Hide resolved
plugins/inputs/directory_monitor/directory_monitor.go Outdated Show resolved Hide resolved
plugins/inputs/directory_monitor/directory_monitor.go Outdated Show resolved Hide resolved
plugins/inputs/directory_monitor/directory_monitor.go Outdated Show resolved Hide resolved
@ssoroka ssoroka merged commit 6008168 into master Mar 2, 2021
@ssoroka ssoroka deleted the directoryMonitorPlugin branch March 2, 2021 22:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new plugin plugin/input 1. Request for new input plugins 2. Issues/PRs that are related to input plugins
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add plugin that implements a simple file based queue system
4 participants