Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(agent): Introduce CLI option to set config URL retry attempts #15377

Merged
merged 1 commit into from
May 21, 2024
Merged

feat(agent): Introduce CLI option to set config URL retry attempts #15377

merged 1 commit into from
May 21, 2024

Conversation

powersj
Copy link
Contributor

@powersj powersj commented May 17, 2024

Summary

This introduces a new cli option to allow the user to set the number of retry attempts to something other than 3. It also allows the user to set the attempt count to -1 to infinitely retry.

Checklist

  • No AI generated code was used in this PR

Related issues

fixes: #8854

@powersj
feat(agent): Introduce CLI option to set config URL retry attempts
470da02 
This introduces a new cli option to allow the user to set the number of
retry attempts to something other than 3. It also allows the user to set
the attempt count to -1 to infinitely retry.

fixes: influxdata#8854
@telegraf-tiger telegraf-tiger bot added the feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin label May 17, 2024
@telegraf-tiger
Copy link
Contributor

Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

👍 This pull request doesn't change the Telegraf binary size

📦 Click here to get additional PR build artifacts

Artifact URLs

DEB RPM TAR GZ ZIP
amd64.deb aarch64.rpm darwin_amd64.tar.gz windows_amd64.zip
arm64.deb armel.rpm darwin_arm64.tar.gz windows_arm64.zip
armel.deb armv6hl.rpm freebsd_amd64.tar.gz windows_i386.zip
armhf.deb i386.rpm freebsd_armv7.tar.gz
i386.deb ppc64le.rpm freebsd_i386.tar.gz
mips.deb riscv64.rpm linux_amd64.tar.gz
mipsel.deb s390x.rpm linux_arm64.tar.gz
ppc64el.deb x86_64.rpm linux_armel.tar.gz
riscv64.deb linux_armhf.tar.gz
s390x.deb linux_i386.tar.gz
linux_mips.tar.gz
linux_mipsel.tar.gz
linux_ppc64le.tar.gz
linux_riscv64.tar.gz
linux_s390x.tar.gz

@powersj powersj marked this pull request as ready for review May 20, 2024 14:40
@powersj powersj added the ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review. label May 20, 2024
@powersj
Copy link
Contributor Author

powersj commented May 20, 2024 •
edited

Some manual testing results:

Default Case

This is the initial attempt + 3 retires:

❯ ./telegraf --config http://localhost:8080/config.toml
2024-05-20T14:40:49Z I! Loading config: http://localhost:8080/config.toml
2024-05-20T14:40:49Z I! Using default number of attempts to fetch HTTP config: 3
2024-05-20T14:40:49Z I! Error getting HTTP config (attempt 0 of 3): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:40:59Z I! Error getting HTTP config (attempt 1 of 3): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:41:09Z I! Error getting HTTP config (attempt 2 of 3): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:41:19Z I! Error getting HTTP config (attempt 3 of 3): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:41:19Z E! error loading config file http://localhost:8080/config.toml: failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused

Set attempts to 4

More than default 3 retry attempts:

❯ ./telegraf --config http://localhost:8080/config.toml --config-url-retry-attempts=4
2024-05-20T14:42:11Z I! Loading config: http://localhost:8080/config.toml
2024-05-20T14:42:11Z I! Error getting HTTP config (attempt 0 of 4): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:42:21Z I! Error getting HTTP config (attempt 1 of 4): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:42:31Z I! Error getting HTTP config (attempt 2 of 4): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:42:41Z I! Error getting HTTP config (attempt 3 of 4): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:42:51Z I! Error getting HTTP config (attempt 4 of 4): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:42:51Z E! error loading config file http://localhost:8080/config.toml: failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused

Set attempts to 2

Less than default 3 retries:

❯ ./telegraf --config http://localhost:8080/config.toml --config-url-retry-attempts=2
2024-05-20T14:42:58Z I! Loading config: http://localhost:8080/config.toml
2024-05-20T14:42:58Z I! Error getting HTTP config (attempt 0 of 2): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:43:08Z I! Error getting HTTP config (attempt 1 of 2): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:43:18Z I! Error getting HTTP config (attempt 2 of 2): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:43:18Z E! error loading config file http://localhost:8080/config.toml: failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused

Set attempts to 0

Will use the default 3 retry attempts:

❯ ./telegraf --config http://localhost:8080/config.toml --config-url-retry-attempts=0
2024-05-20T14:43:40Z I! Loading config: http://localhost:8080/config.toml
2024-05-20T14:43:40Z I! Using default number of attempts to fetch HTTP config: 3
2024-05-20T14:43:40Z I! Error getting HTTP config (attempt 0 of 3): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:43:50Z I! Error getting HTTP config (attempt 1 of 3): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:44:00Z I! Error getting HTTP config (attempt 2 of 3): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:44:10Z I! Error getting HTTP config (attempt 3 of 3): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:44:10Z E! error loading config file http://localhost:8080/config.toml: failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused

Set attempts to -1

This retries forever:

❯ ./telegraf --config http://localhost:8080/config.toml --config-url-retry-attempts=-1
2024-05-20T14:44:16Z I! Loading config: http://localhost:8080/config.toml
2024-05-20T14:44:16Z I! Using unlimited number of attempts to fetch HTTP config
2024-05-20T14:44:16Z I! Error getting HTTP config (attempt 0 of -1): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:44:26Z I! Error getting HTTP config (attempt 1 of -1): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:44:36Z I! Error getting HTTP config (attempt 2 of -1): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:44:46Z I! Error getting HTTP config (attempt 3 of -1): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:44:56Z I! Error getting HTTP config (attempt 4 of -1): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
2024-05-20T14:45:06Z I! Error getting HTTP config (attempt 5 of -1): failed to connect to HTTP config server: Get "http://localhost:8080/config.toml": dial tcp 127.0.0.1:8080: connect: connection refused
^C

Set attempts to -2

Anything below -1 is invalid:

❯ ./telegraf --config http://localhost:8080/config.toml --config-url-retry-attempts=-2
2024-05-20T14:45:21Z I! Loading config: http://localhost:8080/config.toml
2024-05-20T14:45:21Z E! error loading config file http://localhost:8080/config.toml: invalid number of attempts: -2

Retry works

One positive case, where retry works after I bring server up:

❯ ./telegraf --config http://localhost:8000/config.toml --config-url-retry-attempts=-1
2024-05-20T14:47:04Z I! Loading config: http://localhost:8000/config.toml
2024-05-20T14:47:04Z I! Using unlimited number of attempts to fetch HTTP config
2024-05-20T14:47:04Z I! Error getting HTTP config (attempt 0 of -1): failed to connect to HTTP config server: Get "http://localhost:8000/config.toml": dial tcp 127.0.0.1:8000: connect: connection refused
2024-05-20T14:47:14Z I! Error getting HTTP config (attempt 1 of -1): failed to connect to HTTP config server: Get "http://localhost:8000/config.toml": dial tcp 127.0.0.1:8000: connect: connection refused
2024-05-20T14:47:24Z W! DeprecationWarning: Option "fielddrop" of plugin "inputs.mysql" deprecated since version 1.29.0 and will be removed in 2.0.0: use 'fieldexclude' instead
2024-05-20T14:47:24Z I! Starting Telegraf 1.31.0-cda0f1d9 brought to you by InfluxData the makers of InfluxDB
2024-05-20T14:47:24Z I! Available plugins: 234 inputs, 9 aggregators, 32 processors, 26 parsers, 60 outputs, 6 secret-stores
2024-05-20T14:47:24Z I! Loaded inputs: mysql
2024-05-20T14:47:24Z I! Loaded aggregators: 
2024-05-20T14:47:24Z I! Loaded processors: 
2024-05-20T14:47:24Z I! Loaded secretstores: 
2024-05-20T14:47:24Z I! Loaded outputs: file
2024-05-20T14:47:24Z I! Tags enabled: 
2024-05-20T14:47:24Z I! [agent] Config: Interval:10s, Quiet:false, Hostname:"", Flush Interval:10s
2024-05-20T14:47:24Z D! [agent] Initializing plugins
2024-05-20T14:47:24Z D! [agent] Connecting outputs
2024-05-20T14:47:24Z D! [agent] Attempting connection to [outputs.file]
2024-05-20T14:47:24Z D! [agent] Successfully connected to outputs.file
2024-05-20T14:47:24Z D! [agent] Starting service inputs
^C2024-05-20T14:47:28Z D! [agent] Stopping service inputs
2024-05-20T14:47:28Z D! [agent] Input channel closed
2024-05-20T14:47:28Z I! [agent] Hang on, flushing any cached metrics before shutdown
2024-05-20T14:47:28Z D! [outputs.file]  Buffer fullness: 0 / 10000 metrics
2024-05-20T14:47:28Z I! [agent] Stopping running outputs
2024-05-20T14:47:28Z D! [agent] Stopped Successfully

DStrand1
DStrand1 approved these changes May 20, 2024
View reviewed changes
Copy link
Contributor

@DStrand1 DStrand1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@DStrand1 DStrand1 removed their assignment May 20, 2024
srebhan
srebhan approved these changes May 21, 2024
View reviewed changes
Copy link
Contributor

@srebhan srebhan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

@srebhan srebhan merged commit ad59290 into influxdata:master May 21, 2024
27 checks passed
@github-actions github-actions bot added this to the v1.31.0 milestone May 21, 2024
pvlltvk pushed a commit to devopsext/telegraf that referenced this pull request May 31, 2024
@powersj @pvlltvk
feat(agent): Introduce CLI option to set config URL retry attempts (i…
3a94fa8 
…nfluxdata#15377)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@srebhan srebhan srebhan approved these changes

@DStrand1 DStrand1 DStrand1 approved these changes

Assignees

@srebhan srebhan

Labels
area/agent feat Improvement on an existing feature such as adding a new setting/mode to an existing plugin ready for final review This pull request has been reviewed and/or tested by multiple users and is ready for a final review.
Projects
None yet
Milestone
v1.31.0
Development

Successfully merging this pull request may close these issues.

Configuration capabilities to retry for loading config via URL
3 participants
@powersj @srebhan @DStrand1