Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NEW FEATURE: Gather data for providers concurrently #2873

Merged
merged 55 commits into from
Mar 27, 2024
Merged

Conversation

tlimoncelli
Copy link
Contributor

@tlimoncelli tlimoncelli commented Mar 12, 2024

Goal: collect "preview" information in parallel to improve dnscontrol peformance.

  • dnscontrol ppreview -- Like preview, but collects data in parallel for providers that permit it.
  • dnscontrol ppush -- Like ppreview but pushes
  • Flags:
    • --cmode=none Don't do any work concurrently (useful when you find a bug)
    • --cmode=all Run all providers concurrently (dangerous!)

NOTE: Eventually ppreview and ppush will replace preview and push. These are temporary commands. We'll probably do 1-2 public releases with the pp versions before we remove the non-parallel verions.

How to test this

Here's how to try the changes:

$ git clone https://github.com/StackExchange/dnscontrol.git$ cd dnscontrol
$ git checkout tlim_parallel$ go install
$ (cd to where your dnsconfig.js is)
$ dnscontrol preview                 # Old code$ dnscontrol ppreview --cmode=none   # New code, one provider at a time
$ dnscontrol ppreview                # New code, gather data concurrently when safe
$ dnscontrol ppreview --cmode=all    # New code, gather data concurrently no matter what! (unsafe!)

Is your provider safe to run concurrently?

Look for this line in the source file:

providers.CanConcur:              providers.Cannot(),

If you see providers.Can() then your provider has already been tested and verified to be working.

If you'd like to test your provider, change Cannot() to Can() and compile with the -race flag. This will attempt to detect any race conditions.

cd /my/source/code
vim provider/foo/fooProvider.go       # Change Cannot to Can
go install -race

cd /my/dns/config
dnscontrol ppreview --full

Please report any bugs, suggestions, or performance results in the comments below.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes detected by Check Git Status Action

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes detected by Check Git Status Action

@tlimoncelli tlimoncelli changed the title Gather data for providers concurently Gather data for providers concurrently Mar 23, 2024
@tlimoncelli tlimoncelli changed the title Gather data for providers concurrently NEW FEATURE: Gather data for providers concurrently Mar 23, 2024
@tlimoncelli tlimoncelli merged commit 68c5e87 into main Mar 27, 2024
18 checks passed
@tlimoncelli tlimoncelli deleted the tlim_parallel branch March 27, 2024 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant