Large Number of Records?

[ITBlogger]

We're getting timeouts when trying to do puppet runs with large number of zones (5000+) and records (5000+)...is anyone using this module with a large amount of records successfully?

[solarkennedy]

Wow! That is a lot of records!
You are probably the biggest user! :)

That kind of use case is probably better served via a different implementation. This module makes heavy use of concat, which is not very efficient.

Can you tell me more about your use case and how you use this module? Maybe I could suggest some alternatives?

[ITBlogger]

We were looking to automate management of bind for all of our dns zones and
records, internal and external, for where I work. We have a huge amount of
vanity domains.

On Tue, Apr 28, 2015 at 7:49 PM, Kyle Anderson notifications@github.com
wrote:

Wow! That is a lot of records!
You are probably the biggest user! :)

That kind of use case is probably better served via a different
implementation. This module makes heavy use of concat, which is not very
efficient.

Can you tell me more about your use case and how you use this module?
Maybe I could suggest some alternatives?

—
Reply to this email directly or view it on GitHub
#104 (comment).

[solarkennedy]

But do they need to be defined in the puppet DSL?
Are they all very similar/templated?

[ITBlogger]

no, they just need to be automated...they all fit within the constraints of
this module

On Wed, Apr 29, 2015 at 7:24 AM, Kyle Anderson notifications@github.com
wrote:

But do they need to be defined in the puppet DSL?
Are they all very similar/templated?

—
Reply to this email directly or view it on GitHub
#104 (comment).

[ITBlogger]

I got past the timeouts with:

http_connect_timeout = 1h
configtimeout = 1h

in puppet.conf

Also had to increase java memory maximums in /etc/sysconfig/pe-puppetdb and
/etc/sysconfig/pe-puppetserver

--Alex

On Wed, Apr 29, 2015 at 9:02 AM, Alex Scoble bloggingitguy@gmail.com
wrote:

no, they just need to be automated...they all fit within the constraints
of this module

On Wed, Apr 29, 2015 at 7:24 AM, Kyle Anderson notifications@github.com
wrote:

But do they need to be defined in the puppet DSL?
Are they all very similar/templated?

—
Reply to this email directly or view it on GitHub
#104 (comment).

[solarkennedy]

Ok. Certainly do what you need to do. I personally think this module is not the right tool for the job to scale to thousands of records and thousands of domains. This is certainly true if many of them are DNAMEs or can be easily templated.

Or, if you do need to scale and must use puppet, you could do a thing where you slave to a different server for subdomains or a subset of zones, to separate the megazones into smaller domains. I don't know if that is possible in your use case, but it works for many. (.dev, .stage, .prod, etc)

We can keep this issue here for others who need to search, or we can reference back to it in the future.