Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Catalog zones #304

Merged
merged 57 commits into from Feb 16, 2024
Merged

Catalog zones #304

merged 57 commits into from Feb 16, 2024

Conversation

wtoorop
Copy link
Member

@wtoorop wtoorop commented Nov 26, 2023

This version does the task from within the tranfer daemon (xfrd) which makes everything easier and more straight forward.

Also change ident to "load" when reloading and the backup main process
to "old-main". When reload succeeds, "load" will become "main" and
"old-main" will quit. Otherwise "load" will be exited (because of
failure) and "old_main" will continue as "main".
A version of dname_to_string() that writes to a buffer instead of
returning a static buffer. This is useful when you need to print two
dnames in a log message.
+ marshalling of the catalog zone pattern options
nsd.conf.5.in Outdated Show resolved Hide resolved
nsd.conf.5.in Outdated Show resolved Hide resolved
@pettai
Copy link

pettai commented Nov 27, 2023

Current version works fine for us 👍 (just replaced one of our test-installations of a previous catz-branch build)

@pettai
Copy link

pettai commented Nov 27, 2023

First issue encountered:
Nov 27 20:10:01 sunic nsd[2498035]: invalid catalog consumer zone 'test.catalog': 'version.test.catalog TXT RRset not found

zone:	test.catalog
	catalog: consumer
	catalog-invalid: 'version.test.catalog TXT RRset not found
	state: ok
	served-serial: "1698045664 since 2023-11-27T19:31:01"
	commit-serial: none
	wait: "84395 sec between attempts"

A manual nsd-control transfer test.catalog fixed the issue.

@pettai
Copy link

pettai commented Nov 27, 2023

Next issue I see then I restart nsd with systemctl restart nsd. There are some defunct nsd processes around, but nothing insightful in the logs. (nsd with the catzones seems to work fine...)

 22:08 root@sunic: /etc/nsd/nsd.conf.d # ps -ef | grep nsd
nsd      2850692       1  0 22:07 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2850694 2850692 98 22:07 ?        00:01:12 /usr/sbin/nsd -d -P
nsd      2853485 2850694  3 22:08 ?        00:00:00 [nsd: server 2] <defunct>
nsd      2853486 2850694  3 22:08 ?        00:00:00 [nsd: server 3] <defunct>
nsd      2853487 2850694  3 22:08 ?        00:00:00 [nsd: server 4] <defunct>
nsd      2853488 2850694  3 22:08 ?        00:00:00 [nsd: server 5] <defunct>
nsd      2853490 2850694  3 22:08 ?        00:00:00 [nsd: server 7] <defunct>
nsd      2853491 2850694  3 22:08 ?        00:00:00 [nsd: server 8] <defunct>
nsd      2853492 2850694  3 22:08 ?        00:00:00 [nsd: server 9] <defunct>
nsd      2853498 2850694  3 22:08 ?        00:00:00 [nsd: server 15] <defunct>
nsd      2853507 2850694  4 22:08 ?        00:00:00 [nsd: server 24] <defunct>
nsd      2853508 2850694  4 22:08 ?        00:00:00 [nsd: server 25] <defunct>
nsd      2853510 2850694  4 22:08 ?        00:00:00 [nsd: server 27] <defunct>
nsd      2853511 2850694  4 22:08 ?        00:00:00 [nsd: server 28] <defunct>
nsd      2853512 2850694  4 22:08 ?        00:00:00 [nsd: server 29] <defunct>
nsd      2853513 2850694  4 22:08 ?        00:00:00 [nsd: server 30] <defunct>
nsd      2853516 2850694  4 22:08 ?        00:00:00 [nsd: old-main] <defunct>
nsd      2853519 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853520 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853521 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853522 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853523 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853524 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853525 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853526 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853527 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853528 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853529 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853530 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853531 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853532 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853533 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853534 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853535 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853536 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853537 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853538 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853539 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853540 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853541 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853542 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853543 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853544 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853545 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853546 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853547 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853548 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853549 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853550 2850694  0 22:08 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853551 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853552 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853553 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853554 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853555 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853556 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853557 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853558 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853559 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853560 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853561 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853562 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853563 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853564 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853565 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853566 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853567 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853568 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853569 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853570 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
nsd      2853571 2850694  0 22:09 ?        00:00:00 /usr/sbin/nsd -d -P
root     2853573 2837108  0 22:09 pts/0    00:00:00 grep --color=auto nsd

@pettai
Copy link

pettai commented Nov 28, 2023

Next issue I see then I restart nsd with systemctl restart nsd. There are some defunct nsd processes around, but nothing insightful in the logs. (nsd with the catzones seems to work fine...)

This seems to be related to configuration processing (like the previous catz work)

pattern:
	name: "test.catalog"
	zonefile: "/var/lib/nsd/catzones/%s"

zone:
	name: "test.catalog."   <--- by adding a tailing dot (.) the <defunct> processes goes away
	catalog: consumer
	catalog-member-pattern: "test.catalog"

instead of struct domain
TODO:
- Check all cases where zones are added, deleted or changed for status
  change of: producer zones and producer member zones.
- A zone could be a consumer member zone, *and* a producer zone or a
  producer member.
and process more than one transfer file.
TODO change processing after successful SOAINFO
	... before xfrd_unlink_xfrfile().
The function can still exit via apply_ixfr() -> addRR() or deleteRR()
when out of memory, but in that case all bets are off anyway, right?
Copy link
Member

@wcawijngaards wcawijngaards left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change looks okay to me, there are some issues in the list of suggestions. The consumer and producer parts look nice to have for catalog zones.

nsd-checkconf.c Outdated Show resolved Hide resolved
nsd.conf.5.in Outdated Show resolved Hide resolved
nsd.conf.5.in Outdated Show resolved Hide resolved
nsd.conf.5.in Outdated Show resolved Hide resolved
options.c Outdated Show resolved Hide resolved
xfrd-catalog-zones.c Outdated Show resolved Hide resolved
xfrd-catalog-zones.c Outdated Show resolved Hide resolved
xfrd-catalog-zones.c Outdated Show resolved Hide resolved
xfrd-catalog-zones.c Outdated Show resolved Hide resolved
xfrd-catalog-zones.c Outdated Show resolved Hide resolved
nsd.conf.5.in Outdated Show resolved Hide resolved
wtoorop and others added 3 commits January 29, 2024 08:50
Thanks Wouter! I'll resolve the remaining reduce/reduce conflicts from command line.

Co-authored-by: Wouter Wijngaards <wcawijngaards@users.noreply.github.com>
@k0ekk0ek k0ekk0ek mentioned this pull request Jan 29, 2024
Copy link
Contributor

@k0ekk0ek k0ekk0ek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still looking into this, but thought I'd post one initial remark.

dname.c Outdated Show resolved Hide resolved
util.c Outdated Show resolved Hide resolved
difffile.h Outdated Show resolved Hide resolved
Comment on lines +315 to +318
uint8_t catalog_role;
uint8_t catalog_role_is_default;
const char* catalog_member_pattern;
const char* catalog_producer_zone;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Back when the first PR got made, we kind-of decided catalog properties must be set on a zone. A reason for that being so that you cannot all of a sudden break things simply by re-applying patterns. Why should this be part of a pattern?

Copy link
Member Author

@wtoorop wtoorop Jan 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I evaluated this extensively. Have a look at the catzones.tdir and the text in the man page to see how this all works. The re-applying patterns issues are also covered in catzones.tdir.
Short summary: A catalog producer (catalog_role == CATALOG_ROLE_PRODUCER) is only functional when in a zone clause which is checked by evaluating the value for "catalog_producer_zone" (which only makes sense in a pattern and not in a zone; i.e. catalog member zones are always instantiated through patterns).
Currently only a single catalog consumer is allowed, but this may change in the future. In that future extension there is no reason why you would not have a catalog member zone which is a catalog consumer itself. But that's the future; we do it step by step. Currently it at least allows the user to configure a consumer zone in all the for nsd conventional manner.

xfrd-catalog-zones.c Outdated Show resolved Hide resolved
xfrd-catalog-zones.c Outdated Show resolved Hide resolved
xfrd-catalog-zones.c Outdated Show resolved Hide resolved
dname.c Outdated Show resolved Hide resolved
util.c Outdated Show resolved Hide resolved
util.c Outdated Show resolved Hide resolved
dname.h Outdated Show resolved Hide resolved
wtoorop and others added 6 commits January 30, 2024 11:02
Co-authored-by: Wouter Wijngaards <wcawijngaards@users.noreply.github.com>
Recover line to report an error

Co-authored-by: Wouter Wijngaards <wcawijngaards@users.noreply.github.com>
... now used in a function prototype in dname.h
+ Some clarifying comments in the label_plus_dname() function
@wtoorop wtoorop requested a review from k0ekk0ek January 30, 2024 14:58
@k0ekk0ek k0ekk0ek removed their request for review February 2, 2024 15:54
@wtoorop wtoorop merged commit e3d63f9 into master Feb 16, 2024
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants