Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Overly long TXT record breaks lmdbbackend import #8012

Open
mzealey opened this issue Jul 1, 2019 · 8 comments

Comments

@mzealey
Copy link
Contributor

commented Jul 1, 2019

  • Program: Authoritative
  • Issue type: Bug report

Short description

Long TXT records cause lmdb import process to break with the error message:

$ pdnsutil load-zone example.com example.com.db
Error: putting data: MDB_BAD_VALSIZE: Unsupported size of key/DB name/data, or wrong DUPFIXED size

Example zone file example.com.db:

example.com.    86400   IN      SOA     example.com.    example.com.    (
                                                2018031306 ;Serial Number
                                                14400 ;refresh
                                                7200 ;retry
                                                3600000 ;expire
                                                14400   )

example.com.    10      IN      TXT     "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"

I guess you are storing the TXT aspect of the detail in the lmdb key record which causes it to go over the 512 byte key limit? If this is the case it may need a db design change before launching the new lmdb backend as I suspect TXT records should be stored as values.

Environment

  • Operating system: centos7
  • Software version: pdns-4.2.0-rc2.169.master.g85474361f-centos-7.tar.bz2
@Habbie

This comment has been minimized.

Copy link
Member

commented Jul 1, 2019

We appear to be using DUPSORT which makes the value participate in the 512 byte limit, indeed.

@Habbie Habbie added auth defect labels Jul 1, 2019

@Habbie Habbie added this to the auth-4.2.0 milestone Jul 1, 2019

@mzealey

This comment has been minimized.

Copy link
Contributor Author

commented Jul 1, 2019

I'm not 100% sure of the physicalities of the underlying DNS packets but I think that a TXT record can not be over 255 bytes (are other records allowed to be longer?). This may mean that this is not an issue and you should expect input data that is accordingly valid somehow... Either way I think this needs a better error message ie marking the zone as invalid in the bind backend (or elsewhere in the system).

@Habbie

This comment has been minimized.

Copy link
Member

commented Jul 1, 2019

I think that a TXT record can not be over 255 bytes (are other records allowed to be longer?).

TXT can be up to ~64K (but anything beyond a kilobyte or two is unrealistic for practical uses). Many other records can also be >255 but most have no business being over a few hundred bytes.

@mzealey

This comment has been minimized.

Copy link
Contributor Author

commented Jul 1, 2019

You may need to look at having a break-out table for storing longer records in that case then :-/

@gibson042

This comment has been minimized.

Copy link
Contributor

commented Jul 2, 2019

TXT RDATA can be up to 64 KB, but it is composed of <character-string>s, each of which is itself limited to 256 bytes (a single length octet followed by up to 255 octets of contents). The sample zone file includes an unacceptable 600-character <character-string>.

Invalid (too many characters in <character-string> contents)

example.com.    10      IN      TXT     "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"

Valid (multiple <character-string>s, each with contents not exceeding 255 characters)

example.com.    10      IN      TXT    (
  "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
  "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
  "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
  "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
  "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
  "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
)
@Habbie

This comment has been minimized.

Copy link
Member

commented Jul 2, 2019

Invalid (too many characters in contents)

While invalid in formal syntax, it has been legal in PowerDNS (in all backends, including the bindbackend) for decades, and PowerDNS will automatically split it into chunks.

@Habbie

This comment has been minimized.

Copy link
Member

commented Jul 17, 2019

It looks like we might not fix this for 4.2; what we need to do for 4.2 is add a schema version to the 'main' LMDB so we can do migrations later, and update docs to clarify LMDB as experimental, mentioning this limitation.

@Habbie Habbie referenced this issue Jul 26, 2019
3 of 8 tasks complete
@Habbie

This comment has been minimized.

Copy link
Member

commented Aug 6, 2019

4.2 now has schema versioning (once #8165 is merged). Moving this to milestone 4.3.0.

@Habbie Habbie modified the milestones: auth-4.2.0, auth-4.3.0 Aug 6, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.