IllegalArgumentException when calling UpdatableRecord methods on tables with embedded domains that replace their underlying fields

[NorbertKawinski]

Expected behavior

New record is created in the database

Actual behavior

Exception in thread "main" java.lang.IllegalArgumentException: Field ("author"."id") is not contained in Row (
  "author"."id",
  "author"."name"
)

Steps to reproduce the problem

For such database:

CREATE DOMAIN author_id AS uuid NOT NULL;
CREATE TABLE author
(
    id   author_id    NOT NULL DEFAULT gen_random_uuid() PRIMARY KEY,
    name varchar(255) NOT NULL
);

Generated with
embeddableDomains = ".*"

This code

AuthorRecord author = sql.newRecord(AUTHOR);
author.setName("Sample Author");
author.store();

throws an exception.
Without the embeddableDomains flag, everything works fine.

jOOQ Version

org.jooq.trial:jooq:3.18.7

Database product and version

postgres:16.1-alpine Docker image

Java Version

Openjdk 21.0.1

OS Version

Microsoft Windows 11 Pro [10.0.22621 Build 22621]

JDBC driver name and version (include name if unofficial driver)

org.postgresql:postgresql:42.6.0

[NorbertKawinski]

MCVE available here: https://github.com/NorbertKawinski/jOOQ-mcve

EDIT: Probably related, the embedded keys generation have the same issue.

[lukaseder]

Thanks a lot for your report. I'll look into this right away.

[lukaseder]

Yeah, I can reproduce it, with or without defaults.

[lukaseder]

Stack trace of the new integration test:

java.lang.IllegalArgumentException: Field ("EMBEDDABLE_DOMAINS_IN_KEYS"."ID") is not contained in Row (
  "EMBEDDABLE_DOMAINS_IN_KEYS"."ID",
  "EMBEDDABLE_DOMAINS_IN_KEYS"."V"
)
	at org.jooq.impl.Tools.indexFail(Tools.java:2157)
	at org.jooq.impl.Tools.indexOrFail(Tools.java:2164)
	at org.jooq.impl.AbstractRecord.changed(AbstractRecord.java:574)
	at org.jooq.impl.UpdatableRecordImpl.store0(UpdatableRecordImpl.java:205)
	at org.jooq.impl.UpdatableRecordImpl.lambda$0(UpdatableRecordImpl.java:150)
	at org.jooq.impl.RecordDelegate.operate(RecordDelegate.java:144)
	at org.jooq.impl.UpdatableRecordImpl.store(UpdatableRecordImpl.java:149)
	at org.jooq.impl.UpdatableRecordImpl.store(UpdatableRecordImpl.java:141)
	at org.jooq.codegen.test.embeddabledomains.EmbeddableDomainsTest.testEmbeddableDomainsInKeys(EmbeddableDomainsTest.java:130)

[lukaseder]

The problem is that the generated Table.getPrimaryKey() is referencing the underlying columns, not the embeddables. I'll investigate separately, if this needs to be changed. Such a change couldn't be backported for compatibility reasons:

• Embedded keys do not appear in generated key metadata, such as Table.getPrimaryKey() #15875

However, it's probably possible to fix this even without the above change, and backport the fix.

[lukaseder]

Within jOOQ, there is an internal UpdatableRecord::getPrimaryKey() method to fetch generated keys:

Given that it's internal (package private), we could just fix that particular method for this case here.

[lukaseder]

As hinted by the call stack, the other UpdatableRecord methods are also likely affected. E.g. refresh():

java.lang.IllegalArgumentException: Field ("EMBEDDABLE_DOMAINS_IN_KEYS"."ID") is not contained in Row (
  "EMBEDDABLE_DOMAINS_IN_KEYS"."ID",
  "EMBEDDABLE_DOMAINS_IN_KEYS"."V"
)
	at org.jooq.impl.Tools.indexFail(Tools.java:2157)
	at org.jooq.impl.AbstractRecord.get(AbstractRecord.java:333)
	at org.jooq.impl.Tools.addCondition(Tools.java:4133)
	at org.jooq.impl.Tools.addConditions(Tools.java:4121)
	at org.jooq.impl.UpdatableRecordImpl.refresh(UpdatableRecordImpl.java:433)
	at org.jooq.impl.UpdatableRecordImpl.refresh(UpdatableRecordImpl.java:425)
	at org.jooq.codegen.test.embeddabledomains.EmbeddableDomainsTest.testEmbeddableDomainsInKeys(EmbeddableDomainsTest.java:137)

[lukaseder]

Another related issue that will be fixed with this one:

• Internal RecordDataType.getDataType() should be non-nullable for records consisting of all non-null fields #15876

[lukaseder]

This only happens when the embeddable replaces the underlying column(s)

[lukaseder]

Duplicate of #12689

[lukaseder]

Fixed in versions:

• 3.19.0
• 3.18.8 (IllegalArgumentException when calling UpdatableRecord methods on tables with embedded domains that replace their underlying fields - Merge [#15873] #15880)
• 3.17.17 (IllegalArgumentException when calling UpdatableRecord methods on tables with embedded domains that replace their underlying fields - Merge [#15873] #15881)
• 3.16.23 (IllegalArgumentException when calling UpdatableRecord methods on tables with embedded domains that replace their underlying fields - Merge [#15873] #15882)

Thanks again for your report! I will continue investigating related edge cases (e.g. embeddable keys, keys combining domains and non-embeddables) to see if other, related problems still exist. Will link to further issues from here.

[lukaseder]

I realise there was a comment here:

EDIT: Probably related, the embedded keys generation have the same issue.

What did you mean by this? Enabling both embedded keys and domains? This is currently not working due to the inability to nest embeddables:

• Nested embeddables in codegen #13059
• Fix known limitations of embeddable types #11975

Or did you have something else in mind?

[lukaseder]

Another issue when a composite key referneces a domain type:

• Compilation error in generated code when using <embeddableDomains/> and the domain is part of a composite key #15883

[NorbertKawinski]

Ah, no, not that complex with nesting, sorry for not being clear :)
I just meant that the same issue "Field (xxx) is not contained in Row" happened when I used "embeddablePrimaryKeys" --instead of-- domains.

Actually this is how I first bumped into this issue because I started generating the embedded keys.
When it didn't work I tried using domains but got the same issue.

If you'd like to see what I mean, then I updated my MCVE with "embeddablePrimaryKeys"
NorbertKawinski/jOOQ-mcve@e04cc82

[lukaseder]

Ah, no, not that complex with nesting, sorry for not being clear :)
I just meant that the same issue "Field (xxx) is not contained in Row" happened when I used "embeddablePrimaryKeys" --instead of-- domains.

I see, I'll check that as well, thanks. When combining the two flags (which isn't unlikely), then, nesting would have to occur, though. The domain generates an embeddable first, and then the key generates another one after, based on the first. This doesn't work yet.

[lukaseder]

I can confirm, this fixes also the case of using embeddablePrimaryKeys (or embeddableUniqueKeys)