Skip to content

Change Datum to only perform checked addition#1039

Open
sgrif wants to merge 3 commits into
mainfrom
sg-checked-addition
Open

Change Datum to only perform checked addition#1039
sgrif wants to merge 3 commits into
mainfrom
sg-checked-addition

Conversation

@sgrif
Copy link
Copy Markdown
Contributor

@sgrif sgrif commented Jun 5, 2026

Datum does not represent a type that can unconditionally add to itself. Currently, any time we encounter two types which do not match, or any types which can't be added, we silently return NULL. This changes that behavior to return a specific error. The Add impl is removed, as these cases will most likely represent a bug in pgdog and shouldn't be silently swallowed.

Two potential future changes stand out to me. The first is that we currently unconditionally error on types that differ, even though we can absolutely add Integer to Bigint for example. I'm not sure that we want to be responsible for maintaining the whole matrix of cross type addition that PG supports, so for the time being I maintained the current behavior.

The second is that how we handle NULL is currently the exact opposite of what PG does. I suspect this will be because all calls to addition come from the implementation of aggregate functions, where we might get NULL from a shard that returned no rows, and we don't want that NULL to cancel out the results from other shards.

That behavior in general is reasonable, but in such a generic function like Datum::add, diverging from PG is surprising behavior, and a potential footgun waiting to go off. I think instead we should have the aggregate functions be responsible for handling NULLs, and mirror PG in this more general case.

However, I opted not to make that change in this PR, as I'm not confident we have test coverage for this case, where a cross shard query returns results from one query and NULL from another. As I refactor this code further, I intend to make that change down the line.

Datum does not represent a type that can unconditionally add to itself.
Currently, any time we encounter two types which do not match, or any
types which can't be added, we silently return NULL. This changes that
behavior to return a specific error. The Add impl is removed, as these
cases will most likely represent a bug in pgdog and shouldn't be
silently swallowed.

Two potential future changes stand out to me. The first is that we
currently unconditionally error on types that differ, even though we can
absolutely add Integer to Bigint for example. I'm not sure that we want
to be responsible for maintaining the whole matrix of cross type
addition that PG supports, so for the time being I maintained the
current behavior.

The second is that how we handle NULL is currently the exact opposite of
what PG does. I suspect this will be because all calls to addition come
from the implementation of aggregate functions, where we might get NULL
from a shard that returned no rows, and we don't want that NULL to
cancel out the results from other shards.

That behavior in general is reasonable, but in such a generic function
like Datum::add, diverging from PG is surprising behavior, and a
potential footgun waiting to go off. I think instead we should have the
aggregate functions be responsible for handling NULLs, and mirror PG in
this more general case.

However, I opted not to make that change in this PR, as I'm not
confident we have test coverage for this case, where a cross shard query
returns results from one query and NULL from another. As I refactor this
code further, I intend to make that change down the line.
@sgrif sgrif requested review from levkk and meskill June 5, 2026 18:21
@blacksmith-sh

This comment has been minimized.

@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 5, 2026

Codecov Report

❌ Patch coverage is 45.88235% with 46 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
pgdog-postgres-types/src/datum.rs 44.44% 40 Missing ⚠️
pgdog-postgres-types/src/interval.rs 0.00% 3 Missing ⚠️
pgdog-postgres-types/src/numeric.rs 0.00% 3 Missing ⚠️

📢 Thoughts on this report? Let us know!

@meskill
Copy link
Copy Markdown
Contributor

meskill commented Jun 5, 2026

I like the direction to more explicit handling that return Result instead of implicit nulls somewhere.

But, considering the 2 points you've mentioned I do have strong feeling that the current add implementation is aggregate specific and exactly that's why the types coercion is not relevant here and nulls are handled like this. I'd actually suggest to move this implementation to the aggregate module and use it only there without affecting the base logic for Datum type (and I think we don't have a case for Datum math outside aggregate?).

Also, please note that the things we are discussing are also related to Ord, Eq implementations on Datum.

P.S.: do we need to encode NULL as the part of Datum or Option could be better fit in some cases?

@sgrif
Copy link
Copy Markdown
Contributor Author

sgrif commented Jun 5, 2026

My personal preference would be Option, but NULL as part of Datum matches PG's semantics most closely.

@sgrif
Copy link
Copy Markdown
Contributor Author

sgrif commented Jun 5, 2026 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants