Skip to content

Execution size columns silently truncate: Long byte counts stored into INT columns via size.toInt #6978

Description

@mengw15

What happened?

updateResultSize / updateRuntimeStatsSize / updateConsoleMessageSize (WorkflowExecutionsResource.scala:407/430/455) store a Long byte count into INT columns via Integer.valueOf(size.toInt). The columns result_size, runtime_stats_size, console_messages_size are INT (sql/texera_ddl.sql). Scala's Long.toInt keeps only the low 32 bits with no exception, so a size ≥ 2 GiB wraps — values in [2 GiB, 4 GiB) become negative.

UserQuotaResource (UserQuotaResource.scala:144/153-156/165) sums these three columns to compute a user's storage quota, so a truncated/negative value corrupts the reported quota. With >2 GB results now supported (BigObject), this is reachable in practice.

Fix: widen the three columns to BIGINT and store the Long directly (drop .toInt).

How to reproduce?

Materialize an execution result larger than 2 GiB. The stored result_size is the low-32-bit truncation of the real byte count (negative for sizes in [2 GiB, 4 GiB)), and the user's quota total in UserQuotaResource reflects the wrong value.

Version

1.1.0-incubating (Pre-release/Master)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions