Skip to content

[fix](debug) roll byte sizes up at exact unit boundaries in getByteUint#65778

Closed
arpitjain099 wants to merge 1 commit into
apache:masterfrom
arpitjain099:fix/debugutil-byte-unit-boundary
Closed

[fix](debug) roll byte sizes up at exact unit boundaries in getByteUint#65778
arpitjain099 wants to merge 1 commit into
apache:masterfrom
arpitjain099:fix/debugutil-byte-unit-boundary

Conversation

@arpitjain099

Copy link
Copy Markdown
Contributor

What problem does this PR solve?

getByteUint used strict > for each unit threshold, so a value that lands exactly on a boundary drops to the next smaller unit. A 1 TB data quota shows up as "1024.000 GB", 1 GB as "1024.000 MB", and so on. This is visible in SHOW DATA and in the "data size exceeds quota[...]" error from Database.checkDataSizeQuota, and round quotas like SET DATA QUOTA 1T hit it directly.

The sibling method getUint already uses >= for the same kind of formatting, so this just makes getByteUint consistent with it. Changed the four comparisons to >= so exact multiples of 1024 roll up to the right unit.

Release note

None

Check List

Added boundary cases (1 KB, 1 MB, 1 GB, 1 TB) to DebugUtilTest. The existing non-boundary assertions are unchanged.

@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

morningman
morningman previously approved these changes Jul 23, 2026
Signed-off-by: Arpit Jain <arpitjain099@gmail.com>
@morningman
morningman force-pushed the fix/debugutil-byte-unit-boundary branch from 5372342 to c9a8a24 Compare July 23, 2026 06:10
@morningman

Copy link
Copy Markdown
Contributor

run buildall

@morningman

Copy link
Copy Markdown
Contributor

/review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated review opinion: approve; I found no patch-scoped blocking defect.

Critical checkpoint conclusions:

  • Goal and proof: the descending >= comparisons correctly make exact KB/MB/GB/TB thresholds display as 1 of the larger unit, and the added direct unit cases exercise every changed comparator. Existing cases continue to cover byte and interior KB/MB/GB values.
  • Scope and clarity: the change is minimal and focused on the shared formatter plus its unit test.
  • Concurrency and lifecycle: no new state, locking, asynchronous work, ownership, or lifecycle behavior is introduced.
  • Compatibility and parallel paths: there is no configuration, persistence, storage-format, transaction, FE/BE protocol, or data-write change. All repository consumers use this result as human-readable SHOW/proc/profile/quota/error text; none parses or persists it.
  • Correctness and performance: threshold-minus-one, exact-threshold, and threshold-plus-one behavior is continuous at all four boundaries; zero, negative sentinels, and large values retain their prior behavior. The implementation remains constant-time with no new allocation or observability need.
  • Tests: the leading 1024L keeps all new power expressions in long, and each power-of-two division yields exactly representable 1.0. This review was static-only as required by the review runner; no local build or test was run. CheckStyle was green when reviewed, while FE UT and compile jobs were still queued.
  • User focus: no additional review focus was provided.

Non-blocking metadata note: the PR intentionally changes user-visible SHOW/profile/quota formatting, so the repository's PR standard calls for a release note instead of None. Please summarize that exact KB/MB/GB/TB thresholds now display in the larger unit (for example, 1.000 TB instead of 1024.000 GB).

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-H: Total hot run time: 29392 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c9a8a24938aa7b478814e58bd7fc6ae16cb1d76e, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17824	4160	4160	4160
q2	2054	320	193	193
q3	10266	1460	808	808
q4	4684	466	342	342
q5	7481	853	570	570
q6	188	178	149	149
q7	770	800	631	631
q8	9886	1549	1537	1537
q9	5998	4315	4249	4249
q10	6808	1732	1451	1451
q11	512	374	323	323
q12	773	574	455	455
q13	18123	3268	2681	2681
q14	272	268	242	242
q15	q16	795	768	706	706
q17	1039	945	935	935
q18	7047	5874	5556	5556
q19	1493	1276	1088	1088
q20	836	666	577	577
q21	5690	2631	2441	2441
q22	435	357	298	298
Total cold run time: 102974 ms
Total hot run time: 29392 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4529	4474	4430	4430
q2	296	315	206	206
q3	4634	4953	4467	4467
q4	2058	2177	1416	1416
q5	4392	4269	4324	4269
q6	234	180	130	130
q7	1973	1933	1573	1573
q8	2483	2186	2127	2127
q9	7857	7809	7751	7751
q10	4712	4656	4166	4166
q11	607	442	418	418
q12	963	748	537	537
q13	3305	3642	2915	2915
q14	308	304	286	286
q15	q16	692	730	648	648
q17	1358	1360	1332	1332
q18	8152	7316	7090	7090
q19	1124	1057	1074	1057
q20	2209	2186	1940	1940
q21	5276	4575	4434	4434
q22	522	465	443	443
Total cold run time: 57684 ms
Total hot run time: 51635 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
TPC-DS: Total hot run time: 177310 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c9a8a24938aa7b478814e58bd7fc6ae16cb1d76e, data reload: false

query5	4310	636	480	480
query6	471	236	210	210
query7	4850	627	341	341
query8	342	190	172	172
query9	8771	4080	4066	4066
query10	486	375	334	334
query11	5944	2359	2114	2114
query12	154	102	100	100
query13	1253	607	443	443
query14	6170	5230	4858	4858
query14_1	4243	4231	4248	4231
query15	215	212	186	186
query16	1039	474	482	474
query17	1137	726	588	588
query18	2449	484	343	343
query19	209	188	153	153
query20	120	113	109	109
query21	239	163	137	137
query22	13465	13610	13383	13383
query23	17323	16478	16006	16006
query23_1	16136	16135	16182	16135
query24	7564	1766	1263	1263
query24_1	1291	1260	1301	1260
query25	585	465	398	398
query26	1319	346	222	222
query27	2600	592	376	376
query28	4492	2035	1972	1972
query29	1080	631	507	507
query30	341	264	229	229
query31	1116	1121	1029	1029
query32	112	65	61	61
query33	532	330	271	271
query34	1196	1216	661	661
query35	805	785	692	692
query36	1197	1219	1030	1030
query37	156	105	92	92
query38	1885	1734	1648	1648
query39	885	890	848	848
query39_1	836	873	834	834
query40	243	164	137	137
query41	66	62	63	62
query42	93	91	90	90
query43	320	321	281	281
query44	1424	761	767	761
query45	194	174	170	170
query46	1069	1225	712	712
query47	2123	2088	2003	2003
query48	368	422	291	291
query49	573	414	308	308
query50	1069	436	337	337
query51	10941	10757	10596	10596
query52	86	86	73	73
query53	259	274	207	207
query54	277	227	233	227
query55	76	78	65	65
query56	312	308	287	287
query57	1293	1266	1175	1175
query58	285	255	254	254
query59	1571	1593	1457	1457
query60	302	270	243	243
query61	155	144	150	144
query62	536	490	426	426
query63	242	202	202	202
query64	2848	1046	898	898
query65	4706	4634	4633	4633
query66	1826	512	382	382
query67	29210	29228	29031	29031
query68	3248	1553	1013	1013
query69	451	304	261	261
query70	1037	967	909	909
query71	367	322	313	313
query72	3019	2641	2340	2340
query73	853	775	420	420
query74	5047	4887	4736	4736
query75	2494	2486	2131	2131
query76	2318	1167	783	783
query77	349	369	286	286
query78	11917	11907	11257	11257
query79	1363	1188	723	723
query80	1300	549	468	468
query81	526	327	289	289
query82	581	152	119	119
query83	369	317	290	290
query84	278	157	131	131
query85	976	635	518	518
query86	420	288	284	284
query87	1813	1825	1763	1763
query88	3699	2788	2794	2788
query89	430	375	326	326
query90	1900	197	191	191
query91	202	185	163	163
query92	60	60	53	53
query93	1681	1466	1013	1013
query94	710	355	291	291
query95	787	501	486	486
query96	1021	815	342	342
query97	2623	2618	2495	2495
query98	213	210	200	200
query99	1098	1113	962	962
Total cold run time: 263163 ms
Total hot run time: 177310 ms

@hello-stephen

Copy link
Copy Markdown
Contributor
ClickBench: Total hot run time: 25.14 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c9a8a24938aa7b478814e58bd7fc6ae16cb1d76e, data reload: false

query1	0.01	0.01	0.00
query2	0.09	0.05	0.05
query3	0.26	0.14	0.13
query4	1.62	0.15	0.14
query5	0.24	0.22	0.22
query6	1.26	1.06	1.07
query7	0.04	0.01	0.00
query8	0.05	0.04	0.04
query9	0.39	0.31	0.32
query10	0.53	0.54	0.58
query11	0.20	0.15	0.14
query12	0.18	0.14	0.14
query13	0.47	0.47	0.47
query14	1.00	1.02	1.00
query15	0.60	0.58	0.60
query16	0.32	0.32	0.32
query17	1.09	1.17	1.10
query18	0.23	0.21	0.21
query19	2.03	1.99	1.99
query20	0.01	0.01	0.02
query21	15.43	0.21	0.14
query22	4.99	0.06	0.05
query23	16.14	0.30	0.12
query24	2.96	0.38	0.30
query25	0.11	0.06	0.03
query26	0.75	0.20	0.16
query27	0.04	0.04	0.04
query28	3.44	0.91	0.52
query29	12.53	4.14	3.29
query30	0.27	0.16	0.15
query31	2.76	0.59	0.32
query32	3.22	0.58	0.48
query33	3.10	3.21	3.18
query34	15.61	4.25	3.56
query35	3.54	3.55	3.55
query36	0.55	0.44	0.44
query37	0.10	0.07	0.06
query38	0.05	0.05	0.04
query39	0.04	0.02	0.02
query40	0.18	0.17	0.15
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.04	0.03	0.04
Total cold run time: 96.6 s
Total hot run time: 25.14 s

@hello-stephen

Copy link
Copy Markdown
Contributor

FE UT Coverage Report

Increment line coverage 100.00% (4/4) 🎉
Increment coverage report
Complete coverage report

@morningman

Copy link
Copy Markdown
Contributor

hi @arpitjain099 , this modification may change some behavior, if user already use previous format to match something.
you can see several test cases failed like:

Exception:
java.lang.IllegalStateException: Check tag 'show_data_sql_1024g' failed:
Check tag 'show_data_sql_1024g' failed, line 2, CHAR result mismatch.
Expect cell is: 1024.000 GB
But real is   : 1.000 TB
line 2 mismatch
ExpectRow: [Quota, 1024.000 GB, 1073741824, , ]
RealRow  : [Quota, 1.000 TB, 1073741824, , ]

So I suggest not change it, although it is inconsistent with getUint, but not a very big deal

@morningman
morningman dismissed their stale review July 23, 2026 08:04

dismiss

@arpitjain099

Copy link
Copy Markdown
Contributor Author

Makes sense, thanks for running it down. You're right that the boundary output is load-bearing once anything matches on the current format, and those regression cases make it concrete. The consistency with getUint isn't worth changing output that users and tests already depend on, so I'll close this one out. Appreciate the review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants