Skip to content

[fix](compile) disambiguate Aws::Utils::DateTime ctor in kinesis_conf#63727

Merged
airborne12 merged 1 commit into
apache:masterfrom
airborne12:fix-compile-kinesis-datetime-macos
May 27, 2026
Merged

[fix](compile) disambiguate Aws::Utils::DateTime ctor in kinesis_conf#63727
airborne12 merged 1 commit into
apache:masterfrom
airborne12:fix-compile-kinesis-datetime-macos

Conversation

@airborne12
Copy link
Copy Markdown
Member

What problem does this PR solve?

Issue Number: N/A

Related PR: N/A

Problem Summary:

be/src/load/routine_load/kinesis_conf.cpp:118 constructs Aws::Utils::DateTime from std::stol(...). The AWS SDK declares two constructors:

DateTime(int64_t millisSinceEpoch);
DateTime(double epoch_millis);

On macOS ARM64 (Apple LP64 with libc++), int64_t is typedef'd to long long — distinct from the long that std::stol returns. Both long -> long long and long -> double are same-rank conversions, so overload resolution is ambiguous and the build fails:

error: ambiguous conversion for functional-style cast from 'long' to 'Aws::Utils::DateTime'
  118 |             request.SetTimestamp(Aws::Utils::DateTime(std::stol(it->second)));
note: candidate constructor   DateTime(int64_t millisSinceEpoch);
note: candidate constructor   DateTime(double epoch_millis);

Note: a naive switch to std::stoll would symmetrically break Linux x86_64 (glibc) builds, where int64_t is typedef'd to long — there, long long is ambiguous against (long) vs (double) for the same reason.

The portable fix is to parse with std::stoll (explicit 64-bit, semantically correct for milliseconds-since-epoch) and static_cast<int64_t> to bind to the integer constructor regardless of what int64_t resolves to on the target platform:

request.SetTimestamp(
        Aws::Utils::DateTime(static_cast<int64_t>(std::stoll(it->second))));

Verified with an overload-resolution mini-repro using the same clang-20 toolchain: only the explicit-cast form is unambiguous on both int64_t = long long (macOS) and int64_t = long (Linux x86_64) targets.

Release note

None

Check List (For Author)

  • Test

    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.

    Manual test: verified ./build.sh --be -j 20 now succeeds on macOS ARM64. The runtime value is identical (std::stoll parses the same string to a 64-bit integer; static_cast<int64_t> is a no-op cast on every supported platform).

  • Behavior changed:

    • No.
  • Does this need documentation?

    • No.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

…sis_conf

Aws::Utils::DateTime has both DateTime(int64_t) and DateTime(double)
constructors. Passing the result of std::stol() makes overload
resolution ambiguous on platforms where int64_t is typedef'd to
long long (macOS ARM64), because long->long long and long->double
are both same-rank conversions. Switching to std::stoll() would
break the symmetric Linux x86_64 case where int64_t is long.

Use std::stoll() to parse 64-bit explicitly, then static_cast<int64_t>
to pick the integer constructor unambiguously on every platform.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@airborne12
Copy link
Copy Markdown
Member Author

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-H: Total hot run time: 31378 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 8b1d21bc6c9bdc310a199c88fb0fc9a5fbcc8dba, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17632	4018	4027	4018
q2	q3	10770	1466	801	801
q4	4690	470	353	353
q5	7706	2280	2089	2089
q6	244	182	145	145
q7	1004	793	649	649
q8	9523	1799	1622	1622
q9	5157	4955	4985	4955
q10	6396	2242	1868	1868
q11	430	282	248	248
q12	627	437	296	296
q13	18170	3392	2774	2774
q14	265	258	233	233
q15	q16	817	777	709	709
q17	925	987	971	971
q18	6861	5747	5574	5574
q19	1305	1346	1048	1048
q20	502	427	265	265
q21	5828	2557	2451	2451
q22	442	373	309	309
Total cold run time: 99294 ms
Total hot run time: 31378 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4345	4430	4256	4256
q2	q3	4496	4930	4358	4358
q4	2086	2232	1361	1361
q5	4478	4310	4299	4299
q6	236	176	131	131
q7	2443	1983	1760	1760
q8	2557	2247	2189	2189
q9	8300	8118	8113	8113
q10	4816	4731	4277	4277
q11	608	454	380	380
q12	765	755	541	541
q13	3298	3736	2943	2943
q14	295	314	290	290
q15	q16	740	724	682	682
q17	1356	1373	1483	1373
q18	8105	7384	7419	7384
q19	3564	1137	1075	1075
q20	2233	2209	1948	1948
q21	5284	4574	4478	4478
q22	536	484	424	424
Total cold run time: 60541 ms
Total hot run time: 52262 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-DS: Total hot run time: 171474 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 8b1d21bc6c9bdc310a199c88fb0fc9a5fbcc8dba, data reload: false

query5	4312	663	504	504
query6	313	216	203	203
query7	4289	588	323	323
query8	319	231	245	231
query9	8773	4015	4005	4005
query10	445	339	299	299
query11	5844	2507	2255	2255
query12	184	130	120	120
query13	1280	609	446	446
query14	6070	5451	5152	5152
query14_1	4446	4445	4470	4445
query15	215	207	183	183
query16	983	455	423	423
query17	938	715	582	582
query18	2454	504	363	363
query19	220	213	168	168
query20	136	133	131	131
query21	219	143	118	118
query22	13616	13621	13364	13364
query23	17335	16530	16123	16123
query23_1	16308	16343	16344	16343
query24	7530	1770	1331	1331
query24_1	1311	1332	1315	1315
query25	549	510	453	453
query26	1336	334	176	176
query27	2666	581	349	349
query28	4548	1995	1954	1954
query29	996	642	531	531
query30	305	242	201	201
query31	1141	1093	969	969
query32	85	80	76	76
query33	546	368	300	300
query34	1210	1181	645	645
query35	776	811	697	697
query36	1414	1418	1217	1217
query37	155	106	92	92
query38	3207	3134	3077	3077
query39	930	930	888	888
query39_1	872	867	885	867
query40	239	153	128	128
query41	69	80	69	69
query42	112	109	111	109
query43	331	332	286	286
query44	
query45	222	208	202	202
query46	1103	1243	766	766
query47	2379	2386	2257	2257
query48	424	410	296	296
query49	654	540	386	386
query50	1005	362	253	253
query51	4336	4306	4313	4306
query52	106	108	96	96
query53	255	283	206	206
query54	336	297	269	269
query55	95	93	88	88
query56	315	324	330	324
query57	1425	1434	1364	1364
query58	316	282	280	280
query59	1621	1647	1469	1469
query60	327	329	315	315
query61	157	157	158	157
query62	700	659	581	581
query63	250	204	214	204
query64	2391	811	634	634
query65	
query66	1715	478	360	360
query67	29841	29857	29586	29586
query68	
query69	464	373	313	313
query70	1040	1028	1002	1002
query71	309	264	268	264
query72	3090	2719	2406	2406
query73	817	745	439	439
query74	5148	5046	4797	4797
query75	2732	2618	2272	2272
query76	2336	1143	792	792
query77	401	409	341	341
query78	12373	12409	11894	11894
query79	1422	1018	744	744
query80	646	553	452	452
query81	453	284	254	254
query82	1403	156	125	125
query83	353	281	249	249
query84	303	145	111	111
query85	869	545	460	460
query86	401	332	309	309
query87	3444	3401	3251	3251
query88	3598	2719	2713	2713
query89	450	396	342	342
query90	1939	178	178	178
query91	186	171	142	142
query92	81	78	71	71
query93	1465	1458	827	827
query94	550	347	299	299
query95	673	482	365	365
query96	1032	758	348	348
query97	2704	2727	2601	2601
query98	240	231	227	227
query99	1178	1156	1002	1002
Total cold run time: 254095 ms
Total hot run time: 171474 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.77% (28053/38028)
Line Coverage 57.71% (304624/527852)
Region Coverage 54.79% (254637/464755)
Branch Coverage 56.38% (110131/195345)

@hello-stephen
Copy link
Copy Markdown
Contributor

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/2) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.76% (28050/38028)
Line Coverage 57.68% (304485/527852)
Region Coverage 54.69% (254171/464755)
Branch Coverage 56.31% (109997/195345)

@airborne12
Copy link
Copy Markdown
Member Author

/review

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review result: no blocking issues found. The change resolves the AWS SDK DateTime constructor ambiguity by explicitly selecting the int64_t overload after parsing a 64-bit value, and it preserves the existing request application/error propagation behavior.

Critical checkpoint conclusions:

  • Goal and proof: The PR goal is a compile fix for ambiguous Aws::Utils::DateTime construction; the code accomplishes this directly. PR description reports a macOS ARM64 BE build verification, but I did not run a build in this review.
  • Scope: The actual GitHub PR patch is small and focused on the single ambiguous constructor call.
  • Concurrency: No new concurrency or shared mutable state is introduced.
  • Lifecycle/static initialization: No lifecycle or static initialization behavior is changed.
  • Configuration: No new configuration item is added; existing aws.kinesis.timestamp handling is unchanged except for constructor disambiguation.
  • Compatibility/storage/transactions: No protocol, storage format, transaction, or persistence behavior is involved.
  • Parallel paths: The modified path is the GetShardIterator timestamp application path; other Kinesis request paths do not construct Aws::Utils::DateTime.
  • Error handling: Existing Status propagation is preserved; parse/apply failures remain reported as non-OK Status.
  • Tests: No new test is required for this compile-only overload-resolution fix; existing runtime semantics are unchanged.
  • Observability/performance: No observability change is needed; no meaningful performance impact.

User focus: No additional user-provided review focus was present.

@airborne12 airborne12 merged commit 05690fc into apache:master May 27, 2026
31 of 32 checks passed
@airborne12 airborne12 deleted the fix-compile-kinesis-datetime-macos branch May 27, 2026 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants