Failed to add core affinity to core 64 on linux but not on windows

[pnnehome]

When I run y-cruncher component stress test on Ubuntu it issues an warning in red: failed to add core affinity to core 64. However there is no error if I run the same stress test on windows.

Ubuntu 22.04
Y-cruncher 0.8.2
Epyc 9004 processor

[Mysticial]

Can you show the entire error?

[nnehome]

That was the entire error/warning. To reproduce it, press 1 to do stress test, then press enter to run. Everything will run as usual, except that when allocating memory it will display that warning in red. Both the static and dynamic versions show the same warning. No such warning in windows.

If you like a screenshot I can get it to you tomorrow.

[Mysticial]

So I cannot reproduce this myself because my biggest system only has 16 cores. So I can only debug this through whatever information you can provide.

[Mysticial]

Can you also post the full contents of /proc/cpuinfo?

[nnehome]

The output is very long. So I am pasting only the output for core #1 and #64

processor : 0
vendor_id : AuthenticAMD
cpu family : 25
model : 17
model name : AMD EPYC 9454 48-Core Processor
stepping : 1
microcode : 0xa10113e
cpu MHz : 1500.000
cache size : 1024 KB
physical id : 0
siblings : 96
core id : 0
cpu cores : 48
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 16
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good amd_lbr_v2 nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 invpcid_single hw_pstate ssbd mba perfmon_v2 ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local avx512_bf16 clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin cppc arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif x2avic v_spec_ctrl avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq la57 rdpid overflow_recov succor smca fsrm flush_l1d
bugs : sysret_ss_attrs spectre_v1 spectre_v2 spec_store_bypass srso
bogomips : 5499.87
TLB size : 3584 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 52 bits physical, 57 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

processor : 64
vendor_id : AuthenticAMD
cpu family : 25
model : 17
model name : AMD EPYC 9454 48-Core Processor
stepping : 1
microcode : 0xa10113e
cpu MHz : 1499.209
cache size : 1024 KB
physical id : 0
siblings : 96
core id : 20
cpu cores : 48
apicid : 41
initial apicid : 41
fpu : yes
fpu_exception : yes
cpuid level : 16
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good amd_lbr_v2 nopl nonstop_tsc cpuid extd_apicid aperfmperf rapl pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 invpcid_single hw_pstate ssbd mba perfmon_v2 ibrs ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 erms invpcid cqm rdt_a avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local avx512_bf16 clzero irperf xsaveerptr rdpru wbnoinvd amd_ppin cppc arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif x2avic v_spec_ctrl avx512vbmi umip pku ospke avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq la57 rdpid overflow_recov succor smca fsrm flush_l1d
bugs : sysret_ss_attrs spectre_v1 spectre_v2 spec_store_bypass srso
bogomips : 5499.87
TLB size : 3584 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 52 bits physical, 57 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

[Mysticial]

Can you zip the entire file and attach it? I need to see all of it to see if it's skipping any IDs.

Also, can you post a screenshot of the exact error(s)?

[Mysticial]

Nevermind, I think I found an issue that could cause this. Fix will be rolled out in v0.8.3.

Thanks for reporting this!

[nnehome]

Thanks! Can you shed a bit light on this? It is a bug or something wrong with my end that I need to look into?

[Mysticial]

Off-by-one error in the CPU mask allocation:

Wrong Code:
CPU_ALLOC_SIZE(core_id)

Correct Code:
CPU_ALLOC_SIZE(core_id + 1)

When called with (core_id = 64), the old code only allocates enough mask bits for 64 cores (0-63). Assuming the underlying representation is an array of 64-bit integers, core 64 would spill to the next integer - which never got allocated. So I expect every core ID that's a multiple of 64 to potentially fail.

Nothing you can do it about it on your side.

[nnehome]

I see. Just to make sure, there are 96 logical cores in this CPU. There is no warning for core 65, 66, ..., 95. Only core 64 had a warning.

[Mysticial]

Only core #'s that are divisible by 64 will under allocate from this off-by-one error.

[pnnehome]

Got it. Makes sense. Thanks for the fix.

[Mysticial]

See if latest fixed this: https://github.com/Mysticial/y-cruncher/releases/tag/v0.8.3.9531

[Mysticial]

This should be fixed now. But I can't verify since I don't have such a big system. Feel free to reopen if it's still an issue.