Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

command line option --hpx:cores does not work as expected #1705

Closed
justwagle opened this issue Aug 6, 2015 · 2 comments
Closed

command line option --hpx:cores does not work as expected #1705

justwagle opened this issue Aug 6, 2015 · 2 comments
Labels
category: init type: defect
Milestone
0.9.11

Comments

@justwagle
Copy link
Member

The hpx command line option --hpx:cores=XX does not reserve the right number of cores.
For instance,
srun -p marv_noht -N 1 -n 2 ./bin/hello_world --hpx:localities=2 --hpx:threads=4 --hpx:cores=8 --hpx:print-bind --hpx:bind=compact
results in the following output:

*******************************************************************************
locality: 1
   0: PU L#4(P#8), Core L#4(P#4), Socket L#0(P#0), Node L#0(P#0)
   1: PU L#5(P#10), Core L#5(P#5), Socket L#0(P#0), Node L#0(P#0)
   2: PU L#6(P#12), Core L#6(P#6), Socket L#0(P#0), Node L#0(P#0)
   3: PU L#7(P#14), Core L#7(P#7), Socket L#0(P#0), Node L#0(P#0)
*******************************************************************************
locality: 0
   0: PU L#0(P#0), Core L#0(P#0), Socket L#0(P#0), Node L#0(P#0)
   1: PU L#1(P#2), Core L#1(P#1), Socket L#0(P#0), Node L#0(P#0)
   2: PU L#2(P#4), Core L#2(P#2), Socket L#0(P#0), Node L#0(P#0)
   3: PU L#3(P#6), Core L#3(P#3), Socket L#0(P#0), Node L#0(P#0)
hello world from OS-thread 0 on locality 0
hello world from OS-thread 1 on locality 0
hello world from OS-thread 3 on locality 0
hello world from OS-thread 2 on locality 0
hello world from OS-thread 3 on locality 1
hello world from OS-thread 2 on locality 1
hello world from OS-thread 1 on locality 1
hello world from OS-thread 0 on locality 1

I expect the localities to be split between the two numa domains.
output of lstopo for marv_noht:

srun -p marv_noht -N 1 lstopo
Machine (48GB)
  NUMANode L#0 (P#0 24GB) + Socket L#0 + L3 L#0 (20MB)
    L2 L#0 (256KB) + L1d L#0 (32KB) + L1i L#0 (32KB) + Core L#0 + PU L#0 (P#0)
    L2 L#1 (256KB) + L1d L#1 (32KB) + L1i L#1 (32KB) + Core L#1 + PU L#1 (P#2)
    L2 L#2 (256KB) + L1d L#2 (32KB) + L1i L#2 (32KB) + Core L#2 + PU L#2 (P#4)
    L2 L#3 (256KB) + L1d L#3 (32KB) + L1i L#3 (32KB) + Core L#3 + PU L#3 (P#6)
    L2 L#4 (256KB) + L1d L#4 (32KB) + L1i L#4 (32KB) + Core L#4 + PU L#4 (P#8)
    L2 L#5 (256KB) + L1d L#5 (32KB) + L1i L#5 (32KB) + Core L#5 + PU L#5 (P#10)
    L2 L#6 (256KB) + L1d L#6 (32KB) + L1i L#6 (32KB) + Core L#6 + PU L#6 (P#12)
    L2 L#7 (256KB) + L1d L#7 (32KB) + L1i L#7 (32KB) + Core L#7 + PU L#7 (P#14)
  NUMANode L#1 (P#1 24GB) + Socket L#1 + L3 L#1 (20MB)
    L2 L#8 (256KB) + L1d L#8 (32KB) + L1i L#8 (32KB) + Core L#8 + PU L#8 (P#1)
    L2 L#9 (256KB) + L1d L#9 (32KB) + L1i L#9 (32KB) + Core L#9 + PU L#9 (P#3)
    L2 L#10 (256KB) + L1d L#10 (32KB) + L1i L#10 (32KB) + Core L#10 + PU L#10 (P#5)
    L2 L#11 (256KB) + L1d L#11 (32KB) + L1i L#11 (32KB) + Core L#11 + PU L#11 (P#7)
    L2 L#12 (256KB) + L1d L#12 (32KB) + L1i L#12 (32KB) + Core L#12 + PU L#12 (P#9)
    L2 L#13 (256KB) + L1d L#13 (32KB) + L1i L#13 (32KB) + Core L#13 + PU L#13 (P#11)
    L2 L#14 (256KB) + L1d L#14 (32KB) + L1i L#14 (32KB) + Core L#14 + PU L#14 (P#13)
    L2 L#15 (256KB) + L1d L#15 (32KB) + L1i L#15 (32KB) + Core L#15 + PU L#15 (P#15)
@hkaiser hkaiser added this to the 0.9.11 milestone Aug 6, 2015
@hkaiser
Copy link
Member

hkaiser commented Aug 7, 2015

@justwagle Please try branch fixing_1705 and report back whether this fixes your issue.

@justwagle
Copy link
Member Author

branch fixing_1705 fixes the issue. here is the output :

srun -p marv_noht -N 1 -n 2 ./bin/hello_world --hpx:localities=2 --hpx:threads=4 \
    --hpx:cores=8 --hpx:print-bind --hpx:bind=compact
*******************************************************************************
locality: 0
   0: PU L#0(P#0), Core L#0(P#0), Socket L#0(P#0), Node L#0(P#0)
   1: PU L#1(P#2), Core L#1(P#1), Socket L#0(P#0), Node L#0(P#0)
   2: PU L#2(P#4), Core L#2(P#2), Socket L#0(P#0), Node L#0(P#0)
   3: PU L#3(P#6), Core L#3(P#3), Socket L#0(P#0), Node L#0(P#0)
hello world from OS-thread 2 on locality 0
hello world from OS-thread 1 on locality 0
hello world from OS-thread 0 on locality 1
hello world from OS-thread 2 on locality 1
hello world from OS-thread 3 on locality 0
hello world from OS-thread 3 on locality 1
hello world from OS-thread 1 on locality 1
hello world from OS-thread 0 on locality 0
*******************************************************************************
locality: 1
   0: PU L#8(P#1), Core L#8(P#0), Socket L#1(P#1), Node L#1(P#1)
   1: PU L#9(P#3), Core L#9(P#1), Socket L#1(P#1), Node L#1(P#1)
   2: PU L#10(P#5), Core L#10(P#2), Socket L#1(P#1), Node L#1(P#1)
   3: PU L#11(P#7), Core L#11(P#3), Socket L#1(P#1), Node L#1(P#1)

@hkaiser hkaiser mentioned this issue Aug 7, 2015
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
category: init type: defect
Projects
None yet
Milestone
0.9.11
Development

No branches or pull requests
2 participants
@hkaiser @justwagle