Add builtin numaid to get NUMA Node ID #2177

Rtoax · 2022-03-30T05:01:56Z

Add node builtin to get NUMA Node ID:

    $ sudo ./bpftrace -e 'BEGIN{printf("numaid= %d\n", numaid);}'
    Attaching 1 probe...
    numaid = 0

It't useful in multi-node system, like numactl --hardware:

    $ numactl --hardware
    [...]
    node   0   1   2   3   4   5   6   7
      0:  10  16  16  16  28  28  22  28
      1:  16  10  16  16  28  28  28  22
      2:  16  16  10  16  22  28  28  28
      3:  16  16  16  10  28  22  28  28
      4:  28  28  22  28  10  16  16  16
      5:  28  28  28  22  16  10  16  16
      6:  22  28  28  28  16  16  10  16
      7:  28  22  28  28  16  16  16  10

Add node builtin variable for getting NUMA node ID.

viktormalik

Not sure if node is the best name here, it's quite generic. What about numa, numanode, or numaid? Any particular reason why it should be node?

Also I'd love to see some actual example where this can be useful.

Rtoax · 2022-03-30T12:27:02Z

Not sure if node is the best name here, it's quite generic. What about numa, numanode, or numaid? Any particular reason why it should be node?

Also I'd love to see some actual example where this can be useful.

Rename node to numaid and example in Commit f605119.

Such as iperf3 server and client running on same numa node get good performance, builtin numaid helpful for that.

bpftrace script

kprobe:net_tx_action,
kprobe:net_rx_action
{
	@[numaid, comm] = count();
}

numa information

$ numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59
node 0 size: 191567 MB
node 0 free: 11676 MB
node 1 cpus: 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79
node 1 size: 193517 MB
node 1 free: 12768 MB
node distances:
node   0   1
  0:  10  21
  1:  21  10

Example 1:

$ taskset -c 1 iperf3 -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 127.0.0.1, port 45214
[  5] local 127.0.0.1 port 5201 connected to 127.0.0.1 port 45216
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  6.04 GBytes  51.9 Gbits/sec
[  5]   1.00-2.00   sec  6.05 GBytes  52.0 Gbits/sec
[  5]   2.00-3.00   sec  6.05 GBytes  52.0 Gbits/sec
[  5]   3.00-4.00   sec  6.05 GBytes  51.9 Gbits/sec
[  5]   4.00-5.00   sec  6.06 GBytes  52.0 Gbits/sec
[  5]   5.00-6.00   sec  6.05 GBytes  51.9 Gbits/sec
[  5]   6.00-7.00   sec  6.06 GBytes  52.0 Gbits/sec
[  5]   7.00-8.00   sec  6.06 GBytes  52.0 Gbits/sec
[  5]   8.00-9.00   sec  6.07 GBytes  52.1 Gbits/sec
[  5]   9.00-10.00  sec  6.04 GBytes  51.8 Gbits/sec
[  5]  10.00-10.00  sec   256 KBytes  45.6 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec  60.5 GBytes  52.0 Gbits/sec                  receiver



$ taskset -c 2 iperf3 -c 0
Connecting to host 0, port 5201
[  5] local 127.0.0.1 port 45216 connected to 127.0.0.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  6.04 GBytes  51.9 Gbits/sec    0   1.19 MBytes
[  5]   1.00-2.00   sec  6.05 GBytes  52.0 Gbits/sec    0   1.19 MBytes
[  5]   2.00-3.00   sec  6.05 GBytes  52.0 Gbits/sec    0   1.19 MBytes
[  5]   3.00-4.00   sec  6.04 GBytes  51.9 Gbits/sec    0   1.19 MBytes
[  5]   4.00-5.00   sec  6.06 GBytes  52.0 Gbits/sec    0   1.19 MBytes
[  5]   5.00-6.00   sec  6.05 GBytes  51.9 Gbits/sec    0   1.19 MBytes
[  5]   6.00-7.00   sec  6.06 GBytes  52.0 Gbits/sec    0   1.19 MBytes
[  5]   7.00-8.00   sec  6.06 GBytes  52.0 Gbits/sec    0   1.19 MBytes
[  5]   8.00-9.00   sec  6.07 GBytes  52.1 Gbits/sec    0   1.19 MBytes
[  5]   9.00-10.00  sec  6.04 GBytes  51.8 Gbits/sec    0   1.19 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  60.5 GBytes  52.0 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  60.5 GBytes  52.0 Gbits/sec                  receiver

iperf Done.

numaid result:

@[0, iperf3]: 2430857

Example 2:

$ taskset -c 1 iperf3 -s
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 127.0.0.1, port 45218
[  5] local 127.0.0.1 port 5201 connected to 127.0.0.1 port 45220
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  2.79 GBytes  24.0 Gbits/sec
[  5]   1.00-2.00   sec  2.79 GBytes  24.0 Gbits/sec
[  5]   2.00-3.00   sec  2.79 GBytes  24.0 Gbits/sec
[  5]   3.00-4.00   sec  2.78 GBytes  23.9 Gbits/sec
[  5]   4.00-5.00   sec  2.80 GBytes  24.0 Gbits/sec
[  5]   5.00-6.00   sec  2.80 GBytes  24.1 Gbits/sec
[  5]   6.00-7.00   sec  2.78 GBytes  23.9 Gbits/sec
[  5]   7.00-8.00   sec  2.78 GBytes  23.9 Gbits/sec
[  5]   8.00-9.00   sec  2.78 GBytes  23.9 Gbits/sec
[  5]   9.00-10.00  sec  2.78 GBytes  23.9 Gbits/sec
[  5]  10.00-10.00  sec  1.12 MBytes  21.9 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec  27.9 GBytes  24.0 Gbits/sec                  receiver


$ taskset -c 21 iperf3 -c 0
Connecting to host 0, port 5201
[  5] local 127.0.0.1 port 45220 connected to 127.0.0.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  2.79 GBytes  24.0 Gbits/sec    0   1023 KBytes
[  5]   1.00-2.00   sec  2.79 GBytes  24.0 Gbits/sec    0   1023 KBytes
[  5]   2.00-3.00   sec  2.79 GBytes  24.0 Gbits/sec    0   1023 KBytes
[  5]   3.00-4.00   sec  2.78 GBytes  23.9 Gbits/sec    0   1023 KBytes
[  5]   4.00-5.00   sec  2.80 GBytes  24.0 Gbits/sec    0   1023 KBytes
[  5]   5.00-6.00   sec  2.80 GBytes  24.1 Gbits/sec    0   1023 KBytes
[  5]   6.00-7.00   sec  2.78 GBytes  23.9 Gbits/sec    0   1023 KBytes
[  5]   7.00-8.00   sec  2.78 GBytes  23.9 Gbits/sec    0   1023 KBytes
[  5]   8.00-9.00   sec  2.78 GBytes  23.9 Gbits/sec    0   1023 KBytes
[  5]   9.00-10.00  sec  2.78 GBytes  23.9 Gbits/sec    0   1023 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  27.9 GBytes  24.0 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  27.9 GBytes  24.0 Gbits/sec                  receiver

iperf Done.

numaid result:

@[0, iperf3]: 456193
@[1, iperf3]: 684853

viktormalik

Ok, this makes sense to me, though I'd like to have a review from at least one another maintainer.

Also please squash the commits, thanks!

Rtoax · 2022-03-31T01:44:41Z

Ok, this makes sense to me, though I'd like to have a review from at least one another maintainer.

Also please squash the commits, thanks!

Ok, squash done.

viktormalik · 2022-04-02T07:11:42Z

It'd be better if you rebased your one commit onto master, i.e. when checked on patch-12 run git rebase master. There should be no merge commit.

Once you have it done, I'll run the tests and help you with potential failures.

Rtoax · 2022-04-02T07:58:40Z

It'd be better if you rebased your one commit onto master, i.e. when checked on patch-12 run git rebase master. There should be no merge commit.

Once you have it done, I'll run the tests and help you with potential failures.

Thanks a lot.

I don't know what's wrong with CI / clang-format (pull_request) Failing after 10s — clang-format:

Run git diff --exit-code
diff --git a/src/ast/irbuilderbpf.cpp b/src/ast/irbuilderbpf.cpp
index 05f2f70..6580ced 1006[4](https://github.com/iovisor/bpftrace/runs/5798421067?check_suite_focus=true#step:8:4)4
--- a/src/ast/irbuilderbpf.cpp
+++ b/src/ast/irbuilderbpf.cpp
@@ -911,7 +911,8 @@ CallInst *IRBuilderBPF::CreateGetNumaId()
   // u32 bpf_get_numa_node_id(void)
   // Return: NUMA Node ID
   FunctionType *getnumaid_func_type = FunctionType::get(getInt[6](https://github.com/iovisor/bpftrace/runs/5798421067?check_suite_focus=true#step:8:6)4Ty(), false);
-  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type, 0);
+  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type,
+                                                          0);
   Constant *getnumaid_func = ConstantExpr::getCast(
       Instruction::IntToPtr,
       getInt64(libbpf::BPF_FUNC_get_numa_node_id),
diff --git a/src/ast/irbuilderbpf.h b/src/ast/irbuilderbpf.h
index f190184..b4ff160 100644
--- a/src/ast/irbuilderbpf.h
+++ b/src/ast/irbuilderbpf.h
@@ -120,[7](https://github.com/iovisor/bpftrace/runs/5798421067?check_suite_focus=true#step:8:7) +[12](https://github.com/iovisor/bpftrace/runs/5798421067?check_suite_focus=true#step:8:12)0,7 @@ public:
   CallInst   *CreateGetPidTgid();
   CallInst   *CreateGetCurrentCgroupId();
   CallInst   *CreateGetUidGid();
-  CallInst   *CreateGetNumaId();
+  CallInst *CreateGetNumaId();
   CallInst   *CreateGetCpuId();
   CallInst   *CreateGetCurrentTask();
   CallInst   *CreateGetRandom();
Error: Process completed with exit code 1.

fbs · 2022-04-02T15:28:24Z

It'd be better if you rebased your one commit onto master, i.e. when checked on patch-12 run git rebase master. There should be no merge commit.
Once you have it done, I'll run the tests and help you with potential failures.

Thanks a lot.

I don't know what's wrong with CI / clang-format (pull_request) Failing after 10s — clang-format:

Run git diff --exit-code
diff --git a/src/ast/irbuilderbpf.cpp b/src/ast/irbuilderbpf.cpp
index 05f2f70..6580ced 1006[4](https://github.com/iovisor/bpftrace/runs/5798421067?check_suite_focus=true#step:8:4)4
--- a/src/ast/irbuilderbpf.cpp
+++ b/src/ast/irbuilderbpf.cpp
@@ -911,7 +911,8 @@ CallInst *IRBuilderBPF::CreateGetNumaId()
   // u32 bpf_get_numa_node_id(void)
   // Return: NUMA Node ID
   FunctionType *getnumaid_func_type = FunctionType::get(getInt[6](https://github.com/iovisor/bpftrace/runs/5798421067?check_suite_focus=true#step:8:6)4Ty(), false);
-  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type, 0);
+  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type,
+                                                          0);
   Constant *getnumaid_func = ConstantExpr::getCast(
       Instruction::IntToPtr,
       getInt64(libbpf::BPF_FUNC_get_numa_node_id),
diff --git a/src/ast/irbuilderbpf.h b/src/ast/irbuilderbpf.h
index f190184..b4ff160 100644
--- a/src/ast/irbuilderbpf.h
+++ b/src/ast/irbuilderbpf.h
@@ -120,[7](https://github.com/iovisor/bpftrace/runs/5798421067?check_suite_focus=true#step:8:7) +[12](https://github.com/iovisor/bpftrace/runs/5798421067?check_suite_focus=true#step:8:12)0,7 @@ public:
   CallInst   *CreateGetPidTgid();
   CallInst   *CreateGetCurrentCgroupId();
   CallInst   *CreateGetUidGid();
-  CallInst   *CreateGetNumaId();
+  CallInst *CreateGetNumaId();
   CallInst   *CreateGetCpuId();
   CallInst   *CreateGetCurrentTask();
   CallInst   *CreateGetRandom();
Error: Process completed with exit code 1.

you can ignore these two if they're consistent with the rest of them. clang-format doesn't work the way we want in every case.

tests/codegen/builtin_numaid.cpp

fbs

looks good! Would it be possible to add a runtime test so we have some kind of coverage for it. Not sure if numactl is possible in the ci

src/ast/irbuilderbpf.cpp

Rtoax · 2022-04-05T06:21:12Z

looks good! Would it be possible to add a runtime test so we have some kind of coverage for it. Not sure if numactl is possible in the ci

ok, i'll try.

src/ast/irbuilderbpf.cpp

CHANGELOG.md

viktormalik · 2022-05-25T09:06:46Z

We did a release, so there's another CHANGELOG conflict, sorry. Also clang-format has some complaints, could you check? Thanks!

clang-format error:

diff --git a/src/ast/irbuilderbpf.h b/src/ast/irbuilderbpf.h
index f[19](https://github.com/iovisor/bpftrace/runs/6589044827?check_suite_focus=true#step:8:20)0184..b4ff160 100644
--- a/src/ast/irbuilderbpf.h
+++ b/src/ast/irbuilderbpf.h
@@ -1[20](https://github.com/iovisor/bpftrace/runs/6589044827?check_suite_focus=true#step:8:21),7 +120,7 @@ public:
   CallInst   *CreateGetPidTgid();
   CallInst   *CreateGetCurrentCgroupId();
   CallInst   *CreateGetUidGid();
-  CallInst   *CreateGetNumaId();
+  CallInst *CreateGetNumaId();
   CallInst   *CreateGetCpuId();
   CallInst   *CreateGetCurrentTask();
   CallInst   *CreateGetRandom();
Error: Process completed with exit code 1.

Yeah, ignore this one and keep it consistent with the rest.

viktormalik · 2022-05-25T09:14:08Z

There's another clang-format issue which should be fixed.

Rtoax · 2022-05-25T09:15:14Z

There's another clang-format issue which should be fixed.

Still:

diff --git a/src/ast/irbuilderbpf.cpp b/src/ast/irbuilderbpf.cpp
index eb1e8c7..31d87c9 100[6](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:7)44
--- a/src/ast/irbuilderbpf.cpp
+++ b/src/ast/irbuilderbpf.cpp
@@ -911,[7](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:8) +911,[8](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:9) @@ CallInst *IRBuilderBPF::CreateGetNumaId()
   // long bpf_get_numa_node_id(void)
   // Return: NUMA Node ID
   FunctionType *getnumaid_func_type = FunctionType::get(getInt32Ty(), false);
-  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type, 0);
+  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type,
+                                                          0);
   Constant *getnumaid_func = ConstantExpr::getCast(
       Instruction::IntToPtr,
       getInt64(libbpf::BPF_FUNC_get_numa_node_id),
diff --git a/src/ast/irbuilderbpf.h b/src/ast/irbuilderbpf.h
index f1[9](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:10)0184..b4ff160 [10](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:11)0644
--- a/src/ast/irbuilderbpf.h
+++ b/src/ast/irbuilderbpf.h
@@ -[12](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:13)0,7 +1[20](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:21),7 @@ public:
   CallInst   *CreateGetPidTgid();
   CallInst   *CreateGetCurrentCgroupId();
   CallInst   *CreateGetUidGid();
-  CallInst   *CreateGetNumaId();
+  CallInst *CreateGetNumaId();
   CallInst   *CreateGetCpuId();
   CallInst   *CreateGetCurrentTask();
   CallInst   *CreateGetRandom();
Error: Process completed with exit code 1.

viktormalik · 2022-05-25T09:17:20Z

diff --git a/src/ast/irbuilderbpf.cpp b/src/ast/irbuilderbpf.cpp
index eb1e8c7..31d87c9 100[6](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:7)44
--- a/src/ast/irbuilderbpf.cpp
+++ b/src/ast/irbuilderbpf.cpp
@@ -911,[7](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:8) +911,[8](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:9) @@ CallInst *IRBuilderBPF::CreateGetNumaId()
   // long bpf_get_numa_node_id(void)
   // Return: NUMA Node ID
   FunctionType *getnumaid_func_type = FunctionType::get(getInt32Ty(), false);
-  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type, 0);
+  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type,
+                                                          0);
   Constant *getnumaid_func = ConstantExpr::getCast(
       Instruction::IntToPtr,
       getInt64(libbpf::BPF_FUNC_get_numa_node_id),

Fix this one (apply the patch proposed by clang-format).

diff --git a/src/ast/irbuilderbpf.h b/src/ast/irbuilderbpf.h
index f1[9](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:10)0184..b4ff160 [10](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:11)0644
--- a/src/ast/irbuilderbpf.h
+++ b/src/ast/irbuilderbpf.h
@@ -[12](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:13)0,7 +1[20](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:21),7 @@ public:
   CallInst   *CreateGetPidTgid();
   CallInst   *CreateGetCurrentCgroupId();
   CallInst   *CreateGetUidGid();
-  CallInst   *CreateGetNumaId();
+  CallInst *CreateGetNumaId();
   CallInst   *CreateGetCpuId();
   CallInst   *CreateGetCurrentTask();
   CallInst   *CreateGetRandom();

Ignore this one.

Thanks!

Rtoax · 2022-05-25T09:18:44Z

diff --git a/src/ast/irbuilderbpf.cpp b/src/ast/irbuilderbpf.cpp
index eb1e8c7..31d87c9 100[6](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:7)44
--- a/src/ast/irbuilderbpf.cpp
+++ b/src/ast/irbuilderbpf.cpp
@@ -911,[7](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:8) +911,[8](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:9) @@ CallInst *IRBuilderBPF::CreateGetNumaId()
   // long bpf_get_numa_node_id(void)
   // Return: NUMA Node ID
   FunctionType *getnumaid_func_type = FunctionType::get(getInt32Ty(), false);
-  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type, 0);
+  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type,
+                                                          0);
   Constant *getnumaid_func = ConstantExpr::getCast(
       Instruction::IntToPtr,
       getInt64(libbpf::BPF_FUNC_get_numa_node_id),

Fix this one (apply the patch proposed by clang-format).

diff --git a/src/ast/irbuilderbpf.h b/src/ast/irbuilderbpf.h
index f1[9](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:10)0184..b4ff160 [10](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:11)0644
--- a/src/ast/irbuilderbpf.h
+++ b/src/ast/irbuilderbpf.h
@@ -[12](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:13)0,7 +1[20](https://github.com/iovisor/bpftrace/runs/6589129901?check_suite_focus=true#step:8:21),7 @@ public:
   CallInst   *CreateGetPidTgid();
   CallInst   *CreateGetCurrentCgroupId();
   CallInst   *CreateGetUidGid();
-  CallInst   *CreateGetNumaId();
+  CallInst *CreateGetNumaId();
   CallInst   *CreateGetCpuId();
   CallInst   *CreateGetCurrentTask();
   CallInst   *CreateGetRandom();

Ignore this one.

Thanks!

Thank you, Is there anything else the code needs to improve?

viktormalik · 2022-05-25T09:22:17Z

Thank you, Is there anything else the code needs to improve?

Nope, otherwise looks good to me!

viktormalik

Codegen test LLVM needs to be updated to the new version including the zext instruction.

Rtoax · 2022-05-25T10:27:11Z

Codegen test LLVM needs to be updated to the new version including the zext instruction.

Just fix it. the clang-format error still can be ignored.

viktormalik · 2022-05-25T11:32:46Z

Codegen test LLVM needs to be updated to the new version including the zext instruction.

Just fix it. the clang-format error still can be ignored.

One part of the clang-format can be fixed, please do it (see my comment above).

Also codegen test is still failing for embedded build.

Rtoax · 2022-05-25T11:46:43Z

Codegen test LLVM needs to be updated to the new version including the zext instruction.

Just fix it. the clang-format error still can be ignored.

One part of the clang-format can be fixed, please do it (see my comment above).

Also codegen test is still failing for embedded build.

clang-format still:

diff --git a/src/ast/irbuilderbpf.cpp b/src/ast/irbuilderbpf.cpp
index eb1e8c7..31d87c9 100[6](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:7)44
--- a/src/ast/irbuilderbpf.cpp
+++ b/src/ast/irbuilderbpf.cpp
@@ -911,[7](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:8) +911,[8](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:9) @@ CallInst *IRBuilderBPF::CreateGetNumaId()
   // long bpf_get_numa_node_id(void)
   // Return: NUMA Node ID
   FunctionType *getnumaid_func_type = FunctionType::get(getInt32Ty(), false);
-  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type, 0);
+  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type,
+                                                          0);
   Constant *getnumaid_func = ConstantExpr::getCast(
       Instruction::IntToPtr,
       getInt64(libbpf::BPF_FUNC_get_numa_node_id),
diff --git a/src/ast/irbuilderbpf.h b/src/ast/irbuilderbpf.h
index f1[9](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:10)0184..b4ff160 [10](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:11)0644
--- a/src/ast/irbuilderbpf.h
+++ b/src/ast/irbuilderbpf.h
@@ -[12](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:13)0,7 +1[20](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:21),7 @@ public:
   CallInst   *CreateGetPidTgid();
   CallInst   *CreateGetCurrentCgroupId();
   CallInst   *CreateGetUidGid();
-  CallInst   *CreateGetNumaId();
+  CallInst *CreateGetNumaId();
   CallInst   *CreateGetCpuId();
   CallInst   *CreateGetCurrentTask();
   CallInst   *CreateGetRandom();
Error: Process completed with exit code 1.

and i don't know how to fix, builtin_numaid.ll test OK:

[....]
4: [   OK   ]	/home/runner/work/bpftrace/bpftrace/tests/codegen/llvm/builtin_nsecs.ll
4: [   OK   ]	/home/runner/work/bpftrace/bpftrace/tests/codegen/llvm/builtin_numaid.ll
4: [   OK   ]	/home/runner/work/bpftrace/bpftrace/tests/codegen/llvm/builtin_pid_tid.ll
[....]
The following tests passed:
Errors while running CTest
Output from these tests are in: /home/runner/work/bpftrace/bpftrace/build-embedded/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
	runtime_test
	tools-parsing-test
	codegen-validator

75% tests passed, 1 tests failed out of 4

Total Test time (real) = 447.76 sec

The following tests FAILED:
	  1 - bpftrace_test (Failed)
Error: Process completed with exit code 8.

On my server there are some errors always with bpftrace_test:

[       OK ] semantic_analyser.call_lhist (3115 ms)
[ RUN      ] semantic_analyser.call_lhist_posparam
stdin:1:33-35: ERROR: $4 used numerically but given "hello"
kprobe:f { @ = lhist(5, $1, $2, $4); }
                                ~~
[       OK ] semantic_analyser.call_lhist_posparam (274 ms)

[       OK ] semantic_analyser.call_buf_expr (518 ms)
[ RUN      ] semantic_analyser.call_buf_posparam
stdin:1:22-24: ERROR: $2 used numerically but given "hello"
kprobe:f { buf(arg0, $2); }
                     ~~
[       OK ] semantic_analyser.call_buf_posparam (273 ms)

[ RUN      ] semantic_analyser.call_stack
stdin:1:12-21: ERROR: $2 used numerically but given "hello"
kprobe:f { kstack($2) }
           ~~~~~~~~~
stdin:1:12-21: ERROR: $2 used numerically but given "hello"
kprobe:f { ustack($2) }
           ~~~~~~~~~
stdin:1:25-27: ERROR: $2 used numerically but given "hello"
kprobe:f { kstack(perf, $2) }
                        ~~
stdin:1:25-27: ERROR: $2 used numerically but given "hello"
kprobe:f { ustack(perf, $2) }
                        ~~
[       OK ] semantic_analyser.call_stack (8160 ms)

In the end of bpftrace_test not relate to this PR:

[==========] 355 tests from 16 test suites ran. (372941 ms total)
[  PASSED  ] 354 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] clang_parser.nested_struct_no_type

 1 FAILED TEST

I try master branch of bpftrace_test, those ERRORs happen too, and totally same:

[==========] 355 tests from 16 test suites ran. (371021 ms total)
[  PASSED  ] 354 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] clang_parser.nested_struct_no_type

 1 FAILED TEST

Rtoax · 2022-05-25T14:30:33Z

Codegen test LLVM needs to be updated to the new version including the zext instruction.

Just fix it. the clang-format error still can be ignored.

One part of the clang-format can be fixed, please do it (see my comment above).

Also codegen test is still failing for embedded build.

I know what happen, see #2240

viktormalik · 2022-05-25T14:32:19Z

clang-format still:

diff --git a/src/ast/irbuilderbpf.cpp b/src/ast/irbuilderbpf.cpp
index eb1e8c7..31d87c9 100[6](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:7)44
--- a/src/ast/irbuilderbpf.cpp
+++ b/src/ast/irbuilderbpf.cpp
@@ -911,[7](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:8) +911,[8](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:9) @@ CallInst *IRBuilderBPF::CreateGetNumaId()
   // long bpf_get_numa_node_id(void)
   // Return: NUMA Node ID
   FunctionType *getnumaid_func_type = FunctionType::get(getInt32Ty(), false);
-  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type, 0);
+  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type,
+                                                          0);
   Constant *getnumaid_func = ConstantExpr::getCast(
       Instruction::IntToPtr,
       getInt64(libbpf::BPF_FUNC_get_numa_node_id),
diff --git a/src/ast/irbuilderbpf.h b/src/ast/irbuilderbpf.h
index f1[9](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:10)0184..b4ff160 [10](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:11)0644
--- a/src/ast/irbuilderbpf.h
+++ b/src/ast/irbuilderbpf.h
@@ -[12](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:13)0,7 +1[20](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:21),7 @@ public:
   CallInst   *CreateGetPidTgid();
   CallInst   *CreateGetCurrentCgroupId();
   CallInst   *CreateGetUidGid();
-  CallInst   *CreateGetNumaId();
+  CallInst *CreateGetNumaId();
   CallInst   *CreateGetCpuId();
   CallInst   *CreateGetCurrentTask();
   CallInst   *CreateGetRandom();
Error: Process completed with exit code 1.

and i don't know how to fix,

Just apply the first patch that clang-format suggests.

builtin_numaid.ll test OK:

[....]
4: [   OK   ]	/home/runner/work/bpftrace/bpftrace/tests/codegen/llvm/builtin_nsecs.ll
4: [   OK   ]	/home/runner/work/bpftrace/bpftrace/tests/codegen/llvm/builtin_numaid.ll
4: [   OK   ]	/home/runner/work/bpftrace/bpftrace/tests/codegen/llvm/builtin_pid_tid.ll
[....]

This test is just validating the LLVM IR files (checks that they have correct syntax). The actual comparison of the produced files with expected ones is done in the codegen suite of bpftrace_test. Unfortunately, the current CI is broken (it doesn't run bpftrace_test at all) but the failure for your codegen can be observed in the embedded tests, too:

Check this one: https://github.com/iovisor/bpftrace/runs/6590317804?check_suite_focus=true

On my server there are some errors always with bpftrace_test:

[       OK ] semantic_analyser.call_lhist (3115 ms)
[ RUN      ] semantic_analyser.call_lhist_posparam
stdin:1:33-35: ERROR: $4 used numerically but given "hello"
kprobe:f { @ = lhist(5, $1, $2, $4); }
                                ~~
[       OK ] semantic_analyser.call_lhist_posparam (274 ms)

[       OK ] semantic_analyser.call_buf_expr (518 ms)
[ RUN      ] semantic_analyser.call_buf_posparam
stdin:1:22-24: ERROR: $2 used numerically but given "hello"
kprobe:f { buf(arg0, $2); }
                     ~~
[       OK ] semantic_analyser.call_buf_posparam (273 ms)

[ RUN      ] semantic_analyser.call_stack
stdin:1:12-21: ERROR: $2 used numerically but given "hello"
kprobe:f { kstack($2) }
           ~~~~~~~~~
stdin:1:12-21: ERROR: $2 used numerically but given "hello"
kprobe:f { ustack($2) }
           ~~~~~~~~~
stdin:1:25-27: ERROR: $2 used numerically but given "hello"
kprobe:f { kstack(perf, $2) }
                        ~~
stdin:1:25-27: ERROR: $2 used numerically but given "hello"
kprobe:f { ustack(perf, $2) }
                        ~~
[       OK ] semantic_analyser.call_stack (8160 ms)

These are not test failures when they say "OK".

In the end of bpftrace_test not relate to this PR:

[==========] 355 tests from 16 test suites ran. (372941 ms total)
[  PASSED  ] 354 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] clang_parser.nested_struct_no_type

 1 FAILED TEST

I try master branch of bpftrace_test, those ERRORs happen too, and totally same:

[==========] 355 tests from 16 test suites ran. (371021 ms total)
[  PASSED  ] 354 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] clang_parser.nested_struct_no_type

 1 FAILED TEST

This is a known issue with LLVM 13, that's why that test is disabled in CI.

Rtoax · 2022-05-25T14:40:16Z

You mean this patch? It's looks like wired.

--- a/src/ast/irbuilderbpf.cpp
+++ b/src/ast/irbuilderbpf.cpp
@@ -911,[7](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:8) +911,[8](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:9) @@ CallInst *IRBuilderBPF::CreateGetNumaId()
   // long bpf_get_numa_node_id(void)
   // Return: NUMA Node ID
   FunctionType *getnumaid_func_type = FunctionType::get(getInt32Ty(), false);
-  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type, 0);
+  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type,
+                                                          0);

Yes, that one. I agree it looks a bit weird, but it follows our coding standard (max 80-char lines). If you don't like it, you can e.g. shorten the variable names so that it fits into one line, but we should be following the rules as much as possible.

Rtoax · 2022-05-25T15:27:39Z

You mean this patch? It's looks like wired.

--- a/src/ast/irbuilderbpf.cpp
+++ b/src/ast/irbuilderbpf.cpp
@@ -911,[7](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:8) +911,[8](https://github.com/iovisor/bpftrace/runs/6590317810?check_suite_focus=true#step:8:9) @@ CallInst *IRBuilderBPF::CreateGetNumaId()
   // long bpf_get_numa_node_id(void)
   // Return: NUMA Node ID
   FunctionType *getnumaid_func_type = FunctionType::get(getInt32Ty(), false);
-  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type, 0);
+  PointerType *getnumaid_func_ptr_type = PointerType::get(getnumaid_func_type,
+                                                          0);

Yes, that one. I agree it looks a bit weird, but it follows our coding standard (max 80-char lines). If you don't like it, you can e.g. shorten the variable names so that it fits into one line, but we should be following the rules as much as possible.

Just one clang-format check error left.

diff --git a/src/ast/irbuilderbpf.h b/src/ast/irbuilderbpf.h
index f190184..b4ff1[6](https://github.com/iovisor/bpftrace/runs/6594935209?check_suite_focus=true#step:8:7)0 100644
--- a/src/ast/irbuilderbpf.h
+++ b/src/ast/irbuilderbpf.h
@@ -120,[7](https://github.com/iovisor/bpftrace/runs/6594935209?check_suite_focus=true#step:8:8) +[12](https://github.com/iovisor/bpftrace/runs/6594935209?check_suite_focus=true#step:8:13)0,7 @@ public:
   CallInst   *CreateGetPidTgid();
   CallInst   *CreateGetCurrentCgroupId();
   CallInst   *CreateGetUidGid();
-  CallInst   *CreateGetNumaId();
+  CallInst *CreateGetNumaId();
   CallInst   *CreateGetCpuId();
   CallInst   *CreateGetCurrentTask();
   CallInst   *CreateGetRandom();
Error: Process completed with exit code 1.

viktormalik

Looks good. One last thing - please rebase onto the current master (there's a CI bug fixed).

Example: NUMA node information: $ numactl --hardware available: 2 nodes (0-1) node 0 cpus: 0 1 2 3 4 5 6 7 ... [...] node 1 cpus: 20 21 22 23 24 25 ... [...] node distances: node 0 1 0: 10 21 1: 21 10 bpftrace script code net_action.bt: kprobe:net_tx_action, kprobe:net_rx_action { @[numaid, comm] = count(); } Example1: ------------------------------------------------------------ iperf3 server: $ taskset -c 1 iperf3 -s [...] [ 5] 0.00-1.00 sec 6.04 GBytes 51.9 Gbits/sec [...] iperf3 client: $ taskset -c 2 iperf3 -c 0 [...] [ 5] 0.00-1.00 sec 6.04 GBytes 51.9 Gbits/sec 0 1.19 MBytes [...] net_action.bt output: @[0, iperf3]: 2430857 Because both cpu1 and cpu2 are on the numa node 0. Example2: ------------------------------------------------------------ iperf3 server: $ taskset -c 1 iperf3 -s [...] [ 5] 0.00-1.00 sec 2.79 GBytes 24.0 Gbits/sec [...] iperf3 client: $ taskset -c 21 iperf3 -c 0 [...] [ 5] 0.00-1.00 sec 2.79 GBytes 24.0 Gbits/sec 0 1023 KBytes [...] net_action.bt output: @[0, iperf3]: 456193 @[1, iperf3]: 684853 Because cpu1 on numa node0, cpu21 on numa node1, Processes that need to communicate running on different nodes cause poor performance, It is very easy to get numa with `numaid`.

Rtoax · 2022-05-26T11:19:52Z

Looks good. One last thing - please rebase onto the current master (there's a CI bug fixed).

rebase done.

viktormalik · 2022-05-26T12:14:20Z

@fbs if you could have one more look at this, we can merge then.

fbs · 2022-05-31T14:04:45Z

looks good, lets get this in

Guess it would be nice to have kernel feature detecting as 5.8 is a relatively new kernel but that doesn't have to block this mr

Rtoax · 2022-06-01T09:16:23Z

@fbs if you could have one more look at this, we can merge then.

Hi, looks like it could merge

Rtoax changed the title ~~Add node builtin to get NUMA Node ID~~ Add builtin node to get NUMA Node ID Mar 30, 2022

Rtoax force-pushed the patch-12 branch from 65354ff to ffddc31 Compare March 30, 2022 05:11

viktormalik reviewed Mar 30, 2022

View reviewed changes

viktormalik approved these changes Mar 30, 2022

View reviewed changes

Rtoax force-pushed the patch-12 branch 3 times, most recently from 0d9d7da to afa7801 Compare March 31, 2022 01:38

Rtoax changed the title ~~Add builtin node to get NUMA Node ID~~ Add builtin numaid to get NUMA Node ID Mar 31, 2022

Rtoax force-pushed the patch-12 branch 2 times, most recently from fcb70ad to afa7801 Compare April 2, 2022 06:33

Rtoax force-pushed the patch-12 branch 2 times, most recently from afa7801 to b56529a Compare April 2, 2022 07:52

fbs reviewed Apr 2, 2022

View reviewed changes

tests/codegen/builtin_numaid.cpp Show resolved Hide resolved

fbs requested changes Apr 2, 2022

View reviewed changes

fbs reviewed Apr 2, 2022

View reviewed changes

src/ast/irbuilderbpf.cpp Outdated Show resolved Hide resolved

Rtoax force-pushed the patch-12 branch 3 times, most recently from 7b5da59 to 4367c54 Compare April 6, 2022 02:07

viktormalik reviewed Apr 6, 2022

View reviewed changes

src/ast/irbuilderbpf.cpp Outdated Show resolved Hide resolved

Rtoax force-pushed the patch-12 branch from 4367c54 to 41546a8 Compare April 12, 2022 07:16

viktormalik requested changes Apr 29, 2022

View reviewed changes

src/ast/irbuilderbpf.cpp Outdated Show resolved Hide resolved

src/ast/irbuilderbpf.cpp Outdated Show resolved Hide resolved

Rtoax force-pushed the patch-12 branch 2 times, most recently from dbd6e5f to bc4a12c Compare May 13, 2022 13:01

Rtoax requested a review from viktormalik May 22, 2022 10:29

viktormalik reviewed May 25, 2022

View reviewed changes

CHANGELOG.md Outdated Show resolved Hide resolved

Rtoax force-pushed the patch-12 branch from bce3f28 to ce5eb86 Compare May 25, 2022 09:08

Rtoax requested a review from viktormalik May 25, 2022 09:12

viktormalik approved these changes May 25, 2022

View reviewed changes

viktormalik requested changes May 25, 2022

View reviewed changes

Rtoax force-pushed the patch-12 branch from ce5eb86 to e9b7cd3 Compare May 25, 2022 10:26

Rtoax requested a review from viktormalik May 25, 2022 10:30

Rtoax force-pushed the patch-12 branch 2 times, most recently from 4b5b00e to 592a116 Compare May 25, 2022 15:10

viktormalik approved these changes May 26, 2022

View reviewed changes

Rtoax force-pushed the patch-12 branch from 592a116 to c4736de Compare May 26, 2022 11:18

viktormalik merged commit 121f73d into bpftrace:master Jun 1, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add builtin numaid to get NUMA Node ID #2177

Add builtin numaid to get NUMA Node ID #2177

Rtoax commented Mar 30, 2022 •

edited

viktormalik left a comment

Rtoax commented Mar 30, 2022 •

edited

viktormalik left a comment

Rtoax commented Mar 31, 2022

viktormalik commented Apr 2, 2022 •

edited

Rtoax commented Apr 2, 2022 •

edited

fbs commented Apr 2, 2022

fbs left a comment

Rtoax commented Apr 5, 2022

viktormalik commented May 25, 2022

viktormalik commented May 25, 2022

Rtoax commented May 25, 2022

viktormalik commented May 25, 2022

Rtoax commented May 25, 2022

viktormalik commented May 25, 2022

viktormalik left a comment

Rtoax commented May 25, 2022 •

edited

viktormalik commented May 25, 2022

Rtoax commented May 25, 2022 •

edited

Rtoax commented May 25, 2022

viktormalik commented May 25, 2022

Rtoax commented May 25, 2022 •

edited by viktormalik

Rtoax commented May 25, 2022

viktormalik left a comment

Rtoax commented May 26, 2022

viktormalik commented May 26, 2022

fbs commented May 31, 2022

Rtoax commented Jun 1, 2022

Add builtin numaid to get NUMA Node ID #2177

Add builtin numaid to get NUMA Node ID #2177

Conversation

Rtoax commented Mar 30, 2022 • edited

viktormalik left a comment

Choose a reason for hiding this comment

Rtoax commented Mar 30, 2022 • edited

bpftrace script

numa information

Example 1:

Example 2:

viktormalik left a comment

Choose a reason for hiding this comment

Rtoax commented Mar 31, 2022

viktormalik commented Apr 2, 2022 • edited

Rtoax commented Apr 2, 2022 • edited

fbs commented Apr 2, 2022

fbs left a comment

Choose a reason for hiding this comment

Rtoax commented Apr 5, 2022

viktormalik commented May 25, 2022

viktormalik commented May 25, 2022

Rtoax commented May 25, 2022

viktormalik commented May 25, 2022

Rtoax commented May 25, 2022

viktormalik commented May 25, 2022

viktormalik left a comment

Choose a reason for hiding this comment

Rtoax commented May 25, 2022 • edited

viktormalik commented May 25, 2022

Rtoax commented May 25, 2022 • edited

Rtoax commented May 25, 2022

viktormalik commented May 25, 2022

Rtoax commented May 25, 2022 • edited by viktormalik

Rtoax commented May 25, 2022

viktormalik left a comment

Choose a reason for hiding this comment

Rtoax commented May 26, 2022

viktormalik commented May 26, 2022

fbs commented May 31, 2022

Rtoax commented Jun 1, 2022

Rtoax commented Mar 30, 2022 •

edited

Rtoax commented Mar 30, 2022 •

edited

viktormalik commented Apr 2, 2022 •

edited

Rtoax commented Apr 2, 2022 •

edited

Rtoax commented May 25, 2022 •

edited

Rtoax commented May 25, 2022 •

edited

Rtoax commented May 25, 2022 •

edited by viktormalik