tests: fix t151 recv_runcmd test code. #1765
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kang-hyuck
force-pushed
the
fix-t151_recv_runcmd
branch
from
a7b894b
to
53a6148
Compare
|
Hello @kang-hyuck, thank you for your contribution. :D |
kang-hyuck
force-pushed
the
fix-t151_recv_runcmd
branch
from
53a6148
to
e503d16
Compare
|
I've tested it in Ubuntu 22.04 x86_64. (venv) vagrant@vagrant:~/uftrace_kang-kyuck/tests$ ./runtest.py -d 151 -j3
Start 1 tests without worker pool
Compiler gcc clang
Runtime test case pg finstrument-fu fpatchable-fun pg finstrument-fu fpatchable-fun
------------------------: O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os
151 recv_runcmd : OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK
runtime test stats
====================
total 30 Tests executed (success: 100.00%)
OK: 30 Test succeeded
OK: 0 Test succeeded (with some fixup)
NG: 0 Different test result
NZ: 0 Non-zero return value
SG: 0 Abnormal exit by signal
TM: 0 Test ran too long
BI: 0 Build failed
LA: 0 Unsupported Language
SK: 0 Skipped
(venv) vagrant@vagrant:~/uftrace_kang-kyuck/tests$ ./runtest.py -d 151 -j8
Start 1 tests without worker pool
Compiler gcc clang
Runtime test case pg finstrument-fu fpatchable-fun pg finstrument-fu fpatchable-fun
------------------------: O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os
t151_recv_runcmd: diff result of clang -finstrument-functions -O1
--- expect 2023-07-19 22:26:49.425558432 +0000
+++ result 2023-07-19 22:26:49.425558432 +0000
@@ -1,9 +1 @@
-main() {
- a() {
- b() {
- c() {
- getpid();
- } /* c */
- } /* b */
- } /* a */
- } /* main */
+
t151_recv_runcmd: diff result of clang -finstrument-functions -O2
--- expect 2023-07-19 22:26:49.525608436 +0000
+++ result 2023-07-19 22:26:49.525608436 +0000
@@ -1,9 +1 @@
-main() {
- a() {
- b() {
- c() {
- getpid();
- } /* c */
- } /* b */
- } /* a */
- } /* main */
+
151 recv_runcmd : OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK NG NG OK OK OK OK OK OK OK
runtime test stats
====================
total 30 Tests executed (success: 93.33%)
OK: 28 Test succeeded
OK: 0 Test succeeded (with some fixup)
NG: 2 Different test result
NZ: 0 Non-zero return value
SG: 0 Abnormal exit by signal
TM: 0 Test ran too long
BI: 0 Build failed
LA: 0 Unsupported Language
SK: 0 Skipped
(venv) vagrant@vagrant:~/uftrace_kang-kyuck/tests$ ./runtest.py -d 151 -j8
Start 1 tests without worker pool
Compiler gcc clang
Runtime test case pg finstrument-fu fpatchable-fun pg finstrument-fu fpatchable-fun
------------------------: O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os
151 recv_runcmd : OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK
runtime test stats
====================
total 30 Tests executed (success: 100.00%)
OK: 30 Test succeeded
OK: 0 Test succeeded (with some fixup)
NG: 0 Different test result
NZ: 0 Non-zero return value
SG: 0 Abnormal exit by signal
TM: 0 Test ran too long
BI: 0 Build failed
LA: 0 Unsupported Language
SK: 0 Skipped
(venv) vagrant@vagrant:~/uftrace_kang-kyuck/tests$ ./runtest.py -d 151 -j8
Start 1 tests without worker pool
Compiler gcc clang
Runtime test case pg finstrument-fu fpatchable-fun pg finstrument-fu fpatchable-fun
------------------------: O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os
t151_recv_runcmd: diff result of clang -pg -O2
--- expect 2023-07-19 22:33:28.036779923 +0000
+++ result 2023-07-19 22:33:28.036779923 +0000
@@ -1,9 +1 @@
-main() {
- a() {
- b() {
- c() {
- getpid();
- } /* c */
- } /* b */
- } /* a */
- } /* main */
+
151 recv_runcmd : OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK OK NG OK OK OK OK OK OK OK OK OK OK OK OK
runtime test stats
====================
total 30 Tests executed (success: 96.67%)
OK: 29 Test succeeded
OK: 0 Test succeeded (with some fixup)
NG: 1 Different test result
NZ: 0 Non-zero return value
SG: 0 Abnormal exit by signal
TM: 0 Test ran too long
BI: 0 Build failed
LA: 0 Unsupported Language
SK: 0 Skipped |
|
Hi @kang-hyuck, thanks for the patch. It looks good but I still see some random failures as @MichelleJin12 mentioned. |
kang-hyuck
force-pushed
the
fix-t151_recv_runcmd
branch
2 times, most recently
from
da2e4a1
to
13ac4fa
Compare
|
thank you for your reviews ! :) @MichelleJin12 @honggyukim In addition, I found that when |
|
This is my gcc and clang version $ gcc --version
gcc (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0
$ clang --version
Ubuntu clang version 14.0.0-1ubuntu1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/binWhen I tested with commit 13ac4fa, the NG rate seems to have decreased, but it still happens. vagrant@vagrant:~/uftrace_kang-kyuck/tests$ ./runtest.py -d 151
Start 1 tests without worker pool
Compiler gcc clang
Runtime test case pg finstrument-fu fpatchable-fun pg finstrument-fu fpatchable-fun
------------------------: O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os O0 O1 O2 O3 Os
t151_recv_runcmd: diff result of gcc -fpatchable-function-entry=5 -Os -fno-ipa-sra
--- expect 2023-07-21 00:23:41.352019764 +0000
+++ result 2023-07-21 00:23:41.352019764 +0000
@@ -1,9 +1 @@
-main() {
- a() {
- b() {
- c() {
- getpid();
- } /* c */
- } /* b */
- } /* a */
- } /* main */
+
t151_recv_runcmd: diff result of clang -fpatchable-function-entry=5 -O0
--- expect 2023-07-21 00:23:48.011347910 +0000
+++ result 2023-07-21 00:23:48.011347910 +0000
@@ -1,9 +1 @@
-main() {
- a() {
- b() {
- c() {
- getpid();
- } /* c */
- } /* b */
- } /* a */
- } /* main */
+
151 recv_runcmd : OK OK OK OK OK OK OK OK OK OK OK OK OK OK NG OK OK OK OK OK OK OK OK OK OK NG OK OK OK OK
I think the NG is caused by a random port that's already in use, so I put in code to check if that port is in use, but it's not a complete solution. def gen_port(self):
random.seed(time.time())
port = False
while (port == False):
try:
self.port = random.randint(40000, 60000)
with socketserver.TCPServer(("127.0.0.1", self.port), None) as s:
port = True
pass
except OSError as e:
port = False
break
print(self.port)If it's blocked by sp.call, I guess we could use timeout, which is an argument to sp.call. |
namhyung
reviewed
Jul 22, 2023
| self.recv_p.terminate() | ||
|
|
||
| out = self.recv_p.communicate()[0].decode(errors='ignore') | ||
| self.file_p.write(out) | ||
| self.file_p.flush() | ||
| self.file_p.close() |
There was a problem hiding this comment.
I think you need to close the epoll object here.
tests/t151_recv_runcmd.py
Outdated
| @@ -42,10 +46,14 @@ def prerun(self, timeout): | |||
| record_cmd += ['t-' + self.name] | |||
| self.pr_debug('prerun command: ' + ' '.join(record_cmd)) | |||
| sp.call(record_cmd, stderr=sp.PIPE) | |||
|
|
|||
| epolls.poll(timeout=5) | |||
There was a problem hiding this comment.
Can we use timeout parameter in the prerun() method directly?
wait for recv_cmd server to process record_cmd client's data. Fixed: namhyung#1578 recv runcmd run test need result (x86_64, aarch64) Signed-off-by: kang-hyuck <kang-hyuck@naver.com>
kang-hyuck
force-pushed
the
fix-t151_recv_runcmd
branch
from
13ac4fa
to
bc8c36a
Compare
|
I found that there is a temporarily NG port between 40000~50000. But after some time, all tests were OK even if I use NG port before. I found that all test code where using here is same issue case code:
|
namhyung
approved these changes
Jul 23, 2023
| self.recv_p.terminate() | ||
|
|
||
| out = self.recv_p.communicate()[0].decode(errors='ignore') | ||
| self.file_p.write(out) | ||
| self.file_p.flush() | ||
| self.file_p.close() |
There was a problem hiding this comment.
I think close() implies flush() but maybe it's better to have it explicitly.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
problem:
Fix the issue that prints no results in prerun test. Nonetheless the recv server hasn't finished processing the client's data, recv server is forcibly terminated and reading the results from the pipe. recv_cmd server needs time to process record_cmd client's data.
solution:
Use epoll to check server's pipe status.
Fixed: #1578 recv runcmd run test need result (x86_64, aarch64)