Skip to content

Conversation

@grom72
Copy link
Contributor

@grom72 grom72 commented Feb 26, 2025

NLT results analysis reports an error when it can not get source code of the reported warning.
PMDK reports warnings/errors via VOS logging system but the source code is not available.

Static analysis of NLT results does not work properly when
a warning source code is out of scope of the project.
An error is reported in such case and the whole stage status
is changed to FAILURE.

[NLT results] [-ERROR-] Can't create fingerprints for some files:
[NLT results] [-ERROR-] - 'pmdk/.../src/common/set.c' file not found
...
[NLT results] Failing build because analysis result contains errors

To prevent this problem, all PMDK-related messages are filtered out.

NLT results Warnings section will include warnings but will not
provide any detail information about location of this warning
in the source code.

Based on existing format of nlt-errors.json file the most easy way
to filtered all PMDK messages is to use the predefined "pmdk/" preffix
in the fileName:

...
{
        "fileName": "pmdk/src/../src/common/set.c",
        "type": "warning in strict mode",
        "lineStart": 2796,
        "description": "warning in strict mode",
        "message": "util_replica_check() Possible silent data corruption. The unsafe shutdown detection (SDS) is not supported in the pool: /mnt/daos_0/e5737f5f-fa54-45d6-b081-9b2769921610/vos-1\nwarning in strict mode",
        "severity": "NORMAL"
},
...

The VOS logging system should ensure that the filename of each PMDK-related message starts with a "pmdk/" prefix.
The implementation of the solution is provided in daos-stack/daos#14923 PR

@grom72 grom72 force-pushed the grom72/daos-16661 branch 4 times, most recently from 9c7433f to f18f152 Compare February 27, 2025 17:16
@grom72 grom72 changed the title DAOS-16661 ci: Ignore PMDK warnings detail analysis in NLT DAOS-16661 ci: Ignore detail analysis of PMDK warnings in NLT Feb 27, 2025
@grom72 grom72 force-pushed the grom72/daos-16661 branch 2 times, most recently from f39f869 to b4b67db Compare February 28, 2025 08:02
@grom72 grom72 marked this pull request as ready for review February 28, 2025 12:05
@grom72 grom72 requested a review from ryon-jensen February 28, 2025 22:31
Static analysis of NLT results does not work properly when
a warning source code is out of scope of the project.
An error is reported in such case and  the whole stage status
is changed to FAILURE.
```
[NLT results] [-ERROR-] Can't create fingerprints for some files:
[NLT results] [-ERROR-] - 'pmdk/.../src/common/set.c' file not found
...
[NLT results] Failing build because analysis result contains errors
```

To prevent this problem, all PMDK-related messages are filtered out.

NLT results Warnings section will include warnings but will not
provide any detail information about location of this warning
in the source code.

Based on existing format of nlt-errors.json file the most easy way
to filtered all PMDK messages is to use the predefined "pmdk/" preffix
in the fileName:
```
...
{
        "fileName": "pmdk/src/../src/common/set.c",
        "type": "warning in strict mode",
        "lineStart": 2796,
        "description": "warning in strict mode",
        "message": "util_replica_check() Possible silent data corruption. The unsafe shutdown detection (SDS) is not supported in the pool: /mnt/daos_0/e5737f5f-fa54-45d6-b081-9b2769921610/vos-1\nwarning in strict mode",
        "severity": "NORMAL"
},
...
```
vos logging system should ensure that the file name of each PMDK
related message starts with "pmdk/" prefix.
The implementation of the solution is provided in
daos-stack/daos#14923 PR

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
@grom72 grom72 force-pushed the grom72/daos-16661 branch from b4b67db to 4adcbc5 Compare March 1, 2025 16:10
@grom72 grom72 requested review from daltonbohning and phender March 3, 2025 20:20
This change only may affect NLT and Fault Injection tests
No need to execute HW and VM Functional Tests

Skip-build-el9-rpm: true
Skip-build-leap15-rpm: true
Skip-build-ubuntu20-rpm: true
Skip-build-leap15-icc: true

Skip-unit-test: true
Skip-unit-test-memcheck: true
Skip-bullseye: true

Skip-fault-injection-test: false
Skip-func-test-vm-all: true
Skip-func-test-hw: true

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
@grom72
Copy link
Contributor Author

grom72 commented Mar 4, 2025

@grom72 grom72 requested a review from a team March 4, 2025 21:08
grom72 added 2 commits March 4, 2025 22:09
This change only may affect NLT and Fault Injection tests
No need to execute HW and VM Functional Tests

Skip-build-el9-rpm: true
Skip-build-leap15-rpm: true
Skip-build-ubuntu20-rpm: true
Skip-build-leap15-icc: true

Skip-unit-test: true
Skip-unit-test-memcheck: true
Skip-bullseye: true

Skip-fault-injection-test: false
Skip-func-test-vm-all: true
Skip-func-test-hw: true

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
@grom72
Copy link
Contributor Author

grom72 commented Mar 5, 2025

@daltonbohning can we land with this PR not waiting for all upstream validation to be completed?
This change does not have any impact to HW and Functional tests and it has already been confirmed that the solution works as expected. It is a blocker for https://daosio.atlassian.net/jira/software/c/projects/DAOS/issues/DAOS-16661 and daos-stack/daos#14923

@grom72 grom72 requested a review from a team March 5, 2025 14:36
@daltonbohning
Copy link
Contributor

Woops, too late. It actually finished now :)

@daltonbohning daltonbohning merged commit 5fc4967 into master Mar 5, 2025
10 checks passed
@daltonbohning daltonbohning deleted the grom72/daos-16661 branch March 5, 2025 22:53
grom72 added a commit to daos-stack/daos that referenced this pull request Mar 6, 2025
daos-stack/pipeline-lib#457 landed on
master.

This change only may affect NLT and Fault Injection tests
No need to execute HW and VM Functional Tests

Skip-unit-test: true
Skip-unit-test-memcheck: true
Skip-bullseye: true

Skip-fault-injection-test: false
Skip-func-test-vm-all: true
Skip-func-test-hw: true

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@hpe.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants