reports: enforce UNKNOWN keyword

[3mkusiak]

The logic for checking if test should fail by default is reversed, thus all passes or errors are treated as unknowns. Enforce checking the keyword "UNKNOWN" for the tests.

Fixes:
Dasharo/dasharo-issues#1304

[m-iwanicki]

One small change, apart from that it looks good:

[OK]            PCI configuration space and topology
[OK]            USB devices and topology
[OK]            Super I/O configuration
[OK]            EC configuration
[ERROR]         MSRs
[OK]            SMBIOS tables
[OK]            BIOS information
[OK]            CMOS NVRAM
[UNKNOWN]       Intel configuration registers
[OK]            GPIO configuration C header files
[OK]            kernel dmesg
[OK]            ACPI tables
[UNKNOWN]       Audio devices configuration
[OK]            CPU info
[OK]            I/O ports
[OK]            Input bus types
[ERROR]         Firmware image
[OK]            I2C bus
[UNKNOWN]       ACPI tables
[OK]            Touchpad information
[ERROR]         DIMMs information
[UNKNOWN]       CBMEM table information
[UNKNOWN]       CBMEM console
[ERROR]         TPM information
[UNKNOWN]       AMT information
[OK]            ME information
[UNKNOWN]       Graphics VBT

There is still problem that some items will always be unknown as we are always passing UNKNOWN string to this function e.g.

• Audio devices configuration
• ACPI tables - the second one (I think it's duplicate)
• Graphics VBT

I think to close the issue 100% you would have to fix that too

[DaniilKl]

One small change, apart from that it looks good:

[OK]            PCI configuration space and topology
[OK]            USB devices and topology
[OK]            Super I/O configuration
[OK]            EC configuration
[ERROR]         MSRs
[OK]            SMBIOS tables
[OK]            BIOS information
[OK]            CMOS NVRAM
[UNKNOWN]       Intel configuration registers
[OK]            GPIO configuration C header files
[OK]            kernel dmesg
[OK]            ACPI tables
[UNKNOWN]       Audio devices configuration
[OK]            CPU info
[OK]            I/O ports
[OK]            Input bus types
[ERROR]         Firmware image
[OK]            I2C bus
[UNKNOWN]       ACPI tables
[OK]            Touchpad information
[ERROR]         DIMMs information
[UNKNOWN]       CBMEM table information
[UNKNOWN]       CBMEM console
[ERROR]         TPM information
[UNKNOWN]       AMT information
[OK]            ME information
[UNKNOWN]       Graphics VBT

There is still problem that some items will always be unknown as we are always passing UNKNOWN string to this function e.g.

* `Audio devices configuration`

* `ACPI tables` - the second one (I think it's duplicate)

* `Graphics VBT`

I think to close the issue 100% you would have to fix that too

@3mkusiak, have you checked, why we are not providing the error log file as a second argument to the update_result function in these 3 cases? Maybe the reason is, that these 3 checks do not produce output, basing on which we can conclude the status. If that is the case - then it is not an issue but a feature and you do not need to fix it. Please, provide a confirmation for my assumption.

If my assumption is correct - provide comments near the update_result calls explaining the situation, so we wont waste time guessing in future.

[3mkusiak]

Maybe the reason is, that these 3 checks do not produce output, basing on which we can conclude the status. If that is the case - then it is not an issue but a feature and you do not need to fix it. Please, provide a confirmation for my assumption.

Dunno how about the rest, but the ACPI tables one silently fails, so I wouldn't call it a feature. Either way, Imo we should let it fail, not enforce unknowns, this is just a workaround for lazy coding. Dunno, why it's implemented the way it is as there are no descriptions in commits.

[DaniilKl]

Dunno, why it's implemented the way it is as there are no descriptions in commits.

Does not matter how it is implemented, if it works as we expect it should. Does the missing log files an expected behaviour in these 3 cases?

[m-iwanicki]

@DaniilKl it's likely because all 3 dump more than 1 log and update_result function expects only one.
Also those ACPI tables are not duplicates and do something different, so I think they should have different name

[DaniilKl]

it's likely because all 3 dump more than 1 log and update_result function expects only one.

Why can't we use any of the log files to check the status? Or use all three log files, but name the statuses for every log file differently?

[3mkusiak]

Does the missing log files an expected behaviour in these 3 cases?

I assume you ask if it's expected these commands do not produce logs. They don't, not the "classic" form we expect at least, but this is on our side to collect this logs. The tasks might silently fail and will never fix them. I might as well, remove printing updates for these actions and just perform them in silence.

[DaniilKl]

@3mkusiak, open issues for these 3 cases describing the situation in details and link the issue in the source code as a comment. We will get to them later.

[3mkusiak]

Repro:
rpr.txt

[m-iwanicki]

Works, tested on QEMU without emulation:

[OK]            PCI configuration space and topology
[UNKNOWN]       USB devices and topology
[OK]            Super I/O configuration
[UNKNOWN]       EC configuration
[ERROR]         MSRs
[OK]            SMBIOS tables
[OK]            BIOS information
[OK]            CMOS NVRAM
[UNKNOWN]       Intel configuration registers
[OK]            GPIO configuration C header files
[OK]            kernel dmesg
[OK]            ACPI tables
[UNKNOWN]       Audio devices configuration
[OK]            CPU info
[OK]            I/O ports
[OK]            Input bus types
[OK]            I2C bus
[UNKNOWN]       ACPI tables
[OK]            Touchpad information
[ERROR]         DIMMs information
[ERROR]         CBMEM table information
[ERROR]         CBMEM console
[ERROR]         TPM information
[ERROR]         AMT information
[OK]            ME information
[UNKNOWN]       Graphics VBT