fix: Performance test failure when vela backend installed#6
Merged
Conversation
Running test_backend_vela_performance.py directly via "pytest" with the vela backend installed results in two failures. If the backend is not installed there are no failures. If the tests are run via "tox" then there are no failures. The intent of the failing test is: given layer performance data in CSV format, ensure that we can parse this to create a valid LayerwisePerfInfo object. However, the parsing code appears to have updated to cope with different versions of the underlying vela backend tool; versions that produce data with different field names (column headings). The test is failing because it checks the __repr__ for the layer performance data classes, which now contain a list of field name aliases ["TFLite_operator", "Original Operator"] instead of a single name. After discussion with Wojciech our conclusion was that the test should use direct attribute checks. It should specify both the list of field names and the corresponding list of actual fields (attributes) to validate. The code under test, performance.py, should also be updated as the __repr__ functions for the layerwise performance data classes are non-standard and only used in the tests. They should be removed entirely. Note that a temporary fix for these failures was submitted as part of MLIA-1423. Resolves: MLIA-1552 Change-Id: I3d4cb9ea7bab71857690d6abf9105d5a23d3fade Signed-off-by: James Edgar <james.edgar@arm.com> Reviewed-on: https://eu-gerrit-2.euhpc.arm.com/c/ml/ecosystem/mlia/+/1149746 Tested-by: expkit <svc_expkit@arm.com> Reviewed-by: Wojciech Boncela <wojciech.boncela@arm.com> IP-review: Isabella Gottardi <isabella.gottardi@arm.com> Reviewed-on: https://eu-gerrit-2.euhpc.arm.com/c/ml/ecosystem/mlia/+/1151451 Reviewed-by: Isabella Gottardi <isabella.gottardi@arm.com>
IsabellaARM
approved these changes
Nov 19, 2025
IsabellaARM
pushed a commit
that referenced
this pull request
May 19, 2026
Signed-off-by: Logan Scully <logan.scully@arm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Running test_backend_vela_performance.py directly via "pytest" with the vela backend installed results in two failures. If the backend is not installed there are no failures. If the tests are run via "tox" then there are no failures.
The intent of the failing test is: given layer performance data in CSV format, ensure that we can parse this to create a valid LayerwisePerfInfo object. However, the parsing code appears to have updated to cope with different versions of the underlying vela backend tool; versions that produce data with different field names (column headings). The test is failing because it checks the repr for the layer performance data classes, which now contain a list of field name aliases
["TFLite_operator", "Original Operator"] instead of a single name.
After discussion with Wojciech our conclusion was that the test should use direct attribute checks. It should specify both the list of field names and the corresponding list of actual fields (attributes) to validate. The code under test, performance.py, should also be updated as the repr functions for the layerwise performance data classes are non-standard and only used in the tests. They should be removed entirely.
Note that a temporary fix for these failures was submitted as part of MLIA-1423.
Resolves: MLIA-1552
Change-Id: I3d4cb9ea7bab71857690d6abf9105d5a23d3fade
Reviewed-on: https://eu-gerrit-2.euhpc.arm.com/c/ml/ecosystem/mlia/+/1149746
Tested-by: expkit svc_expkit@arm.com
Reviewed-by: Wojciech Boncela wojciech.boncela@arm.com
IP-review: Isabella Gottardi isabella.gottardi@arm.com
Reviewed-on: https://eu-gerrit-2.euhpc.arm.com/c/ml/ecosystem/mlia/+/1151451
Reviewed-by: Isabella Gottardi isabella.gottardi@arm.com