HPCC-22396 Arrange WU Analysis for automated regression

[shamser]

Signed-off-by: Shamser Ahmed shamser.ahmed@lexisnexis.co.uk

Type of change:

• This change is a bug fix (non-breaking change which fixes an issue).
• This change is a new feature (non-breaking change which adds functionality).
• This change improves the code (refactor or other change that does not change the functionality)
• This change fixes warnings (the fix does not alter the functionality or the generated code)
• This change is a breaking change (fix or feature that will cause existing behavior to change).
• This change alters the query API (existing queries will have to be recompiled)

Checklist:

• My code follows the code style of this project.
  • My code does not create any new warnings from compiler, build system, or lint.
• The commit message is properly formatted and free of typos.
  • The commit message title makes sense in a changelog, by itself.
  • The commit is signed.
• My change requires a change to the documentation.
  • I have updated the documentation accordingly, or...
  • I have created a JIRA ticket to update the documentation.
  • Any new interfaces or exported functions are appropriately commented.
• I have read the CONTRIBUTORS document.
• The change has been fully tested:
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • I have checked that this change does not introduce memory leaks.
  • I have used Valgrind or similar tools to check for potential issues.
• I have given due consideration to all of the following potential concerns:
  • Scalability
  • Performance
  • Security
  • Thread-safety
  • Premature optimization
  • Existing deployed queries will not be broken
  • This change fixes the problem, not just the symptom
  • The target branch of this pull request is appropriate for such a change.
• There are no similar instances of the same problem that should be addressed
  • I have addressed them here
  • I have raised JIRA issues to address them separately
• This is a user interface / front-end modification
  • I have tested my changes in multiple modern browsers
  • The component(s) render as expected

Smoketest:

• Send notifications about my Pull Request position in Smoketest queue.
• Test my draft Pull Request.

Testing:

[hpcc-jirabot]

https://track.hpccsystems.com/browse/HPCC-22396
Jira updated

[AttilaVamos]

@shamser These test cases won't be run automatically with Regression Suite in Smoketest and OBT, based on they are not in the default (testing/regress/ecl/) directory.

[shamser]

@AttilaVamos I know. This is the regression that I am trying to make work with the regression suite. There will be a few dozen WU Analysis regressions in the testing/wuanalysis subdirectory. I can execute with

./ecl-test --suiteDir ../wuanalysis run

For each query, there are 2 versions. The first version of each query is a query that demonstrates an issue with the graph. After the execution of the first version of the query, the WU Analysis writes the Analysis in the exceptions/messages section of the WU. The second version reads back the exceptions/messages to check that the issues have been identified. The output to the 2 versions are different and so require 2 different keys.

Unfortunately, I cannot do this with just one version as the analysis will not be written out until the first query completes. This is why we need the 2 versions to compare with 2 different keys. This is the reason why I created the request https://track.hpccsystems.com/browse/HPCC-22418.

[AttilaVamos]

@shamser

For each query, there are 2 versions. The first version of each query is a query that demonstrates an issue with the graph. After the execution of the first version of the query, the WU Analysis writes the Analysis in the exceptions/messages section of the WU. The second version reads back the exceptions/messages to check that the issues have been identified. The output to the 2 versions are different and so require 2 different keys.

I think this is not the versioning what we have implemented and expects same output for all versions. In your case I can see two separated query one for doing nasty thing and the second which one is checking the nastiness of previous.

I mean instead of versioned anahashdistrib1.ecl you have two separated queries:

• testing/wuanalysis/ecl/anahashdistrib1_query.ecl - for doing nasty thing
• testing/wuanalysis/ecl/anahashdistrib1_result.ecl - to check anahashdistrib1_query.ecl did its job

With these separated ECL file you have two separated functionality, two separated keyfile and everything will be fine,
(The Regression Test Engine, without --PQ parameter, always executes the test cases in alphabetical order.)

[ghalliday]

Shamser, it would be good to target this to 7.4 - it will not break anything, and would mean we could start running the analysis regression on 7.4 code.

[shamser]

@AttilaVamos @ghalliday Please can you review.

[shamser]

Ticket to execute WU Analysis regression as part of OBT: https://track.hpccsystems.com/browse/HPCC-22430

[AttilaVamos]

@shamser This logic, file placements and hierarchy seems good and will work if we don't use --PQ in OBT to execute these queries.

[ghalliday]

picky: possibly simpler to say

import Files from $.common;

but fine as it stands

[HPCCSmoketest]

Automated Smoketest: ✅
OS: centos 7.4.1708 (Linux 3.10.0-327.28.3.el7.x86_64)
Sha: 5b73e63
Build: success
Build: success
Install HPCC Platform
HPCC Start: OK

Unit tests result:

Test	total	passed	failed	errors	timeout	elaps
unittest	113	113	0	0	0	34 sec
wutoolTest(Dali)	19	19	0	0	0	2 sec
wutoolTest(Cassandra)	19	19	0	0	0	7 sec

Regression test result:

phase	total	pass	fail	elaps
setup (hthor)	11	11	0	24 sec (00:00:24)
setup (thor)	11	11	0	45 sec (00:00:45)
setup (roxie)	11	11	0	27 sec (00:00:27)
test (hthor)	841	841	0	186 sec (00:03:06)
test (thor)	764	764	0	683 sec (00:11:23)
test (roxie)	917	917	0	243 sec (00:04:03)

HPCC Stop: OK
Time stats:

Prep time	Build time	Package time	Install time	Start time	Test time	Stop time	Summary
28 sec (00:00:28)	214 sec (00:03:34)	0 sec (00:00:00)	2 sec (00:00:02)	18 sec (00:00:18)	1428 sec (00:23:48)	18 sec (00:00:18)	1708 sec (00:28:28)