chore(agw): 'nose' is replaced by 'pytest' for S1AP integration tests

[mpfirrmann]

Summary

Follow-up PR to #13052. The two PRs are independent to not break master by merge order. The duplicate code will be adapted before the second merge.
After that, there will be a cleanup PR for the remaining occurrences of nose/nosetests and the handling of coverage.
Additionally ...
... pytest did complain about some python imports --> adapted according to the other tests.
... flaky in pytest is enabled by default (https://github.com/box/flaky#activating-the-plugin) --> argument is removed

Test Plan

With all three VMs for AGW testing:
vagrant@magma-test:~/magma/lte/gateway/python/integ_test$ make integ_test

LTE integ_test CI runs on my fork:
https://github.com/mpfirrmann/magma/actions/runs/2548683727
https://github.com/mpfirrmann/magma/actions/runs/2550991806
A different unit test fails at both runs, but probably not due to the code change here.

Additional Information

The test results are stored in .xml files. On this basis, CI html reports are created. The style of the result .xml changes slightly with the change from XUnit XML to JUnit XML.
For s1aptests/test_attach_detatch.xml:
current XUnit XML

<testsuite name="nosetests" tests="1" errors="0" failures="0" skip="0">
<testcase classname="test_attach_detach.TestAttachDetach" name="test_attach_detach" time="5.273">
<system-out>
<![CDATA[ Test Case Execution Count: 1 Start time 06:56:28 ************************* Enb tester config AGW is stateless APN Correction configured Health service is disabled Using subscriber IMSI IMSI001010000000001 Using IMEI 3805468432113171 Using subscriber IMSI IMSI001010000000002 Using IMEI 3805468432113172 ************************* UE device config for ue_id 1 ************************* UE device config for ue_id 2 ************************* Configuring IP block ************************* Waiting for IP changes to propagate ************************* S1 setup ************************* UE App config ************************* Running End to End attach for UE id 1 ************************* Running UE detach for UE id 1 ************************* Running End to End attach for UE id 2 ************************* Running UE detach for UE id 2 ************************* send SCTP SHUTDOWN Keys left in Redis (list should be empty)[ ] Entries in s1ap_imsi_map (should be zero): 0 Entries left in hashtables (should be zero): 0 Entries in mme_ueip_imsi_map (should be zero): 0 ]]>
</system-out>
</testcase>
</testsuite>

new JUnit XML

<testsuites>
<testsuite name="pytest" errors="0" failures="0" skipped="0" tests="1" time="5.188" timestamp="2022-07-06T07:36:02.476018" hostname="magma-test">
<testcase classname="s1aptests.test_attach_detach.TestAttachDetach" name="test_attach_detach" time="4.998">
<system-out>--------------------------------- Captured Out --------------------------------- Test Case Execution Count: 1 Start time 07:36:02 ************************* Enb tester config AGW is stateless APN Correction configured Health service is disabled Using subscriber IMSI IMSI001010000000001 Using IMEI 3805468432113171 Using subscriber IMSI IMSI001010000000002 Using IMEI 3805468432113172 ************************* UE device config for ue_id 1 ************************* UE device config for ue_id 2 ************************* Configuring IP block ************************* Waiting for IP changes to propagate ************************* S1 setup ************************* UE App config ************************* Running End to End attach for UE id 1 ************************* Running UE detach for UE id 1 ************************* Running End to End attach for UE id 2 ************************* Running UE detach for UE id 2 ************************* send SCTP SHUTDOWN Keys left in Redis (list should be empty)[ ] Entries in s1ap_imsi_map (should be zero): 0 Entries left in hashtables (should be zero): 0 Entries in mme_ueip_imsi_map (should be zero): 0 </system-out>
</testcase>
</testsuite>
</testsuites>

• This change is backwards-breaking

[github-actions]

Thanks for opening a PR! 💯

A couple initial guidelines

• All commits must be signed off. This is enforced by DCO check.
• All PR titles must follow the semantic commits format. This is enforced by Semantic PR.

Howto

• Reviews. The "Reviewers" listed for this PR are the Magma maintainers who will shepherd it.
• Checks. All required CI checks must pass before merge.
• Merge. Once approved and passing CI checks, use the ready2merge label to indicate the maintainers can merge your PR.

More info

Please take a moment to read through the Magma project's

• Contributing Conventions for norms around contributed code

If this is your first Magma PR, also consider reading

• Developer Onboarding for onboarding as a new Magma developer
• Development Workflow for guidance on your first pull request
• CI Checks for points of contact for failing or flaky CI checks
• GitHub-to-Slack mappings for Magma maintainers for guidance on how to contact maintainers on Slack

[github-actions]

feg-workflow

    2 files  203 suites   40s ⏱️
374 tests 374 ✔️ 0 💤 0 ❌
388 runs  388 ✔️ 0 💤 0 ❌

Results for commit 725557d.

♻️ This comment has been updated with latest results.

[github-actions]

dp-workflow

14 tests   14 ✔️  3m 40s ⏱️
  1 suites    0 💤
  1 files      0 ❌

Results for commit 725557d.

♻️ This comment has been updated with latest results.

[github-actions]

agw-workflow

615 tests   611 ✔️  3m 52s ⏱️
    2 suites      4 💤
    2 files        0 ❌

Results for commit 725557d.

♻️ This comment has been updated with latest results.

[sebathomas]

It might be good to have a CI run of the integration tests on your fork.

[VinashakAnkitAman]

LGTM

On CI side, are the xml reports same from existing nosetest-xunit and pytest-junit?

We currently use xunit-viewer to generate CI html reports (https://storage.googleapis.com/magma-ci.appspot.com/lte_integ_test_a4934e6e076a23264ccaa65df327bf119857fe26.html). Is that compatible with new xml files generated by pytest-junit?

[mpfirrmann]

@VinashakAnkitAman, that is good to know. What does the CI currently check for in the .xml files? From what I saw, the general result plus the stdout is written there. This can be done with pytest as well. Added example outputs to the PR description.

[Neudrino]

I think the important thing about the output format is, that it integrates well with Github actions result display as the community platform and infrastructure. E.g. as for this job. The integration there seems to be quite good already.

[mpfirrmann]

@VinashakAnkitAman, did you by chance had time to look at the .xml outputs of the pytest tests and how they would fit into the CI? With your approval we could merge this PR.

[VinashakAnkitAman]

@VinashakAnkitAman, did you by chance had time to look at the .xml outputs of the pytest tests and how they would fit into the CI? With your approval we could merge this PR.

Hi @mpfirrmann Right now my local xunit-viewer package is broken and I could not verify the html generation with the xml added here. However the junit-viewer package itself has been deprecated and xunit-viewer will be used for the junit generated xml files as well as per the documentation https://www.npmjs.com/package/junit-viewer and https://github.com/lukejpreston/xunit-viewer. So, from my end, I think we can go ahead and merge the PR.

However, for double check, It would be helpful if @ardzoht or @tmdzk can suggest any method with which we can generate CI reports over lte integ test for this PR to see if xunit-viewer is generating the html reports properly for junit generated xml files.

[ardzoht]

@VinashakAnkitAman as discussed offline, let's merge the PR and monitor closely the HTML results, for validating the updates. We can revert in case there's any issue introduced by the update.

[VinashakAnkitAman]

LGTM. Will be verified in CI for HTML report generation after merge.