Problem: A release process is needed for AMAUAT tests

[ross-spencer]

Please describe the problem you'd like to be solved.

I would like to make it easier to run the AMAUAT tests on different versions of Archivematica. Currently tests take a -D am_version=1.8 parameter, but historically, it is difficult to simply pass a version flag of the tests and expect them to pass where it relies on historical knowledge of releases, e.g. 1.10 will make breaking changes between PREMIS 2 and PREMIS 3 support.

Coding for this type of change in AMAUAT tests is also a challenge. It relies on being able to test simultaneously on n and n-1 versions of Archivematica and not all developers will have this set up. (if any?). It generates a lot of cognitive load.

Describe the solution you'd like to see implemented.

Create a release procedure for AMAUAT tests where:

• A release cycle sees Archivematica qa/1.x released,
• At that moment, a AMAUAT suite is released matching the AM version.

For future testing, e.g. a stable/1.x.x release, the release version of AMAUAT will be downloaded, and the Archivematica will be tested against that. If changes are required for stable/1.x.x then they will be incorporated into a new release branch of the AMAUAT tests.

We shouldn't have to worry about maintaining decision points in AMAUAT code where versions are used to direct logic.

Describe alternatives you've considered.

We can use the version flag discussed, but I'm not sure we have this working practically. Even with a version flag, then it might also be beneficial to:

• Release AMAUAT per version of Archivematica.
• Setup Jenkins to always have an n-1 backwards facing test that enables minor releases to be done more effectively.

Additional context

• AMAUAT refers to our automated test suite found here.
• Related to Problem: acceptance tests rely on lexical ordering to compare versions #288

---

For Artefactual use:
Please make sure these steps are taken before moving this issue from Review to Verified in Waffle:

• All PRs related to this issue are properly linked 👍
• All PRs related to this issue have been merged 👍
• Test plan for this issue has been implemented and passed 👍
• Documentation regarding this issue has been written and it has been added to the release notes, if needed 👍

[sallain]

Consensus in Artefactual meeting was to proceed as Ross has suggested above, adding a release tag to the commit that was used to test the release. We will not try to maintain backwards compatibility. @ross-spencer can you add tagging procedure to release checklist?

[ross-spencer]

Related to #288

[ross-spencer]

Branch and release tag created for 1.10 and 0.15 tests here: https://github.com/artefactual-labs/archivematica-acceptance-tests/releases/tag/v1.10.0