Enable pipeline to apply PTFs

[John-A-Davies]

Enable the Jenkins pipeline to perform an install-test run that is based on an SMP/E install + PTF.

1. perform SMP/E install of base FMID
2. perform SMP/E install of PTF

[John-A-Davies]

In install-test build we already have the parameter
ZOWE_ARTIFACTORY_PATTERN
which specifies the FMID in file zowe-smpe.zip. To this we will add
SYSMOD_ARTIFACTORY_PATTERN
which specifies the SYSMOD in file AZWE001.TMP0001.zip
If this field is blank, there is no SYSMOD. Otherwise it names a zip file containing

ZOWE.AZWE001.TMP0001
ZOWE.AZWE001.TMP0002
ZOWE.AZWE001.TMP0001.readme.htm

When a SYSMOD is present, the pipeline will install the FMID and then install the SYSMOD before continuing with the smoke tests.

A SYSMOD can be a PTF, an APAR, or a USERMOD.

After the FMID is released as GA, it will never change, so
ZOWE_ARTIFACTORY_PATTERN
can then be greyed out and a fixed file will be used.

[John-A-Davies]

Comments from @OnnoVdT , edited by @John-A-Davies .

Just like the FMID JCLs, the PTF JCLs are converted to VTL format and the sample JCL you see show up is generated during the build (see .pax/post-packaging.sh)
The plan Vitek & co have is that like for FMID install, there will be a workflow for service install. But unlike FMID install, the service workflow does not yet exist.

For the AZWE001.zip and AZWE001.TMP001.zip not showing up in artifactory, something was changed in the process that happens after the SMP code is done, as smpe.sh will always produce both.

for the sample JCL in the HTML, I always assumed you would drive the sample JCL we ship to test PTF install, as this way you test both PTF and the JCL.
and now with vtl in the picture, we should be able to make that easier.
If we include the vtl source for the PTF JCL and vtl-cli.jar in the build, then all we need is an XML file with the variables and feed it to the jar, which will spit out JCL customized for your test.
I know they started to do the same for ZWESECUR (with workflow).

Or instead of shipping vtl-cli and the individual PTF vtl files, I could have the SMPE build generate customized PTF JCL in the logs directory, which ends up in artifactory as a pax file. That way you could pull the pax, extract it and run the JCL, We could do the same for the FMID JCL for that matter.

Comments from @John-A-Davies
In order to make FMID and PTF different, we'd need to have a fake FMID containing code from the previous build, and the PTF, containing the code from the current build. This would be an interim-only scenario. That way we can check that the PTF actually changes something.

Comments from @OnnoVdT
The simplest is to keep using the FMID that comes with the PTF build, but you are right, their code is identical.
What you could do is alter something after FMID install, for example overwrite SZWESAMP(ZWE1SMPE), you won't need that one after FMID install anyway.
Then install PTF, and verify content of SZWESAMP(ZWE1SMPE), if it's no longer your injected data the PTF did its job.
the workflow files for FMID install are in F4.

You could add a file to usr/lpp/zowe/bin, the whole directory is removed and recreated during PTF install, so if your new file is gone, the PTF worked.

[OnnoVdT]

regarding
In install-test build we already have the parameter
ZOWE_ARTIFACTORY_PATTERN
which specifies the FMID in file zowe-smpe.zip
once shopz goes live, so everything past 1.9, there will no longer be an FMID in zowe-smpe.zip, but a PTF. You have to point your input variable to AZWE001.zip, which holds the FMID pax adn readme

[John-A-Davies]

@OnnoVdT Packaging #446 has these artefacts

• AZWE001.TMP0001.zip
• AZWE001.zip
• pd.htm
• smpe-build-logs.pax.Z
• smpe-promote.tar
• zowe-smpe.tar
• zowe.pax

And I understand from @NayerNajafi that AZWE001.zip and zowe-smpe.tar are basically the same - for now.

But #503 has only

• pd.htm
• smpe-build-logs.pax.Z
• smpe-promote.tar
• zowe-smpe.zip
• zowe.pax

So what's the plan for these files going forward? Until the FMID is GA, I need both, to test that I can apply the PTF.

It's essential that the pipeline be able to apply the FMID and then the PTF.

[John-A-Davies]

I suggest that I create JCL for PTF: ALLOC, ACCEPT, RECEIVE, APPLY, REJECT, RESTORE at least in the install-test repo, and perhaps better in the install-packaging repo (to allow customers to use them, not just Jenkins). These will be based on what @OnnoVdT put in the readme.htm file, which is like what's in relfile F2, but more evolved.

[John-A-Davies]

I've created a shell script, install-SMPE-SYSMOD.sh to perform SMP/E install of a PTF. It can also un-install the PTF.

[John-A-Davies]

Opened #165 to address my Feb 12 comment above.

[John-A-Davies]

Test plan.

Action         pre-GA     post-GA
-------    
Install FMID    y          y
Run config      y          y
Test behaviour  smoke      smoke
Save ToC        y          y
-
Install PTF     =FMID      new
Test behaviour  same       new
Check ToC       same       new
Check config    same       same
-
Remove PTF      =FMID      =FMID
Test behaviour  same       same
Check ToC       same       same
Check config    same       same

Where

• same means same as for FMID
• new means for PTF
• ToC is table of contents of target USS directory and MVS artefacts.