[Module][ 1 ] [zos_archive] Support various archive formats that can accept data sets as a src to support z/OS

[ddimatos]

New module description

Module that provides users the abilities too:

• Create an archive a src on the remote host where src can be:
  • Sequential Data set
  • PDS
  • PDS/E
  • VSAM [Stretch Goal]
• Support compression formats and options for:
  • gz
  • tar
  • zip
  • bz2
  • pax
  • terse (trs)
  • xmit (transmit)
• Support listing the contents of an archive (community module does not currently perform this operation)

Review:

• Review the community archive module
• For bz2 see bz2
• For gzip/gz see gzip
• For zip see zfile
• For tar, pax see USS commands tar and pax
• For AMATERSE see amaterse and options SPACK and PACK
• TSO TRANSMIT aka XMIT command refeference
• More on TRANSMIT
• IBM Doc on how to Terse a PDSE

Notes:

• This archive module spec is a work in progress.
• The idea of a sub-option for format is obviously up for discussion but the idea here is that even though this does not follow with the community direction how do we allow a user to select format and pass options when options are available. For example AMATERSE can take args SPACK and PACK, while if we really wanted to we could even support compression levels for gzip 1-9 , appears to be an option the API but this compression level 1-9 is not available on the community module.
• It is expected this module will overlap with the existing zos_backup_restore module, I suggest you review that module as well and consider any reasonable input for this module.
• Its been noted we likely will need to support
  • Large format (DSNTYPE=LARGE) data sets
  • When LRECL is less than 32K and LRECL may be more than 32K but less than 64K
  • Placement of data sets into cylinder-managed space above 65K CYL
  • These bullets are still needing investigation but some doc that may help is here we could also reach out to the L2 support teams who often use Amaterse when working with customers to see the importance on these options.
  • Example at the bottom of DSNTYPE=LARGE
• This module spans multiple iterations, its uncertain how many sprints it will need but current there are 4 designated;
  • [Module][ 2 ][zos_archive] Support various archive formats that can accept data sets as a src to support z/OS #475
  • [Module][ 3 ][zos_archive] Support various archive formats that can accept data sets as a src to support z/OS #476
  • [Module][ 4 ][zos_archive] Support various archive formats that can accept data sets as a src to support z/OS #477
• From what I could see no additional imports were needed for bz2, gzip and zip , please double check.

Parameters

	Parameter	Alias	Suboptions	Description
1	src			Remote absolute path, data set, glob, or list of paths or globs for the file or files to compress or archive.
2	dest			The file name of the destination archive. The parent directory must exists on the remote host. The data set name of the destination to archive, can not be a PDS/E If the destination archive already exists, it will be truncated and overwritten.
3	exclude_path			Remote absolute path, glob, or list of paths or globs for the file or files to exclude from path list and glob expansion.
4	force_archive			I am unsure if we want this, likely to get confused with force; Allows you to force the module to treat this as an archive even if only a single file is specified.
5	format		compression, options	The type of compression to use, choices would be (1) gz (2) tar (3) zip (4) bz2 (5)pax (6) terse (trs) (7) XMIT (see notes in module spec). Options should be a list or choice, depends on how many we support e.g.XMIT could support SPACK and PACK
6	group			Name of the group that should own the filesystem object, defaults to the current users group, applies only to USS files
7	mode			The permissions the resulting filesystem object should have. .....
8	owner			Name of the user that should own the filesystem object, defaults to the current user, applies only to USS files
9	remove			Delete src files after archival when replace=True
10	force			Replace existing archive dest
11	list			List the names of the archive contents
12	hlq			High Level Qualifier used for data set

Return Values

Key	Description
archived	Any files that were compressed or added to the archive. Returned: success
arcroot	The archive root. Returned: always
dest_state	The state of the dest file. -absent when the file does not exist.-archive when the file is an archive. -compress when the file is compressed, but not an archive. -incomplete when the file is an archive, but some files under path were not found. Returned: success
expanded_exclude_paths	The list of matching exclude paths from the exclude_path argument. Returned: always
expanded_paths	The list of matching paths from paths argument. Returned: always
missing	Any files that were missing from the source.Returned: success
state	The state of the input path.Returned: always
names or dest	Destination name

See also common values returned, thinking its better to define the ones that we support rather than let users figure it out.

Special thanks, this table was generated with tableconvert.