Update colleciton with functional test cases #9
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ddimatos
changed the title
Contributor
|
I updated the test cases to be compatible with the public repo name |
blakeinate
approved these changes
Feb 19, 2020
Contributor
blakeinate
left a comment
blakeinate
left a comment
There was a problem hiding this comment.
Unit tests required updates for module imports to work with public repo name, updates have been made and tests are passing. Approved.
ddimatos
added a commit
that referenced
this pull request
Feb 20, 2020
* Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting
ddimatos
added a commit
that referenced
this pull request
Feb 20, 2020
* Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting
ddimatos
added a commit
that referenced
this pull request
Apr 23, 2020
* initial commit * begin updating examples * zos_fetch module * return rc, stdout, stderr as separate return parameters on failure * untrack non-fetch related files * add rest of the files * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * added unit and functional tests * check file existence when LISTDS returns invalid dataset name * added 'wait_s' parameter * clarified description of encoding parameter * make use_qaulifer false by default * remove is_uss and os_vsam parameters * remove data_set directory * remove is_uss parameter from examples * message return block * Fix YAML formatting * return code formatting * remove fail_on_missing from action plugin * fail if USS file does not exist * data sets are assumed to be cataloged * instead of entire dict, pass individual values * redundant code when fetching data set member * rename transfer_from_uss to fetch_non_partitioned_data_set * remove test cases for uncataloged datasets * import naming conflict * add __future__ imports * add __future__ imports * assuming all data sets are cataloged * assuming all data sets are cataloged * fix formatting issues * fix formatting issues * remove blank lines with whitespaces * change example member name data to DATA for accuracy. * continuation line unindented for visual indent * conform to PEP8 standard * avoid automatic field numbering for format * specify python interpreter * extract only data set name if member present * Remove newline character from error message * Update RETURN block with sample message * Remove blank line from return block * Update version_added to correspond to Ansible version * Check if temp datasets are being created successfully * Datasets.read() returns None for empty data sets * Use BetterArgParser to parse module args * Added parameter types * Remove trailing whitespaces * Avoid passing extra args from action plugin * Remove unused imports * Ignore lack of GPLv3 license complaint * catch ValueError thrown ny BetterArgParser * Blank line contains whitespace * Reduce line lengths to 80 * Add white space around string addition * Checking volume type no longer required * Check if src or dest is provided first * Fix indentation * Indent multiline string for better readability * Use DataSetUtils to get data set type * Use DataSetUtils to determine data set existence * Remove unit tests * Fix linting * Add test case for empty data sets * default params for _fail_json * Test case for missing source * Test case for insufficient write permission * Test case for insufficient desgt write permission * Catch exceptions while executing module * Use updated DataSetutils constructor * Updated functional tests * Fix linting errors * Use ansible fetch module to fetch USS files * Checking for USS type is unnecessary * Use sha256 instead of sha1 for checksum validation * Remove message return block * Missing uss file test case * Minor cleanup * Add stdout_lines and stderr_lines to module response * Update action plugin with stderr_lines and stdout_lines * YAML syntax issue * Corrected spelling errors * Inaccurate description of flat * No need to wrap error messages returned by DataSetUtils * Update doc regarding HFS and ZFS support * Handle encoding for USS files * Add encoding to doc * Calculate temp file size for VSAM storage * Preserve encoding using SFTP * Resolve PEP8 issues * Check for data set member existence * raw binary not supported in PY2 * Fix linting issues * Change doc to reflect USS absolute path * Prepend dataset hlq before parameter verification * src not declared before being referenced * Add test case using use_qualifier * Updated doc link for fetch L(..) and suboptions * Use tempfile.mkstemp() for temp data storage * Remove default for encoding * default encoding values are None * Create destination dirs when flat is false * Add test case for flat hostname * Fix spelling * exist_ok is not valid keyword for PY2 * Detect read-only file system * Handle sftp errors * Capture module_stderr in case of module failure * Ensure dest is a directory for PDS transfer * Ensure dest is a directory except when fetching a member * Fix data set member destination path * Default values for encoding is not needed * Remove unused imports * Update 'msg' description * Clean up temp files in case of module failure * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com>
ddimatos
added a commit
that referenced
this pull request
May 8, 2020
* Initial collection commit for v0.0.1 (#1) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * added arg parser to module_utils (#3) * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * Updated job output for improved DD handling (#14) Fixes the job summary output being returned when return_output=false. * Security module updates (#15) * patched security holes * updated test cases * removed unnecessary import * updated execution bits * Update modules and documentation corrections (#17) * update documentation, remove unused imports * updated requirements * updated module imports, formatting, and spelling * doc update and formatting update * z/OS Ping Module Feature Delivered (#18) * added zos_ping, test, and test fixture update * Description completeness * Grammar completeness * Pulled repetitive code into functions * updated module docs * Minor spelling corrections * Repo updates to satisfy ansible-test sanity testing requirements (#19) * update formatting * formatting changes to appease ansible-test * update tests * fix string format use and quotes * update README * remove trailing whitespace * update group_vars trailing spaces * remove shebang in attempt to fix encoding issue * add back shebang lines * Added security updates and test cases (#20) * add new types for parser, update arg handling zos_job_submit * updated module argument parsing and security tests * uncommented security tests * updated README to reflect use of MIT license * update hash function * test and formatting updates * updated documentation to pass ansible-test * switch name back to normal.py, add appropriate ignores * update ignore files * added ignore for GPL * updated README, .gitignore and removed MIT code * remove broken tests * increase robustness of zos_job_query * make job query even more robust * Webdocs content for IBM z/OS Core Collection (#21) * Initial commit for reStructuredText content * Updates and clean up to the template * Updated grammar * Grammar edit * Grammar fixes * Text updates * Updates to restructured text, fixed link, renamed collection * Fix ibm_zos_core bold * Fixes ansible-test lint: docs/source/conf.py:51:161: E501: line too long (178 > 160 characters) * ansible doc changes to modules for better formatting * remove original_message return parameter (#23) * remove original_message return parameter * Fix reported linting errors from PR * Fix reported linting errors from PR * Fix reported linting errors from PR for zos_job_query.py * Module document updates, Makefile changes and jinja support for defau… (#24) * Module document updates, Makefile changes and jinja support for default:int * Doc updates per linting rules * Update mmodule doc and remove md files (#25) * Removed all MD files from docs directory * Remove README.md , its replaced by README.rst * Fix bolding for ibm_zos_core * Fix highlight for ibm_zos_core * Updated index.rst to correspond to the README.md updates * Updated README.rst to point to webdocs * Updated README.rst webdocs URL * Update the modules to their ansible-doc * Documentation updates (#26) * Updated galaxy.yml to reflect latest version 1.0 * Documentation copyright updates and formatting * Documentation udpates * Update z/OS data set module and add catalog functionality (#27) * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * catalog for VSAM working, bugs fixed, tests added * satisfy ansible-test requirements, minor arg validation updates * removed unit tests * raise min . ansible requirement due to security vulnerability * added documentation to VTOC * Update product name in webdocs (#28) * Update product name in webdocs * Fix trailing newline * Document updates include some template bugs * Integraded document review comments (#29) * Integrated document review comments * Incorrect break for bulleted line * Bug in Jinja template where property would tab to much (#31) * Updates to documentation and galaxy configuration (#32) * Fix to README.rst, duplicate wording (#34) * Remove duplicate action plugins link (#35) * testing readme rendering (#36) Ported the README.rst file to README.md to meet collection requirements. * Corrected README to point to MD, and long tag removed (#39) * Add two additional arg types to BetterArgHandler (#44) * Add two additional arg types * Remove os from os.path.isabs Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Data Set Utility for Common Data Set Related Tasks (#48) * DataSetUtils initial commit * LISTDS and LISTCAT output processing * Raise exception when data set is open for write * Fix LISTDS regex * Fix indentation * Add USS path support * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/832/fix non vsam non sms existence checking zos data set (#49) * updated catalog handling for non-vsam, non-sms * updated some method comment structures * bugfix/822/determine-data-set-type (#50 * Use mvscmd CLI instead of Python API * Fix linting errors * Remove unused variable Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Extend dynamic function support to the better arg parser (#51) * Dynamic argument support for dynamic functions. * improve default handling for reusability of BetterArgParser object. * add test cases for dynamic argument support * removed print statements from test cases * Updates to data-set-utils exceptions (#52) * Avoid raising exception for uncatalogued data sets * verify listcat output for uncatalogued datasets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Removed return block from documentation (#53) Removed message and changed, empty return block now * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * Fixed PATH (#55) * Update/973/add new supported data set characters betterargparser (#56) * updated valid data set types builtin type name fix * added test cases for data set types * Bugfix when checking if data set exists (#57) * Return False instead of None when data set doesn't exist * Test for membership should ne 'not in' Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Committing module zos_operator_action_query (#43) * Reviewd changes for feature zos_operator_action_query.py * Fix linting errors reported in PR * Linting corrections * fix the syntax error * fix lint errors * fix lint error * fix lint error * fix lint error * fix function test * Added zos_operator_action_query to the sanity exemption list for lack of GPLV3 Lic * Update author tag * Update the collection reference * add function test(need revise later) & doc update * updated regex and value capturing * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * updated docs, made environment agnostic test cases * added support for "*" and fixed splat handling * updated test cases * updated regex matching method, updates tests * Update short description Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Check for data set member existence (#58) * Check for data set member existence * Resolve linting issue Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix for job name capture group (#60) * Better ZOAU import missing messages (#61) * Better ZOAU import missing messages * added missing license and imports * Changed behavior on failure for zosoperator_action_query (#63) * changed behavior on failure * update tests for new expected output * Bugfix 1661 & 1704 fix query of job for different user zos_job_submit and zos_job_action_query (#64) * updates to fix job submission as other user * updated and optimized job utility * removed usage of ZOAU and updated jobs utility usage * update job_output return structure * improved error messages * add job log cleanup for test cases * Initial module zos_tso_command (#46) * Linting corrections * Update doc so html will generate * Lintig updates * Lintig updates v2 * Lintig updates v3 * Lintig updates v4 * Lintig updates v5 * Lintig updates v6 * Lintig updates v7 * Lintig updates v8 * Delete test_zos_tso_command_unit.py Remove the unit test as it only mocks things. * Documenation updates * Update errro message * zos-tso-command update * linting update * testcase linting updates * linting update * linting update * linting update * linting update * linting updates * linting update * new authorized long command handling method * linting update * test cases update * linting update * linting update * linting update * tso-command new version * tso-command linting update * linting update * linting update * linting update' * linting removal * lint removal * linting testcases Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> * Update module_utils with an encode utility (#59) * initial commmit for encode_utils * support for string encoding conversion * Support MVS data set conversion and string conversion modification * Recursively copy to PDS fix * Linting correction * Linting updates * Linting updates 01 * fix ENOENT problem after shutil.move * updates with zos_encode * Merged branch and commit for zos_encode (#65) * Merged branch and commit for zos_encode * Lint error , trailing white space * z/OS Fetch Module (#7) * initial commit * begin updating examples * zos_fetch module * return rc, stdout, stderr as separate return parameters on failure * untrack non-fetch related files * add rest of the files * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * added unit and functional tests * check file existence when LISTDS returns invalid dataset name * added 'wait_s' parameter * clarified description of encoding parameter * make use_qaulifer false by default * remove is_uss and os_vsam parameters * remove data_set directory * remove is_uss parameter from examples * message return block * Fix YAML formatting * return code formatting * remove fail_on_missing from action plugin * fail if USS file does not exist * data sets are assumed to be cataloged * instead of entire dict, pass individual values * redundant code when fetching data set member * rename transfer_from_uss to fetch_non_partitioned_data_set * remove test cases for uncataloged datasets * import naming conflict * add __future__ imports * add __future__ imports * assuming all data sets are cataloged * assuming all data sets are cataloged * fix formatting issues * fix formatting issues * remove blank lines with whitespaces * change example member name data to DATA for accuracy. * continuation line unindented for visual indent * conform to PEP8 standard * avoid automatic field numbering for format * specify python interpreter * extract only data set name if member present * Remove newline character from error message * Update RETURN block with sample message * Remove blank line from return block * Update version_added to correspond to Ansible version * Check if temp datasets are being created successfully * Datasets.read() returns None for empty data sets * Use BetterArgParser to parse module args * Added parameter types * Remove trailing whitespaces * Avoid passing extra args from action plugin * Remove unused imports * Ignore lack of GPLv3 license complaint * catch ValueError thrown ny BetterArgParser * Blank line contains whitespace * Reduce line lengths to 80 * Add white space around string addition * Checking volume type no longer required * Check if src or dest is provided first * Fix indentation * Indent multiline string for better readability * Use DataSetUtils to get data set type * Use DataSetUtils to determine data set existence * Remove unit tests * Fix linting * Add test case for empty data sets * default params for _fail_json * Test case for missing source * Test case for insufficient write permission * Test case for insufficient desgt write permission * Catch exceptions while executing module * Use updated DataSetutils constructor * Updated functional tests * Fix linting errors * Use ansible fetch module to fetch USS files * Checking for USS type is unnecessary * Use sha256 instead of sha1 for checksum validation * Remove message return block * Missing uss file test case * Minor cleanup * Add stdout_lines and stderr_lines to module response * Update action plugin with stderr_lines and stdout_lines * YAML syntax issue * Corrected spelling errors * Inaccurate description of flat * No need to wrap error messages returned by DataSetUtils * Update doc regarding HFS and ZFS support * Handle encoding for USS files * Add encoding to doc * Calculate temp file size for VSAM storage * Preserve encoding using SFTP * Resolve PEP8 issues * Check for data set member existence * raw binary not supported in PY2 * Fix linting issues * Change doc to reflect USS absolute path * Prepend dataset hlq before parameter verification * src not declared before being referenced * Add test case using use_qualifier * Updated doc link for fetch L(..) and suboptions * Use tempfile.mkstemp() for temp data storage * Remove default for encoding * default encoding values are None * Create destination dirs when flat is false * Add test case for flat hostname * Fix spelling * exist_ok is not valid keyword for PY2 * Detect read-only file system * Handle sftp errors * Capture module_stderr in case of module failure * Ensure dest is a directory for PDS transfer * Ensure dest is a directory except when fetching a member * Fix data set member destination path * Default values for encoding is not needed * Remove unused imports * Update 'msg' description * Clean up temp files in case of module failure * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * enhanced zoau import error message (#66) * enhanced zoau import error message * satisfy pep8 witespace requirement * Added new playbook for latest module collection and a filter * fixed linting issues, added license * updated spacing to conform to formatting standards * updated shell to command where possible * Use encode util to convert encoding (#68) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update RST and MD doc files to reflect next release * Update group vars to reflect zoau 1.0.2 new env vars * Update encode short description to fit within the limits of doc length * Update makefile for more generic use by others * Update to fix RST lint error , short line * Rename sub-dir from plugin to filter, doc error * doc tree correction * Update release notes to correspond to beta release tag * Update sample playbooks with doc, configuration and new variables * Upate docs to highlight users to review sample playbooks and JCL before running * Update sample JCL to run on my systems with default values and notes * Add supporting doc for behavioral inventory parameters and ansible.cfg * Fix the md style URL in the index RST file * Update short descriptions for modules * Fix README.md link to IMS collection * Fix README.md link to collection artifacts * Linting updates * Add an example that requires a command to be escaped (#71) * refine backup_file & put it into encode_utils (#70) * refine backup_file & put it into encode_utils * correction for sanity test * refactor for security and usability, fix test case * move copy logic to utility * refactor zos_encode, improve backup functionality Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Update galaxy.yml to reflect the latest version and authors * Fix import name change resulting from refactor * updated usage of encode_utils to encode * Documentation updates, corrections and alterations (#75) * Documentation updates, corrections and alterations * ID review-round 2 * Update the user description in the doc Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Content/81/zos tso command docreview (#73) * ID review of zos-tso-commands * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * Fix indentation Co-authored-by: ddimatos <dimatos@gmail.com> * Fix extra space in doc * fixed zos_ping action plugin issues (#77) * Add out-of-box awx support (#80) * Added encoding supplemental doc (#78) * Added encoding supplemental doc * Removed references to modules not available and causing doc warnings * Fix Inline literal start-string without end-string. * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Add binary support copying across USS and MVS * Copy USS file in binary mode * fixed bug and cleaned up code (#82) * fixed bug and cleaned up code * fixed linting issues * Avoid specifying -B and -F for cp * Copy MVS source to MVS dest * Documentation web content updates (#84) * First round of review of static docs * docs/source/installation.rst * Update block to rendor with syntax highlights Code block was using 'yaml' when it was 'sh' , renders better if we have the correct code block type. * Incorporated Demetri's review comments Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Replace only the first occurance of 'rec' * Minor updates in install,quickstart, and req pages (#86) * Updated make file with comments and recover from failure * Update Backup util (mvs_file_backup) (#85) * Alternative mvs_file_backup * fix backup logic, add encode backup tests * add mvs backup tests, util fixes * fix test for new logic path Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Udpate the restructured text to reflect version release (#89) * minor fixes and additional tests for zos_encode (#88) * Merge oversite;normal.py had been renamed zos_ping.py * removed Jobs.submit usage due to invalid response (#93) Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Jake Jackson <jljacks93@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> Co-authored-by: zlbjlu <16587467+yourfuwa2015@users.noreply.github.com> Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com>
ddimatos
added a commit
that referenced
this pull request
Jun 25, 2020
* Initial collection commit for v0.0.1 (#1) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * added arg parser to module_utils (#3) * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * Updated job output for improved DD handling (#14) Fixes the job summary output being returned when return_output=false. * Security module updates (#15) * patched security holes * updated test cases * removed unnecessary import * updated execution bits * Update modules and documentation corrections (#17) * update documentation, remove unused imports * updated requirements * updated module imports, formatting, and spelling * doc update and formatting update * z/OS Ping Module Feature Delivered (#18) * added zos_ping, test, and test fixture update * Description completeness * Grammar completeness * Pulled repetitive code into functions * updated module docs * Minor spelling corrections * Repo updates to satisfy ansible-test sanity testing requirements (#19) * update formatting * formatting changes to appease ansible-test * update tests * fix string format use and quotes * update README * remove trailing whitespace * update group_vars trailing spaces * remove shebang in attempt to fix encoding issue * add back shebang lines * Added security updates and test cases (#20) * add new types for parser, update arg handling zos_job_submit * updated module argument parsing and security tests * uncommented security tests * updated README to reflect use of MIT license * update hash function * test and formatting updates * updated documentation to pass ansible-test * switch name back to normal.py, add appropriate ignores * update ignore files * added ignore for GPL * updated README, .gitignore and removed MIT code * remove broken tests * increase robustness of zos_job_query * make job query even more robust * Webdocs content for IBM z/OS Core Collection (#21) * Initial commit for reStructuredText content * Updates and clean up to the template * Updated grammar * Grammar edit * Grammar fixes * Text updates * Updates to restructured text, fixed link, renamed collection * Fix ibm_zos_core bold * Fixes ansible-test lint: docs/source/conf.py:51:161: E501: line too long (178 > 160 characters) * ansible doc changes to modules for better formatting * remove original_message return parameter (#23) * remove original_message return parameter * Fix reported linting errors from PR * Fix reported linting errors from PR * Fix reported linting errors from PR for zos_job_query.py * Module document updates, Makefile changes and jinja support for defau… (#24) * Module document updates, Makefile changes and jinja support for default:int * Doc updates per linting rules * Update mmodule doc and remove md files (#25) * Removed all MD files from docs directory * Remove README.md , its replaced by README.rst * Fix bolding for ibm_zos_core * Fix highlight for ibm_zos_core * Updated index.rst to correspond to the README.md updates * Updated README.rst to point to webdocs * Updated README.rst webdocs URL * Update the modules to their ansible-doc * Documentation updates (#26) * Updated galaxy.yml to reflect latest version 1.0 * Documentation copyright updates and formatting * Documentation udpates * Update z/OS data set module and add catalog functionality (#27) * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * catalog for VSAM working, bugs fixed, tests added * satisfy ansible-test requirements, minor arg validation updates * removed unit tests * raise min . ansible requirement due to security vulnerability * added documentation to VTOC * Update product name in webdocs (#28) * Update product name in webdocs * Fix trailing newline * Document updates include some template bugs * Integraded document review comments (#29) * Integrated document review comments * Incorrect break for bulleted line * Bug in Jinja template where property would tab to much (#31) * Updates to documentation and galaxy configuration (#32) * Fix to README.rst, duplicate wording (#34) * Remove duplicate action plugins link (#35) * testing readme rendering (#36) Ported the README.rst file to README.md to meet collection requirements. * Corrected README to point to MD, and long tag removed (#39) * Add two additional arg types to BetterArgHandler (#44) * Add two additional arg types * Remove os from os.path.isabs Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Data Set Utility for Common Data Set Related Tasks (#48) * DataSetUtils initial commit * LISTDS and LISTCAT output processing * Raise exception when data set is open for write * Fix LISTDS regex * Fix indentation * Add USS path support * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/832/fix non vsam non sms existence checking zos data set (#49) * updated catalog handling for non-vsam, non-sms * updated some method comment structures * bugfix/822/determine-data-set-type (#50) * Use mvscmd CLI instead of Python API * Fix linting errors * Remove unused variable Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Extend dynamic function support to the better arg parser (#51) * Dynamic argument support for dynamic functions. * improve default handling for reusability of BetterArgParser object. * add test cases for dynamic argument support * removed print statements from test cases * Updates to data-set-utils exceptions (#52) * Avoid raising exception for uncatalogued data sets * verify listcat output for uncatalogued datasets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Removed return block from documentation (#53) Removed message and changed, empty return block now * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * Fixed PATH (#55) * Update/973/add new supported data set characters betterargparser (#56) * updated valid data set types builtin type name fix * added test cases for data set types * Bugfix when checking if data set exists (#57) * Return False instead of None when data set doesn't exist * Test for membership should ne 'not in' Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Committing module zos_operator_action_query (#43) * Reviewd changes for feature zos_operator_action_query.py * Fix linting errors reported in PR * Linting corrections * fix the syntax error * fix lint errors * fix lint error * fix lint error * fix lint error * fix function test * Added zos_operator_action_query to the sanity exemption list for lack of GPLV3 Lic * Update author tag * Update the collection reference * add function test(need revise later) & doc update * updated regex and value capturing * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * updated docs, made environment agnostic test cases * added support for "*" and fixed splat handling * updated test cases * updated regex matching method, updates tests * Update short description Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Check for data set member existence (#58) * Check for data set member existence * Resolve linting issue Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix for job name capture group (#60) * Better ZOAU import missing messages (#61) * Better ZOAU import missing messages * added missing license and imports * Changed behavior on failure for zosoperator_action_query (#63) * changed behavior on failure * update tests for new expected output * Bugfix 1661 & 1704 fix query of job for different user zos_job_submit and zos_job_action_query (#64) * updates to fix job submission as other user * updated and optimized job utility * removed usage of ZOAU and updated jobs utility usage * update job_output return structure * improved error messages * add job log cleanup for test cases * Initial module zos_tso_command (#46) * Linting corrections * Update doc so html will generate * Lintig updates * Lintig updates v2 * Lintig updates v3 * Lintig updates v4 * Lintig updates v5 * Lintig updates v6 * Lintig updates v7 * Lintig updates v8 * Delete test_zos_tso_command_unit.py Remove the unit test as it only mocks things. * Documenation updates * Update errro message * zos-tso-command update * linting update * testcase linting updates * linting update * linting update * linting update * linting update * linting updates * linting update * new authorized long command handling method * linting update * test cases update * linting update * linting update * linting update * tso-command new version * tso-command linting update * linting update * linting update * linting update' * linting removal * lint removal * linting testcases Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> * Update module_utils with an encode utility (#59) * initial commmit for encode_utils * support for string encoding conversion * Support MVS data set conversion and string conversion modification * Recursively copy to PDS fix * Linting correction * Linting updates * Linting updates 01 * fix ENOENT problem after shutil.move * updates with zos_encode * Merged branch and commit for zos_encode (#65) * Merged branch and commit for zos_encode * Lint error , trailing white space * z/OS Fetch Module (#7) * initial commit * begin updating examples * zos_fetch module * return rc, stdout, stderr as separate return parameters on failure * untrack non-fetch related files * add rest of the files * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * added unit and functional tests * check file existence when LISTDS returns invalid dataset name * added 'wait_s' parameter * clarified description of encoding parameter * make use_qaulifer false by default * remove is_uss and os_vsam parameters * remove data_set directory * remove is_uss parameter from examples * message return block * Fix YAML formatting * return code formatting * remove fail_on_missing from action plugin * fail if USS file does not exist * data sets are assumed to be cataloged * instead of entire dict, pass individual values * redundant code when fetching data set member * rename transfer_from_uss to fetch_non_partitioned_data_set * remove test cases for uncataloged datasets * import naming conflict * add __future__ imports * add __future__ imports * assuming all data sets are cataloged * assuming all data sets are cataloged * fix formatting issues * fix formatting issues * remove blank lines with whitespaces * change example member name data to DATA for accuracy. * continuation line unindented for visual indent * conform to PEP8 standard * avoid automatic field numbering for format * specify python interpreter * extract only data set name if member present * Remove newline character from error message * Update RETURN block with sample message * Remove blank line from return block * Update version_added to correspond to Ansible version * Check if temp datasets are being created successfully * Datasets.read() returns None for empty data sets * Use BetterArgParser to parse module args * Added parameter types * Remove trailing whitespaces * Avoid passing extra args from action plugin * Remove unused imports * Ignore lack of GPLv3 license complaint * catch ValueError thrown ny BetterArgParser * Blank line contains whitespace * Reduce line lengths to 80 * Add white space around string addition * Checking volume type no longer required * Check if src or dest is provided first * Fix indentation * Indent multiline string for better readability * Use DataSetUtils to get data set type * Use DataSetUtils to determine data set existence * Remove unit tests * Fix linting * Add test case for empty data sets * default params for _fail_json * Test case for missing source * Test case for insufficient write permission * Test case for insufficient desgt write permission * Catch exceptions while executing module * Use updated DataSetutils constructor * Updated functional tests * Fix linting errors * Use ansible fetch module to fetch USS files * Checking for USS type is unnecessary * Use sha256 instead of sha1 for checksum validation * Remove message return block * Missing uss file test case * Minor cleanup * Add stdout_lines and stderr_lines to module response * Update action plugin with stderr_lines and stdout_lines * YAML syntax issue * Corrected spelling errors * Inaccurate description of flat * No need to wrap error messages returned by DataSetUtils * Update doc regarding HFS and ZFS support * Handle encoding for USS files * Add encoding to doc * Calculate temp file size for VSAM storage * Preserve encoding using SFTP * Resolve PEP8 issues * Check for data set member existence * raw binary not supported in PY2 * Fix linting issues * Change doc to reflect USS absolute path * Prepend dataset hlq before parameter verification * src not declared before being referenced * Add test case using use_qualifier * Updated doc link for fetch L(..) and suboptions * Use tempfile.mkstemp() for temp data storage * Remove default for encoding * default encoding values are None * Create destination dirs when flat is false * Add test case for flat hostname * Fix spelling * exist_ok is not valid keyword for PY2 * Detect read-only file system * Handle sftp errors * Capture module_stderr in case of module failure * Ensure dest is a directory for PDS transfer * Ensure dest is a directory except when fetching a member * Fix data set member destination path * Default values for encoding is not needed * Remove unused imports * Update 'msg' description * Clean up temp files in case of module failure * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * enhanced zoau import error message (#66) * enhanced zoau import error message * satisfy pep8 witespace requirement * Added new playbook for latest module collection and a filter * fixed linting issues, added license * updated spacing to conform to formatting standards * updated shell to command where possible * Use encode util to convert encoding (#68) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update RST and MD doc files to reflect next release * Update group vars to reflect zoau 1.0.2 new env vars * Update encode short description to fit within the limits of doc length * Update makefile for more generic use by others * Update to fix RST lint error , short line * Rename sub-dir from plugin to filter, doc error * doc tree correction * Update release notes to correspond to beta release tag * Update sample playbooks with doc, configuration and new variables * Upate docs to highlight users to review sample playbooks and JCL before running * Update sample JCL to run on my systems with default values and notes * Add supporting doc for behavioral inventory parameters and ansible.cfg * Fix the md style URL in the index RST file * Update short descriptions for modules * Fix README.md link to IMS collection * Fix README.md link to collection artifacts * Linting updates * Add an example that requires a command to be escaped (#71) * refine backup_file & put it into encode_utils (#70) * refine backup_file & put it into encode_utils * correction for sanity test * refactor for security and usability, fix test case * move copy logic to utility * refactor zos_encode, improve backup functionality Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Update galaxy.yml to reflect the latest version and authors * Fix import name change resulting from refactor * updated usage of encode_utils to encode * Documentation updates, corrections and alterations (#75) * Documentation updates, corrections and alterations * ID review-round 2 * Update the user description in the doc Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Content/81/zos tso command docreview (#73) * ID review of zos-tso-commands * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * Fix indentation Co-authored-by: ddimatos <dimatos@gmail.com> * Fix extra space in doc * fixed zos_ping action plugin issues (#77) * Add out-of-box awx support (#80) * Added encoding supplemental doc (#78) * Added encoding supplemental doc * Removed references to modules not available and causing doc warnings * Fix Inline literal start-string without end-string. * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Add binary support copying across USS and MVS * Copy USS file in binary mode * fixed bug and cleaned up code (#82) * fixed bug and cleaned up code * fixed linting issues * Avoid specifying -B and -F for cp * Copy MVS source to MVS dest * Documentation web content updates (#84) * First round of review of static docs * docs/source/installation.rst * Update block to rendor with syntax highlights Code block was using 'yaml' when it was 'sh' , renders better if we have the correct code block type. * Incorporated Demetri's review comments Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Replace only the first occurance of 'rec' * Minor updates in install,quickstart, and req pages (#86) * Updated make file with comments and recover from failure * Update Backup util (mvs_file_backup) (#85) * Alternative mvs_file_backup * fix backup logic, add encode backup tests * add mvs backup tests, util fixes * fix test for new logic path Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Udpate the restructured text to reflect version release (#89) * minor fixes and additional tests for zos_encode (#88) * Merge oversite;normal.py had been renamed zos_ping.py * removed Jobs.submit usage due to invalid response (#93) Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Shinx and sphinx-versioning support and updates (#101) * removed Jobs.submit usage due to invalid response (#92) * Update to galaxy.yml to exclude files * Update makefile to support rexx written modules * Generate and push zos_ping rst doc * Corrected clean up of a directory in makefile * Fixes spacing in RETURN values and formatting (#97) * Updated Makefile to support sphinx-versioning and renamed some of the operations * Update configuration to support sphinx-versioning and update options for sphinx * Update configuration with linting corrections Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Correct and update the installation of a pre-release (#102) * Correct and update the installation of a pre-release * minor fixes to docs Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Content/1067/community doc (#115) * removed Jobs.submit usage due to invalid response (#92) * Update to galaxy.yml to exclude files * Update makefile to support rexx written modules * Generate and push zos_ping rst doc * Corrected clean up of a directory in makefile * Fixes spacing in RETURN values and formatting (#97) * Validate bk_dsn after assigning a value * Return backup path * Fix alloc command * Remove trailing whitespace * Fix job utility to work on all z/OS systems (#99) * Shinx and sphinx-versioning support and updates (#100) * Updated Makefile to support sphinx-versioning and renamed some of the operations * Update configuration to support sphinx-versioning and update options for sphinx * Update configuration with linting corrections * Correct and update the installation of a pre-release (#102) * Correct and update the installation of a pre-release * minor fixes to docs Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update data set utils (#104) * Update data set utils * Update function signatures * Module parameter not needed for encode util Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Change default encoding from UTF-8 to ISO8859-1 * Fetching non-existent VSAM * Update doc for encoding parameter * Update encoding to handle different charsets * Remove encoding default from argument spec * encoding validation and doc update * Fix typo * Bugfix/case conversion fix zos tso command (#108) * fixed issues with commands forced to uppercase * add support for single or multi commands, added relevant test cases * updated alias doc * Added index file to support shared doc site (#109) * Remove module parameter from EncodeUtils constructor * fix issue when no default encoding specified (#112) * fix issue when no default encoding specified * fix local job submission * Correct doc link to IMS collection to point to Galaxy * Added testing doc to community guidlines * Clean/delete accidental file * Fix Sphinx warning coming from ping module reference to zos_ssh plugin undefined label: zos_ssh_module * Added better arg parse to new dev addition to community guides * Fixes broken site anchor to the YAML configuration * Fix MD style Link * Remove additional MD style links * Fix mailformed RST link * Fix more RST links * Python syntax fixes in samples * Update betterArgParse to BetterArgParser * Expand pipelining=true doc * Correct Unicode quotes to ASCII quotes * Statement of direction on zoau and python * doc review of community guidelines * Doc review of Community guidelines * Fix malformed RST table * Update group vars requirements needed by IBM Pypthon * Update doc with var settings and sample all.yml * Update environ vars * Added environ LANG doc * Added doc to clarify shell support * Added advanced group_vars sample and doc * Update docs to reflect IBM Python announcement Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update release verson for Hub and Galaxy deployments * Doc review * Update version tag in doc generation conf * Add missing link reference * Update module rst based on module diffs Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Jake Jackson <jljacks93@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> Co-authored-by: zlbjlu <16587467+yourfuwa2015@users.noreply.github.com> Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com>
ddimatos
added a commit
that referenced
this pull request
Jul 11, 2020
* Initial collection commit for v0.0.1 (#1) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * added arg parser to module_utils (#3) * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * Updated job output for improved DD handling (#14) Fixes the job summary output being returned when return_output=false. * Security module updates (#15) * patched security holes * updated test cases * removed unnecessary import * updated execution bits * Update modules and documentation corrections (#17) * update documentation, remove unused imports * updated requirements * updated module imports, formatting, and spelling * doc update and formatting update * z/OS Ping Module Feature Delivered (#18) * added zos_ping, test, and test fixture update * Description completeness * Grammar completeness * Pulled repetitive code into functions * updated module docs * Minor spelling corrections * Repo updates to satisfy ansible-test sanity testing requirements (#19) * update formatting * formatting changes to appease ansible-test * update tests * fix string format use and quotes * update README * remove trailing whitespace * update group_vars trailing spaces * remove shebang in attempt to fix encoding issue * add back shebang lines * Added security updates and test cases (#20) * add new types for parser, update arg handling zos_job_submit * updated module argument parsing and security tests * uncommented security tests * updated README to reflect use of MIT license * update hash function * test and formatting updates * updated documentation to pass ansible-test * switch name back to normal.py, add appropriate ignores * update ignore files * added ignore for GPL * updated README, .gitignore and removed MIT code * remove broken tests * increase robustness of zos_job_query * make job query even more robust * Webdocs content for IBM z/OS Core Collection (#21) * Initial commit for reStructuredText content * Updates and clean up to the template * Updated grammar * Grammar edit * Grammar fixes * Text updates * Updates to restructured text, fixed link, renamed collection * Fix ibm_zos_core bold * Fixes ansible-test lint: docs/source/conf.py:51:161: E501: line too long (178 > 160 characters) * ansible doc changes to modules for better formatting * remove original_message return parameter (#23) * remove original_message return parameter * Fix reported linting errors from PR * Fix reported linting errors from PR * Fix reported linting errors from PR for zos_job_query.py * Module document updates, Makefile changes and jinja support for defau… (#24) * Module document updates, Makefile changes and jinja support for default:int * Doc updates per linting rules * Update mmodule doc and remove md files (#25) * Removed all MD files from docs directory * Remove README.md , its replaced by README.rst * Fix bolding for ibm_zos_core * Fix highlight for ibm_zos_core * Updated index.rst to correspond to the README.md updates * Updated README.rst to point to webdocs * Updated README.rst webdocs URL * Update the modules to their ansible-doc * Documentation updates (#26) * Updated galaxy.yml to reflect latest version 1.0 * Documentation copyright updates and formatting * Documentation udpates * Update z/OS data set module and add catalog functionality (#27) * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * catalog for VSAM working, bugs fixed, tests added * satisfy ansible-test requirements, minor arg validation updates * removed unit tests * raise min . ansible requirement due to security vulnerability * added documentation to VTOC * Update product name in webdocs (#28) * Update product name in webdocs * Fix trailing newline * Document updates include some template bugs * Integraded document review comments (#29) * Integrated document review comments * Incorrect break for bulleted line * Bug in Jinja template where property would tab to much (#31) * Updates to documentation and galaxy configuration (#32) * Fix to README.rst, duplicate wording (#34) * Remove duplicate action plugins link (#35) * testing readme rendering (#36) Ported the README.rst file to README.md to meet collection requirements. * Corrected README to point to MD, and long tag removed (#39) * Add two additional arg types to BetterArgHandler (#44) * Add two additional arg types * Remove os from os.path.isabs Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Data Set Utility for Common Data Set Related Tasks (#48) * DataSetUtils initial commit * LISTDS and LISTCAT output processing * Raise exception when data set is open for write * Fix LISTDS regex * Fix indentation * Add USS path support * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/832/fix non vsam non sms existence checking zos data set (#49) * updated catalog handling for non-vsam, non-sms * updated some method comment structures * bugfix/822/determine-data-set-type (#50) * Use mvscmd CLI instead of Python API * Fix linting errors * Remove unused variable Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Extend dynamic function support to the better arg parser (#51) * Dynamic argument support for dynamic functions. * improve default handling for reusability of BetterArgParser object. * add test cases for dynamic argument support * removed print statements from test cases * Updates to data-set-utils exceptions (#52) * Avoid raising exception for uncatalogued data sets * verify listcat output for uncatalogued datasets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Removed return block from documentation (#53) Removed message and changed, empty return block now * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * Fixed PATH (#55) * Update/973/add new supported data set characters betterargparser (#56) * updated valid data set types builtin type name fix * added test cases for data set types * Bugfix when checking if data set exists (#57) * Return False instead of None when data set doesn't exist * Test for membership should ne 'not in' Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Committing module zos_operator_action_query (#43) * Reviewd changes for feature zos_operator_action_query.py * Fix linting errors reported in PR * Linting corrections * fix the syntax error * fix lint errors * fix lint error * fix lint error * fix lint error * fix function test * Added zos_operator_action_query to the sanity exemption list for lack of GPLV3 Lic * Update author tag * Update the collection reference * add function test(need revise later) & doc update * updated regex and value capturing * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * updated docs, made environment agnostic test cases * added support for "*" and fixed splat handling * updated test cases * updated regex matching method, updates tests * Update short description Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Check for data set member existence (#58) * Check for data set member existence * Resolve linting issue Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix for job name capture group (#60) * Better ZOAU import missing messages (#61) * Better ZOAU import missing messages * added missing license and imports * Changed behavior on failure for zosoperator_action_query (#63) * changed behavior on failure * update tests for new expected output * Bugfix 1661 & 1704 fix query of job for different user zos_job_submit and zos_job_action_query (#64) * updates to fix job submission as other user * updated and optimized job utility * removed usage of ZOAU and updated jobs utility usage * update job_output return structure * improved error messages * add job log cleanup for test cases * Initial module zos_tso_command (#46) * Linting corrections * Update doc so html will generate * Lintig updates * Lintig updates v2 * Lintig updates v3 * Lintig updates v4 * Lintig updates v5 * Lintig updates v6 * Lintig updates v7 * Lintig updates v8 * Delete test_zos_tso_command_unit.py Remove the unit test as it only mocks things. * Documenation updates * Update errro message * zos-tso-command update * linting update * testcase linting updates * linting update * linting update * linting update * linting update * linting updates * linting update * new authorized long command handling method * linting update * test cases update * linting update * linting update * linting update * tso-command new version * tso-command linting update * linting update * linting update * linting update' * linting removal * lint removal * linting testcases Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> * Update module_utils with an encode utility (#59) * initial commmit for encode_utils * support for string encoding conversion * Support MVS data set conversion and string conversion modification * Recursively copy to PDS fix * Linting correction * Linting updates * Linting updates 01 * fix ENOENT problem after shutil.move * updates with zos_encode * Merged branch and commit for zos_encode (#65) * Merged branch and commit for zos_encode * Lint error , trailing white space * z/OS Fetch Module (#7) * initial commit * begin updating examples * zos_fetch module * return rc, stdout, stderr as separate return parameters on failure * untrack non-fetch related files * add rest of the files * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * added unit and functional tests * check file existence when LISTDS returns invalid dataset name * added 'wait_s' parameter * clarified description of encoding parameter * make use_qaulifer false by default * remove is_uss and os_vsam parameters * remove data_set directory * remove is_uss parameter from examples * message return block * Fix YAML formatting * return code formatting * remove fail_on_missing from action plugin * fail if USS file does not exist * data sets are assumed to be cataloged * instead of entire dict, pass individual values * redundant code when fetching data set member * rename transfer_from_uss to fetch_non_partitioned_data_set * remove test cases for uncataloged datasets * import naming conflict * add __future__ imports * add __future__ imports * assuming all data sets are cataloged * assuming all data sets are cataloged * fix formatting issues * fix formatting issues * remove blank lines with whitespaces * change example member name data to DATA for accuracy. * continuation line unindented for visual indent * conform to PEP8 standard * avoid automatic field numbering for format * specify python interpreter * extract only data set name if member present * Remove newline character from error message * Update RETURN block with sample message * Remove blank line from return block * Update version_added to correspond to Ansible version * Check if temp datasets are being created successfully * Datasets.read() returns None for empty data sets * Use BetterArgParser to parse module args * Added parameter types * Remove trailing whitespaces * Avoid passing extra args from action plugin * Remove unused imports * Ignore lack of GPLv3 license complaint * catch ValueError thrown ny BetterArgParser * Blank line contains whitespace * Reduce line lengths to 80 * Add white space around string addition * Checking volume type no longer required * Check if src or dest is provided first * Fix indentation * Indent multiline string for better readability * Use DataSetUtils to get data set type * Use DataSetUtils to determine data set existence * Remove unit tests * Fix linting * Add test case for empty data sets * default params for _fail_json * Test case for missing source * Test case for insufficient write permission * Test case for insufficient desgt write permission * Catch exceptions while executing module * Use updated DataSetutils constructor * Updated functional tests * Fix linting errors * Use ansible fetch module to fetch USS files * Checking for USS type is unnecessary * Use sha256 instead of sha1 for checksum validation * Remove message return block * Missing uss file test case * Minor cleanup * Add stdout_lines and stderr_lines to module response * Update action plugin with stderr_lines and stdout_lines * YAML syntax issue * Corrected spelling errors * Inaccurate description of flat * No need to wrap error messages returned by DataSetUtils * Update doc regarding HFS and ZFS support * Handle encoding for USS files * Add encoding to doc * Calculate temp file size for VSAM storage * Preserve encoding using SFTP * Resolve PEP8 issues * Check for data set member existence * raw binary not supported in PY2 * Fix linting issues * Change doc to reflect USS absolute path * Prepend dataset hlq before parameter verification * src not declared before being referenced * Add test case using use_qualifier * Updated doc link for fetch L(..) and suboptions * Use tempfile.mkstemp() for temp data storage * Remove default for encoding * default encoding values are None * Create destination dirs when flat is false * Add test case for flat hostname * Fix spelling * exist_ok is not valid keyword for PY2 * Detect read-only file system * Handle sftp errors * Capture module_stderr in case of module failure * Ensure dest is a directory for PDS transfer * Ensure dest is a directory except when fetching a member * Fix data set member destination path * Default values for encoding is not needed * Remove unused imports * Update 'msg' description * Clean up temp files in case of module failure * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * enhanced zoau import error message (#66) * enhanced zoau import error message * satisfy pep8 witespace requirement * Added new playbook for latest module collection and a filter * fixed linting issues, added license * updated spacing to conform to formatting standards * updated shell to command where possible * Use encode util to convert encoding (#68) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update RST and MD doc files to reflect next release * Update group vars to reflect zoau 1.0.2 new env vars * Update encode short description to fit within the limits of doc length * Update makefile for more generic use by others * Update to fix RST lint error , short line * Rename sub-dir from plugin to filter, doc error * doc tree correction * Update release notes to correspond to beta release tag * Update sample playbooks with doc, configuration and new variables * Upate docs to highlight users to review sample playbooks and JCL before running * Update sample JCL to run on my systems with default values and notes * Add supporting doc for behavioral inventory parameters and ansible.cfg * Fix the md style URL in the index RST file * Update short descriptions for modules * Fix README.md link to IMS collection * Fix README.md link to collection artifacts * Linting updates * Add an example that requires a command to be escaped (#71) * refine backup_file & put it into encode_utils (#70) * refine backup_file & put it into encode_utils * correction for sanity test * refactor for security and usability, fix test case * move copy logic to utility * refactor zos_encode, improve backup functionality Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Update galaxy.yml to reflect the latest version and authors * Fix import name change resulting from refactor * updated usage of encode_utils to encode * Documentation updates, corrections and alterations (#75) * Documentation updates, corrections and alterations * ID review-round 2 * Update the user description in the doc Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Content/81/zos tso command docreview (#73) * ID review of zos-tso-commands * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * Fix indentation Co-authored-by: ddimatos <dimatos@gmail.com> * Fix extra space in doc * fixed zos_ping action plugin issues (#77) * Add out-of-box awx support (#80) * Added encoding supplemental doc (#78) * Added encoding supplemental doc * Removed references to modules not available and causing doc warnings * Fix Inline literal start-string without end-string. * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Add binary support copying across USS and MVS * Copy USS file in binary mode * fixed bug and cleaned up code (#82) * fixed bug and cleaned up code * fixed linting issues * Avoid specifying -B and -F for cp * Copy MVS source to MVS dest * Documentation web content updates (#84) * First round of review of static docs * docs/source/installation.rst * Update block to rendor with syntax highlights Code block was using 'yaml' when it was 'sh' , renders better if we have the correct code block type. * Incorporated Demetri's review comments Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Replace only the first occurance of 'rec' * Minor updates in install,quickstart, and req pages (#86) * Updated make file with comments and recover from failure * Update Backup util (mvs_file_backup) (#85) * Alternative mvs_file_backup * fix backup logic, add encode backup tests * add mvs backup tests, util fixes * fix test for new logic path Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Udpate the restructured text to reflect version release (#89) * minor fixes and additional tests for zos_encode (#88) * removed Jobs.submit usage due to invalid response (#92) * Update to galaxy.yml to exclude files * Update makefile to support rexx written modules * Generate and push zos_ping rst doc * Corrected clean up of a directory in makefile * Fixes spacing in RETURN values and formatting (#97) * Validate bk_dsn after assigning a value * Return backup path * Fix alloc command * Remove trailing whitespace * Fix job utility to work on all z/OS systems (#99) * Shinx and sphinx-versioning support and updates (#100) * Updated Makefile to support sphinx-versioning and renamed some of the operations * Update configuration to support sphinx-versioning and update options for sphinx * Update configuration with linting corrections * Correct and update the installation of a pre-release (#102) * Correct and update the installation of a pre-release * minor fixes to docs Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update data set utils (#104) * Update data set utils * Update function signatures * Module parameter not needed for encode util Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Change default encoding from UTF-8 to ISO8859-1 * Fetching non-existent VSAM * Update doc for encoding parameter * Update encoding to handle different charsets * Remove encoding default from argument spec * encoding validation and doc update * Fix typo * Bugfix/case conversion fix zos tso command (#108) * fixed issues with commands forced to uppercase * add support for single or multi commands, added relevant test cases * updated alias doc * Added index file to support shared doc site (#109) * Remove module parameter from EncodeUtils constructor * fix issue when no default encoding specified (#112) * fix issue when no default encoding specified * fix local job submission * Feature/26/ansible zos copy module (#105) * initial commit * begin updating examples * default value 'yes/no' -> 'true/false' * updated documentation * copy to seq data set * copy local to seq data set * add data set size to module response * functional test cases for copy * test case for sequential data set * clean up copy to ps function * copy to pdse * assuming all data sets are cataloged * copy to PDS * Copy to VSAM * Copy to PDS(E) member * Actual directory size instead of max file size * Copy from remote source * Update copy * Reduce line lengths to less than 80 * Use mvscmd cli * remote pdse copy * IEBCOPY to copy data sets * saving changes * Create remote data set if it doesn't exist * Reorganize remote copy transfer * Update return params * Avoid using global module * Remove unused imports * Copy to USS * Clean up in case of module failure * Back up USS file * Copy content to remote file * Copy to sequential data set * Copy local dir to PDS * Copy remote PDS to PDS * Update backup file * USS dest not accessible if not present * Explicitly tag destination file * Changed file mode, group and owner * Change common file args for remote src * Change file common args after copying to dest * Preserve local file mode * Convert encoding for remote src * Added comments * Use backup util to back up files and data sets * Add validate parameter * Copy VSAM to VSAM * Support for symlinks * Copy PDS/PDSE member to USS * Fix formatting * Module should fail for empty VSAM * Added function signatures for all test cases * User better_arg_parser to determine member type * Add binary support for copying to USS target * Copy to PS in binary mode * Correct object instantiations * Import copy util * Remove write_rc * Resolve import issues * Move vsam copy handler to main copy handler class * Copy local dir to USS dir * Infer member name when it isn't provided * Added test cases for local and remote copy * src_member evalutes to True * Comments for extract_dsname() * IEBCOPY fails for incompatible recfm * Copy PDS member to PS * Copy data set member to non-existent PS * Copy local symlink * Add test case for uss file backup * Fix mvs file backup * Functional tests complete * Fix linting issues * Remove trailing whitespace * Ignore GPLv3 license * Remove is_vsam parameter * Fix license information * Fail action plugin * Add test cases to ensure module failure * Remove unneded parameters * Refactor to use updated data_set util * I/O abend when copying PDSE member to non-existent PS * Updated doc, comments, linting * Fix linting issues * Remove duplicate key from dic * Avoid hard-coding tmp path name * Change backup_path parameter to backup_file * Copy to data set member partial * Error converting encoding for local dir * Fix tmp cleanup bug * Updated test cases for backup_file * plugins/modules/zos_copy.py * Reactor code for better readability * Fix pep8 issue * Cleanup error on module failure * plugins/modules/zos_copy.py * Doc update mode parameter * Use native shell instead of bash for functional testing * Return invocation parameter * Remove unnecessary imports * plugins/modules/zos_copy.py * Return backup_file on module failure Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Feature/1089/zos lineinfile (#111) * initial version for interface and options * add backup option * update lineinfile * make golden path work * fix the bugs so that method could support none vars * refactor based on new dsed * updated params and return * Initial check in for backup feature support * support all charsets; updated backup feature doc * clean up; used better_arg_parser * use latest data_set module_util * change the returned string to be loadable by json.loads (BOF/EOF) * uss test cases * dataset test cases; create a dataset/file with content instead of being dependent on existing source * uss test env set bug fix * added UTF-8 in test cases; simplified dataset names * use new dedicated ZOAU function for lineinfile * added assert for expected strings * encoding parameter from absent * Linting updates from pipeline tests * Linting updates and corrections * Linting udpates continued * Update sanity exemption file and linting corrections * Doc error, indentation off, breaking rest of doc parse * module arg and doc updates * Formatting updates * Further doc updates * regexp alias fix * updated docs; added comments; updated parameters * fix sanity checks * fix RETURN key type; lint error * code review changes * fixed linting issues * removed incorrectly added choices for insertafter/insertbefore * added negative test cases fo not supported datasets * moved choices to description; fixed test cases lint issue * changed the choices format in description * add escape if string was quoted * added test cases for escaped/unescaped quotation * used 'src' for target location; moved the params checking before taking backup * fixed indentation * Doc edits for lineinfile, encode, and fetch * remove relative import method * removed the default values for insertafter/insertbefore * renamed helper functions; updated pytest.ini for test cases; fixed TEST_INFO * remove jcl tmp directory if it already exists * run all test_* Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: behnam-ibm <behnam@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/1067/community doc (#113) * Correct doc link to IMS collection to point to Galaxy * Added testing doc to community guidlines * Clean/delete accidental file * Fix Sphinx warning coming from ping module reference to zos_ssh plugin undefined label: zos_ssh_module * Added better arg parse to new dev addition to community guides * Fixes broken site anchor to the YAML configuration * Fix MD style Link * Remove additional MD style links * Fix mailformed RST link * Fix more RST links * Python syntax fixes in samples * Update betterArgParse to BetterArgParser * Expand pipelining=true doc * Correct Unicode quotes to ASCII quotes * Statement of direction on zoau and python * doc review of community guidelines * Doc review of Community guidelines * Fix malformed RST table * Update group vars requirements needed by IBM Pypthon * Update doc with var settings and sample all.yml * Update environ vars * Added environ LANG doc * Added doc to clarify shell support * Added advanced group_vars sample and doc * Update docs to reflect IBM Python announcement Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Remove unnecessary description of PDSE transfer * Update version tag in doc generation conf * Add missing link reference * Doc review (#118) Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update IBM Python download link * Add system info to job query (#127) * Add system info to job query * Change sysname to system for job query Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Close file descriptor (#128) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/2298/zos raw (#120) * add utilities for zos_raw * updated DD statement to better handle concatenations * update parameter names * fix concat naming * refactor of zos_data_set, add new features * updated examples * added steplib dd type * fix steplib data set * update valid concat members * fix datasetdefinition for steplibs * update dd statement types * argument parsing working in zos_raw * module argument parsing updates * updated valid arguments for consistency * update tests for new zos_data_set format * zos_raw module working * module doc updates * updated function documentation * backup, reuse and replace working * updated documentation * cleaned up wrapper classes * updated existence checking, function doc updates * spelling fixes, added new parameters for dd_unix * add list of string support for StdinDefinition * properly handle space units * initial set of test cases, added aliases for program, rename args to parms to match jcl naming conventions * added remaining happy path data set DD tests, added input dd tests. Fixed small bug in raw related to invalid parameter options * update example playbooks to account for new zos_data_set arguments * added initial functional tests for unix file dd statements * added access group support, fixed status group * added remaining tests for dd_unix and dd_dummy * fixed bug on dd_concat contents or backup returned * add dd_concat test cases * update parms to parm to match jcl naming * added execution tests for zos_raw * add final test cases * add special type functions for ksds related params * update import handling for special cases * add VIO DD support * uncomment test cases * fix basic linting issues, default arguments * add support for directory blocks * fix some linting issues * Revert "fix some linting issues" This reverts commit 44d0e14. * linting fixes * module doc and linting updates * additional doc fixes for linting sanity tests * added examples * added catch for nonexistent deletion * updated docs for proper formatting * update ignore files for new module * added dd_output support, standardized dd names * removed unnecessary import wrappers * improved validation for encoding types. * added new example with dd_output type * Merge branch 'origin/dev' * Revert "Merge branch 'origin/dev'" This reverts commit 1befb8e. * fixes for python 2 controller support * modify encode utils import * add utf-8 encoding tag to unit tests * improve check mode support * fix mock of AnsibleModule for encoding check cases * fix unit test formatting for pep8 * fix test case helpers * comment out failing test case unrelated to zos_raw * update encoding conversion handling * update rexx variables to capture correct environment variables * update naming based on discussion * rename test case files, update test cases * update default for output data set * update test ignore files for rename * doc edits (#133) * doc edits * fix minor pep8 issues Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * add python 2 support * remove redundant code comments * add temp data set support, backwards compatibility fix with old interface, update docs and tests * update documentation for return_content * minor doc changes for clarity * improve documentation clarity * fix incorrect syntax for option name Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Update configuration vars in doc and yml (#131) * remove ZOAU Python API deps for zos_operator (#132) * remove ZOAU deps which break in 1.0.3 * add back shebang * remove unsafe shell usage * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/2516/release doc update (#136) * Update the version in galaxy.yml * edits to support doc gen * Updated for doc gen * updated all module doc for release * Update template to upper case boolean when no default present * update raw module doc to fomat as the indentation intended * updated module RST * add details to module enhancements * Update message around Rocket's python * Update Rocket Python Message * Reference Rocket Python configuration in requirements * Updated doc to reflect the latest dependencies * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Remove see also reference to external collections * updated zos_copy module RST * Sample playbook for latest version * Updated ZOAU PTF information with the latest PTF UI70435 * Updates to raw doc to handle the error - Definition list ends without a blank line * Update copy module docs * Update and add clarity around Rocket Python * Requirements order change Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update conf for v1.2.0-beta.1 * Fix E231: missing whitespace Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Jake Jackson <jljacks93@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> Co-authored-by: zlbjlu <16587467+yourfuwa2015@users.noreply.github.com> Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: balkajbaf <62961336+balkajbaf@users.noreply.github.com> Co-authored-by: behnam-ibm <behnam@ibm.com>
ddimatos
added a commit
that referenced
this pull request
Aug 14, 2020
* Initial collection commit for v0.0.1 (#1) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * added arg parser to module_utils (#3) * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * Updated job output for improved DD handling (#14) Fixes the job summary output being returned when return_output=false. * Security module updates (#15) * patched security holes * updated test cases * removed unnecessary import * updated execution bits * Update modules and documentation corrections (#17) * update documentation, remove unused imports * updated requirements * updated module imports, formatting, and spelling * doc update and formatting update * z/OS Ping Module Feature Delivered (#18) * added zos_ping, test, and test fixture update * Description completeness * Grammar completeness * Pulled repetitive code into functions * updated module docs * Minor spelling corrections * Repo updates to satisfy ansible-test sanity testing requirements (#19) * update formatting * formatting changes to appease ansible-test * update tests * fix string format use and quotes * update README * remove trailing whitespace * update group_vars trailing spaces * remove shebang in attempt to fix encoding issue * add back shebang lines * Added security updates and test cases (#20) * add new types for parser, update arg handling zos_job_submit * updated module argument parsing and security tests * uncommented security tests * updated README to reflect use of MIT license * update hash function * test and formatting updates * updated documentation to pass ansible-test * switch name back to normal.py, add appropriate ignores * update ignore files * added ignore for GPL * updated README, .gitignore and removed MIT code * remove broken tests * increase robustness of zos_job_query * make job query even more robust * Webdocs content for IBM z/OS Core Collection (#21) * Initial commit for reStructuredText content * Updates and clean up to the template * Updated grammar * Grammar edit * Grammar fixes * Text updates * Updates to restructured text, fixed link, renamed collection * Fix ibm_zos_core bold * Fixes ansible-test lint: docs/source/conf.py:51:161: E501: line too long (178 > 160 characters) * ansible doc changes to modules for better formatting * remove original_message return parameter (#23) * remove original_message return parameter * Fix reported linting errors from PR * Fix reported linting errors from PR * Fix reported linting errors from PR for zos_job_query.py * Module document updates, Makefile changes and jinja support for defau… (#24) * Module document updates, Makefile changes and jinja support for default:int * Doc updates per linting rules * Update mmodule doc and remove md files (#25) * Removed all MD files from docs directory * Remove README.md , its replaced by README.rst * Fix bolding for ibm_zos_core * Fix highlight for ibm_zos_core * Updated index.rst to correspond to the README.md updates * Updated README.rst to point to webdocs * Updated README.rst webdocs URL * Update the modules to their ansible-doc * Documentation updates (#26) * Updated galaxy.yml to reflect latest version 1.0 * Documentation copyright updates and formatting * Documentation udpates * Update z/OS data set module and add catalog functionality (#27) * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * catalog for VSAM working, bugs fixed, tests added * satisfy ansible-test requirements, minor arg validation updates * removed unit tests * raise min . ansible requirement due to security vulnerability * added documentation to VTOC * Update product name in webdocs (#28) * Update product name in webdocs * Fix trailing newline * Document updates include some template bugs * Integraded document review comments (#29) * Integrated document review comments * Incorrect break for bulleted line * Bug in Jinja template where property would tab to much (#31) * Updates to documentation and galaxy configuration (#32) * Fix to README.rst, duplicate wording (#34) * Remove duplicate action plugins link (#35) * testing readme rendering (#36) Ported the README.rst file to README.md to meet collection requirements. * Corrected README to point to MD, and long tag removed (#39) * Add two additional arg types to BetterArgHandler (#44) * Add two additional arg types * Remove os from os.path.isabs Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Data Set Utility for Common Data Set Related Tasks (#48) * DataSetUtils initial commit * LISTDS and LISTCAT output processing * Raise exception when data set is open for write * Fix LISTDS regex * Fix indentation * Add USS path support * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/832/fix non vsam non sms existence checking zos data set (#49) * updated catalog handling for non-vsam, non-sms * updated some method comment structures * bugfix/822/determine-data-set-type (#50) * Use mvscmd CLI instead of Python API * Fix linting errors * Remove unused variable Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Extend dynamic function support to the better arg parser (#51) * Dynamic argument support for dynamic functions. * improve default handling for reusability of BetterArgParser object. * add test cases for dynamic argument support * removed print statements from test cases * Updates to data-set-utils exceptions (#52) * Avoid raising exception for uncatalogued data sets * verify listcat output for uncatalogued datasets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Removed return block from documentation (#53) Removed message and changed, empty return block now * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * Fixed PATH (#55) * Update/973/add new supported data set characters betterargparser (#56) * updated valid data set types builtin type name fix * added test cases for data set types * Bugfix when checking if data set exists (#57) * Return False instead of None when data set doesn't exist * Test for membership should ne 'not in' Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Committing module zos_operator_action_query (#43) * Reviewd changes for feature zos_operator_action_query.py * Fix linting errors reported in PR * Linting corrections * fix the syntax error * fix lint errors * fix lint error * fix lint error * fix lint error * fix function test * Added zos_operator_action_query to the sanity exemption list for lack of GPLV3 Lic * Update author tag * Update the collection reference * add function test(need revise later) & doc update * updated regex and value capturing * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * updated docs, made environment agnostic test cases * added support for "*" and fixed splat handling * updated test cases * updated regex matching method, updates tests * Update short description Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Check for data set member existence (#58) * Check for data set member existence * Resolve linting issue Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix for job name capture group (#60) * Better ZOAU import missing messages (#61) * Better ZOAU import missing messages * added missing license and imports * Changed behavior on failure for zosoperator_action_query (#63) * changed behavior on failure * update tests for new expected output * Bugfix 1661 & 1704 fix query of job for different user zos_job_submit and zos_job_action_query (#64) * updates to fix job submission as other user * updated and optimized job utility * removed usage of ZOAU and updated jobs utility usage * update job_output return structure * improved error messages * add job log cleanup for test cases * Initial module zos_tso_command (#46) * Linting corrections * Update doc so html will generate * Lintig updates * Lintig updates v2 * Lintig updates v3 * Lintig updates v4 * Lintig updates v5 * Lintig updates v6 * Lintig updates v7 * Lintig updates v8 * Delete test_zos_tso_command_unit.py Remove the unit test as it only mocks things. * Documenation updates * Update errro message * zos-tso-command update * linting update * testcase linting updates * linting update * linting update * linting update * linting update * linting updates * linting update * new authorized long command handling method * linting update * test cases update * linting update * linting update * linting update * tso-command new version * tso-command linting update * linting update * linting update * linting update' * linting removal * lint removal * linting testcases Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> * Update module_utils with an encode utility (#59) * initial commmit for encode_utils * support for string encoding conversion * Support MVS data set conversion and string conversion modification * Recursively copy to PDS fix * Linting correction * Linting updates * Linting updates 01 * fix ENOENT problem after shutil.move * updates with zos_encode * Merged branch and commit for zos_encode (#65) * Merged branch and commit for zos_encode * Lint error , trailing white space * z/OS Fetch Module (#7) * initial commit * begin updating examples * zos_fetch module * return rc, stdout, stderr as separate return parameters on failure * untrack non-fetch related files * add rest of the files * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * added unit and functional tests * check file existence when LISTDS returns invalid dataset name * added 'wait_s' parameter * clarified description of encoding parameter * make use_qaulifer false by default * remove is_uss and os_vsam parameters * remove data_set directory * remove is_uss parameter from examples * message return block * Fix YAML formatting * return code formatting * remove fail_on_missing from action plugin * fail if USS file does not exist * data sets are assumed to be cataloged * instead of entire dict, pass individual values * redundant code when fetching data set member * rename transfer_from_uss to fetch_non_partitioned_data_set * remove test cases for uncataloged datasets * import naming conflict * add __future__ imports * add __future__ imports * assuming all data sets are cataloged * assuming all data sets are cataloged * fix formatting issues * fix formatting issues * remove blank lines with whitespaces * change example member name data to DATA for accuracy. * continuation line unindented for visual indent * conform to PEP8 standard * avoid automatic field numbering for format * specify python interpreter * extract only data set name if member present * Remove newline character from error message * Update RETURN block with sample message * Remove blank line from return block * Update version_added to correspond to Ansible version * Check if temp datasets are being created successfully * Datasets.read() returns None for empty data sets * Use BetterArgParser to parse module args * Added parameter types * Remove trailing whitespaces * Avoid passing extra args from action plugin * Remove unused imports * Ignore lack of GPLv3 license complaint * catch ValueError thrown ny BetterArgParser * Blank line contains whitespace * Reduce line lengths to 80 * Add white space around string addition * Checking volume type no longer required * Check if src or dest is provided first * Fix indentation * Indent multiline string for better readability * Use DataSetUtils to get data set type * Use DataSetUtils to determine data set existence * Remove unit tests * Fix linting * Add test case for empty data sets * default params for _fail_json * Test case for missing source * Test case for insufficient write permission * Test case for insufficient desgt write permission * Catch exceptions while executing module * Use updated DataSetutils constructor * Updated functional tests * Fix linting errors * Use ansible fetch module to fetch USS files * Checking for USS type is unnecessary * Use sha256 instead of sha1 for checksum validation * Remove message return block * Missing uss file test case * Minor cleanup * Add stdout_lines and stderr_lines to module response * Update action plugin with stderr_lines and stdout_lines * YAML syntax issue * Corrected spelling errors * Inaccurate description of flat * No need to wrap error messages returned by DataSetUtils * Update doc regarding HFS and ZFS support * Handle encoding for USS files * Add encoding to doc * Calculate temp file size for VSAM storage * Preserve encoding using SFTP * Resolve PEP8 issues * Check for data set member existence * raw binary not supported in PY2 * Fix linting issues * Change doc to reflect USS absolute path * Prepend dataset hlq before parameter verification * src not declared before being referenced * Add test case using use_qualifier * Updated doc link for fetch L(..) and suboptions * Use tempfile.mkstemp() for temp data storage * Remove default for encoding * default encoding values are None * Create destination dirs when flat is false * Add test case for flat hostname * Fix spelling * exist_ok is not valid keyword for PY2 * Detect read-only file system * Handle sftp errors * Capture module_stderr in case of module failure * Ensure dest is a directory for PDS transfer * Ensure dest is a directory except when fetching a member * Fix data set member destination path * Default values for encoding is not needed * Remove unused imports * Update 'msg' description * Clean up temp files in case of module failure * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * enhanced zoau import error message (#66) * enhanced zoau import error message * satisfy pep8 witespace requirement * Added new playbook for latest module collection and a filter * fixed linting issues, added license * updated spacing to conform to formatting standards * updated shell to command where possible * Use encode util to convert encoding (#68) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update RST and MD doc files to reflect next release * Update group vars to reflect zoau 1.0.2 new env vars * Update encode short description to fit within the limits of doc length * Update makefile for more generic use by others * Update to fix RST lint error , short line * Rename sub-dir from plugin to filter, doc error * doc tree correction * Update release notes to correspond to beta release tag * Update sample playbooks with doc, configuration and new variables * Upate docs to highlight users to review sample playbooks and JCL before running * Update sample JCL to run on my systems with default values and notes * Add supporting doc for behavioral inventory parameters and ansible.cfg * Fix the md style URL in the index RST file * Update short descriptions for modules * Fix README.md link to IMS collection * Fix README.md link to collection artifacts * Linting updates * Add an example that requires a command to be escaped (#71) * refine backup_file & put it into encode_utils (#70) * refine backup_file & put it into encode_utils * correction for sanity test * refactor for security and usability, fix test case * move copy logic to utility * refactor zos_encode, improve backup functionality Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Update galaxy.yml to reflect the latest version and authors * Fix import name change resulting from refactor * updated usage of encode_utils to encode * Documentation updates, corrections and alterations (#75) * Documentation updates, corrections and alterations * ID review-round 2 * Update the user description in the doc Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Content/81/zos tso command docreview (#73) * ID review of zos-tso-commands * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * Fix indentation Co-authored-by: ddimatos <dimatos@gmail.com> * Fix extra space in doc * fixed zos_ping action plugin issues (#77) * Add out-of-box awx support (#80) * Added encoding supplemental doc (#78) * Added encoding supplemental doc * Removed references to modules not available and causing doc warnings * Fix Inline literal start-string without end-string. * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Add binary support copying across USS and MVS * Copy USS file in binary mode * fixed bug and cleaned up code (#82) * fixed bug and cleaned up code * fixed linting issues * Avoid specifying -B and -F for cp * Copy MVS source to MVS dest * Documentation web content updates (#84) * First round of review of static docs * docs/source/installation.rst * Update block to rendor with syntax highlights Code block was using 'yaml' when it was 'sh' , renders better if we have the correct code block type. * Incorporated Demetri's review comments Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Replace only the first occurance of 'rec' * Minor updates in install,quickstart, and req pages (#86) * Updated make file with comments and recover from failure * Update Backup util (mvs_file_backup) (#85) * Alternative mvs_file_backup * fix backup logic, add encode backup tests * add mvs backup tests, util fixes * fix test for new logic path Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Udpate the restructured text to reflect version release (#89) * minor fixes and additional tests for zos_encode (#88) * removed Jobs.submit usage due to invalid response (#92) * Update to galaxy.yml to exclude files * Update makefile to support rexx written modules * Generate and push zos_ping rst doc * Corrected clean up of a directory in makefile * Fixes spacing in RETURN values and formatting (#97) * Validate bk_dsn after assigning a value * Return backup path * Fix alloc command * Remove trailing whitespace * Fix job utility to work on all z/OS systems (#99) * Shinx and sphinx-versioning support and updates (#100) * Updated Makefile to support sphinx-versioning and renamed some of the operations * Update configuration to support sphinx-versioning and update options for sphinx * Update configuration with linting corrections * Correct and update the installation of a pre-release (#102) * Correct and update the installation of a pre-release * minor fixes to docs Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update data set utils (#104) * Update data set utils * Update function signatures * Module parameter not needed for encode util Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Change default encoding from UTF-8 to ISO8859-1 * Fetching non-existent VSAM * Update doc for encoding parameter * Update encoding to handle different charsets * Remove encoding default from argument spec * encoding validation and doc update * Fix typo * Bugfix/case conversion fix zos tso command (#108) * fixed issues with commands forced to uppercase * add support for single or multi commands, added relevant test cases * updated alias doc * Added index file to support shared doc site (#109) * Remove module parameter from EncodeUtils constructor * fix issue when no default encoding specified (#112) * fix issue when no default encoding specified * fix local job submission * Feature/26/ansible zos copy module (#105) * initial commit * begin updating examples * default value 'yes/no' -> 'true/false' * updated documentation * copy to seq data set * copy local to seq data set * add data set size to module response * functional test cases for copy * test case for sequential data set * clean up copy to ps function * copy to pdse * assuming all data sets are cataloged * copy to PDS * Copy to VSAM * Copy to PDS(E) member * Actual directory size instead of max file size * Copy from remote source * Update copy * Reduce line lengths to less than 80 * Use mvscmd cli * remote pdse copy * IEBCOPY to copy data sets * saving changes * Create remote data set if it doesn't exist * Reorganize remote copy transfer * Update return params * Avoid using global module * Remove unused imports * Copy to USS * Clean up in case of module failure * Back up USS file * Copy content to remote file * Copy to sequential data set * Copy local dir to PDS * Copy remote PDS to PDS * Update backup file * USS dest not accessible if not present * Explicitly tag destination file * Changed file mode, group and owner * Change common file args for remote src * Change file common args after copying to dest * Preserve local file mode * Convert encoding for remote src * Added comments * Use backup util to back up files and data sets * Add validate parameter * Copy VSAM to VSAM * Support for symlinks * Copy PDS/PDSE member to USS * Fix formatting * Module should fail for empty VSAM * Added function signatures for all test cases * User better_arg_parser to determine member type * Add binary support for copying to USS target * Copy to PS in binary mode * Correct object instantiations * Import copy util * Remove write_rc * Resolve import issues * Move vsam copy handler to main copy handler class * Copy local dir to USS dir * Infer member name when it isn't provided * Added test cases for local and remote copy * src_member evalutes to True * Comments for extract_dsname() * IEBCOPY fails for incompatible recfm * Copy PDS member to PS * Copy data set member to non-existent PS * Copy local symlink * Add test case for uss file backup * Fix mvs file backup * Functional tests complete * Fix linting issues * Remove trailing whitespace * Ignore GPLv3 license * Remove is_vsam parameter * Fix license information * Fail action plugin * Add test cases to ensure module failure * Remove unneded parameters * Refactor to use updated data_set util * I/O abend when copying PDSE member to non-existent PS * Updated doc, comments, linting * Fix linting issues * Remove duplicate key from dic * Avoid hard-coding tmp path name * Change backup_path parameter to backup_file * Copy to data set member partial * Error converting encoding for local dir * Fix tmp cleanup bug * Updated test cases for backup_file * plugins/modules/zos_copy.py * Reactor code for better readability * Fix pep8 issue * Cleanup error on module failure * plugins/modules/zos_copy.py * Doc update mode parameter * Use native shell instead of bash for functional testing * Return invocation parameter * Remove unnecessary imports * plugins/modules/zos_copy.py * Return backup_file on module failure Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Feature/1089/zos lineinfile (#111) * initial version for interface and options * add backup option * update lineinfile * make golden path work * fix the bugs so that method could support none vars * refactor based on new dsed * updated params and return * Initial check in for backup feature support * support all charsets; updated backup feature doc * clean up; used better_arg_parser * use latest data_set module_util * change the returned string to be loadable by json.loads (BOF/EOF) * uss test cases * dataset test cases; create a dataset/file with content instead of being dependent on existing source * uss test env set bug fix * added UTF-8 in test cases; simplified dataset names * use new dedicated ZOAU function for lineinfile * added assert for expected strings * encoding parameter from absent * Linting updates from pipeline tests * Linting updates and corrections * Linting udpates continued * Update sanity exemption file and linting corrections * Doc error, indentation off, breaking rest of doc parse * module arg and doc updates * Formatting updates * Further doc updates * regexp alias fix * updated docs; added comments; updated parameters * fix sanity checks * fix RETURN key type; lint error * code review changes * fixed linting issues * removed incorrectly added choices for insertafter/insertbefore * added negative test cases fo not supported datasets * moved choices to description; fixed test cases lint issue * changed the choices format in description * add escape if string was quoted * added test cases for escaped/unescaped quotation * used 'src' for target location; moved the params checking before taking backup * fixed indentation * Doc edits for lineinfile, encode, and fetch * remove relative import method * removed the default values for insertafter/insertbefore * renamed helper functions; updated pytest.ini for test cases; fixed TEST_INFO * remove jcl tmp directory if it already exists * run all test_* Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: behnam-ibm <behnam@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/1067/community doc (#113) * Correct doc link to IMS collection to point to Galaxy * Added testing doc to community guidlines * Clean/delete accidental file * Fix Sphinx warning coming from ping module reference to zos_ssh plugin undefined label: zos_ssh_module * Added better arg parse to new dev addition to community guides * Fixes broken site anchor to the YAML configuration * Fix MD style Link * Remove additional MD style links * Fix mailformed RST link * Fix more RST links * Python syntax fixes in samples * Update betterArgParse to BetterArgParser * Expand pipelining=true doc * Correct Unicode quotes to ASCII quotes * Statement of direction on zoau and python * doc review of community guidelines * Doc review of Community guidelines * Fix malformed RST table * Update group vars requirements needed by IBM Pypthon * Update doc with var settings and sample all.yml * Update environ vars * Added environ LANG doc * Added doc to clarify shell support * Added advanced group_vars sample and doc * Update docs to reflect IBM Python announcement Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Remove unnecessary description of PDSE transfer * Update version tag in doc generation conf * Add missing link reference * Doc review (#118) Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update IBM Python download link * Add system info to job query (#127) * Add system info to job query * Change sysname to system for job query Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Close file descriptor (#128) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/2298/zos raw (#120) * add utilities for zos_raw * updated DD statement to better handle concatenations * update parameter names * fix concat naming * refactor of zos_data_set, add new features * updated examples * added steplib dd type * fix steplib data set * update valid concat members * fix datasetdefinition for steplibs * update dd statement types * argument parsing working in zos_raw * module argument parsing updates * updated valid arguments for consistency * update tests for new zos_data_set format * zos_raw module working * module doc updates * updated function documentation * backup, reuse and replace working * updated documentation * cleaned up wrapper classes * updated existence checking, function doc updates * spelling fixes, added new parameters for dd_unix * add list of string support for StdinDefinition * properly handle space units * initial set of test cases, added aliases for program, rename args to parms to match jcl naming conventions * added remaining happy path data set DD tests, added input dd tests. Fixed small bug in raw related to invalid parameter options * update example playbooks to account for new zos_data_set arguments * added initial functional tests for unix file dd statements * added access group support, fixed status group * added remaining tests for dd_unix and dd_dummy * fixed bug on dd_concat contents or backup returned * add dd_concat test cases * update parms to parm to match jcl naming * added execution tests for zos_raw * add final test cases * add special type functions for ksds related params * update import handling for special cases * add VIO DD support * uncomment test cases * fix basic linting issues, default arguments * add support for directory blocks * fix some linting issues * Revert "fix some linting issues" This reverts commit 44d0e14. * linting fixes * module doc and linting updates * additional doc fixes for linting sanity tests * added examples * added catch for nonexistent deletion * updated docs for proper formatting * update ignore files for new module * added dd_output support, standardized dd names * removed unnecessary import wrappers * improved validation for encoding types. * added new example with dd_output type * Merge branch 'origin/dev' * Revert "Merge branch 'origin/dev'" This reverts commit 1befb8e. * fixes for python 2 controller support * modify encode utils import * add utf-8 encoding tag to unit tests * improve check mode support * fix mock of AnsibleModule for encoding check cases * fix unit test formatting for pep8 * fix test case helpers * comment out failing test case unrelated to zos_raw * update encoding conversion handling * update rexx variables to capture correct environment variables * update naming based on discussion * rename test case files, update test cases * update default for output data set * update test ignore files for rename * doc edits (#133) * doc edits * fix minor pep8 issues Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * add python 2 support * remove redundant code comments * add temp data set support, backwards compatibility fix with old interface, update docs and tests * update documentation for return_content * minor doc changes for clarity * improve documentation clarity * fix incorrect syntax for option name Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Update configuration vars in doc and yml (#131) * remove ZOAU Python API deps for zos_operator (#132) * remove ZOAU deps which break in 1.0.3 * add back shebang * remove unsafe shell usage * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/2516/release doc update (#136) * Update the version in galaxy.yml * edits to support doc gen * Updated for doc gen * updated all module doc for release * Update template to upper case boolean when no default present * update raw module doc to fomat as the indentation intended * updated module RST * add details to module enhancements * Update message around Rocket's python * Update Rocket Python Message * Reference Rocket Python configuration in requirements * Updated doc to reflect the latest dependencies * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Remove see also reference to external collections * updated zos_copy module RST * Sample playbook for latest version * Updated ZOAU PTF information with the latest PTF UI70435 * Updates to raw doc to handle the error - Definition list ends without a blank line * Update copy module docs * Update and add clarity around Rocket Python * Requirements order change Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update conf for v1.2.0-beta.1 * Fix E231: missing whitespace * Changes to support single source doc site (#145) * Changes to support single source doc site * Under indentation * Under indentation 2 * doc review * doc review of additional files * Update the shell scripts to create a single requirements RST Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update/release v1.2.0 beta.2 (#162) * Fix E231: missing whitespace (#138) * Specify SFTP port for fetch and copy (#141) * Add sftp_port parameter to fetch and copy * Add sftp_port parameter to fetch and copy * Fix spelling * Missing positional arguments * Test case for negative port specification * Check for valid port range * Remove redundant assertions Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/dependency discovery (#142) * add host setup playbook * handle error cases * restructure and improve robustness of host-setup * add max_search_depth variable * change file extension for consistency * add documentation for host-setup.yaml * add in-file docs for tasks files * add important note for task files * Remove redundant data_set_util * Move error classes to the bottom * Rename backup_file to backup_name (#144) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/zos_copy record format (#147) * Update uss2mvs copy format * Default PS allocation should be FB * Use ZOAU Copy for sequential data sets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/148/accept all valid data set name characters (#149) * update regex for data set name * add test case for special characters * Add ignore_sftp_stderr parameter for fetch and copy * improve cleanup (#151) * change file extension * add /usr/sbin to default PATH * Use USS cp command to copy to PS * Bugfix: zos operator action query not returning all outstanding messages (#154) * remove ZOAU python API usage * add back ZOAU python API, update parsing regex * Changes to support single source doc site (#155) * Fix E231: missing whitespace * Changes to support single source doc site * Under indentation * Under indentation 2 * doc review * doc review of additional files * Update the shell scripts to create a single requirements RST Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * add verbose support and steplib fix (#156) * udpate maxdepth for RST to 1 * Content/3397/update docs (#157) * Module known issues and release notes added * Updated doc and sample host config playbook doc * Fix use ASCII quotes instead of Unicode quotes * use ASCII quotes instead of Unicode quotes * Update galaxy to latest doc site and version Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Jake Jackson <jljacks93@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> Co-authored-by: zlbjlu <16587467+yourfuwa2015@users.noreply.github.com> Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: balkajbaf <62961336+balkajbaf@users.noreply.github.com> Co-authored-by: behnam-ibm <behnam@ibm.com>
ddimatos
added a commit
that referenced
this pull request
Aug 17, 2020
* Initial collection commit for v0.0.1 (#1) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * added arg parser to module_utils (#3) * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * Updated job output for improved DD handling (#14) Fixes the job summary output being returned when return_output=false. * Security module updates (#15) * patched security holes * updated test cases * removed unnecessary import * updated execution bits * Update modules and documentation corrections (#17) * update documentation, remove unused imports * updated requirements * updated module imports, formatting, and spelling * doc update and formatting update * z/OS Ping Module Feature Delivered (#18) * added zos_ping, test, and test fixture update * Description completeness * Grammar completeness * Pulled repetitive code into functions * updated module docs * Minor spelling corrections * Repo updates to satisfy ansible-test sanity testing requirements (#19) * update formatting * formatting changes to appease ansible-test * update tests * fix string format use and quotes * update README * remove trailing whitespace * update group_vars trailing spaces * remove shebang in attempt to fix encoding issue * add back shebang lines * Added security updates and test cases (#20) * add new types for parser, update arg handling zos_job_submit * updated module argument parsing and security tests * uncommented security tests * updated README to reflect use of MIT license * update hash function * test and formatting updates * updated documentation to pass ansible-test * switch name back to normal.py, add appropriate ignores * update ignore files * added ignore for GPL * updated README, .gitignore and removed MIT code * remove broken tests * increase robustness of zos_job_query * make job query even more robust * Webdocs content for IBM z/OS Core Collection (#21) * Initial commit for reStructuredText content * Updates and clean up to the template * Updated grammar * Grammar edit * Grammar fixes * Text updates * Updates to restructured text, fixed link, renamed collection * Fix ibm_zos_core bold * Fixes ansible-test lint: docs/source/conf.py:51:161: E501: line too long (178 > 160 characters) * ansible doc changes to modules for better formatting * remove original_message return parameter (#23) * remove original_message return parameter * Fix reported linting errors from PR * Fix reported linting errors from PR * Fix reported linting errors from PR for zos_job_query.py * Module document updates, Makefile changes and jinja support for defau… (#24) * Module document updates, Makefile changes and jinja support for default:int * Doc updates per linting rules * Update mmodule doc and remove md files (#25) * Removed all MD files from docs directory * Remove README.md , its replaced by README.rst * Fix bolding for ibm_zos_core * Fix highlight for ibm_zos_core * Updated index.rst to correspond to the README.md updates * Updated README.rst to point to webdocs * Updated README.rst webdocs URL * Update the modules to their ansible-doc * Documentation updates (#26) * Updated galaxy.yml to reflect latest version 1.0 * Documentation copyright updates and formatting * Documentation udpates * Update z/OS data set module and add catalog functionality (#27) * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * catalog for VSAM working, bugs fixed, tests added * satisfy ansible-test requirements, minor arg validation updates * removed unit tests * raise min . ansible requirement due to security vulnerability * added documentation to VTOC * Update product name in webdocs (#28) * Update product name in webdocs * Fix trailing newline * Document updates include some template bugs * Integraded document review comments (#29) * Integrated document review comments * Incorrect break for bulleted line * Bug in Jinja template where property would tab to much (#31) * Updates to documentation and galaxy configuration (#32) * Fix to README.rst, duplicate wording (#34) * Remove duplicate action plugins link (#35) * testing readme rendering (#36) Ported the README.rst file to README.md to meet collection requirements. * Corrected README to point to MD, and long tag removed (#39) * Add two additional arg types to BetterArgHandler (#44) * Add two additional arg types * Remove os from os.path.isabs Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Data Set Utility for Common Data Set Related Tasks (#48) * DataSetUtils initial commit * LISTDS and LISTCAT output processing * Raise exception when data set is open for write * Fix LISTDS regex * Fix indentation * Add USS path support * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/832/fix non vsam non sms existence checking zos data set (#49) * updated catalog handling for non-vsam, non-sms * updated some method comment structures * bugfix/822/determine-data-set-type (#50) * Use mvscmd CLI instead of Python API * Fix linting errors * Remove unused variable Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Extend dynamic function support to the better arg parser (#51) * Dynamic argument support for dynamic functions. * improve default handling for reusability of BetterArgParser object. * add test cases for dynamic argument support * removed print statements from test cases * Updates to data-set-utils exceptions (#52) * Avoid raising exception for uncatalogued data sets * verify listcat output for uncatalogued datasets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Removed return block from documentation (#53) Removed message and changed, empty return block now * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * Fixed PATH (#55) * Update/973/add new supported data set characters betterargparser (#56) * updated valid data set types builtin type name fix * added test cases for data set types * Bugfix when checking if data set exists (#57) * Return False instead of None when data set doesn't exist * Test for membership should ne 'not in' Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Committing module zos_operator_action_query (#43) * Reviewd changes for feature zos_operator_action_query.py * Fix linting errors reported in PR * Linting corrections * fix the syntax error * fix lint errors * fix lint error * fix lint error * fix lint error * fix function test * Added zos_operator_action_query to the sanity exemption list for lack of GPLV3 Lic * Update author tag * Update the collection reference * add function test(need revise later) & doc update * updated regex and value capturing * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * updated docs, made environment agnostic test cases * added support for "*" and fixed splat handling * updated test cases * updated regex matching method, updates tests * Update short description Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Check for data set member existence (#58) * Check for data set member existence * Resolve linting issue Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix for job name capture group (#60) * Better ZOAU import missing messages (#61) * Better ZOAU import missing messages * added missing license and imports * Changed behavior on failure for zosoperator_action_query (#63) * changed behavior on failure * update tests for new expected output * Bugfix 1661 & 1704 fix query of job for different user zos_job_submit and zos_job_action_query (#64) * updates to fix job submission as other user * updated and optimized job utility * removed usage of ZOAU and updated jobs utility usage * update job_output return structure * improved error messages * add job log cleanup for test cases * Initial module zos_tso_command (#46) * Linting corrections * Update doc so html will generate * Lintig updates * Lintig updates v2 * Lintig updates v3 * Lintig updates v4 * Lintig updates v5 * Lintig updates v6 * Lintig updates v7 * Lintig updates v8 * Delete test_zos_tso_command_unit.py Remove the unit test as it only mocks things. * Documenation updates * Update errro message * zos-tso-command update * linting update * testcase linting updates * linting update * linting update * linting update * linting update * linting updates * linting update * new authorized long command handling method * linting update * test cases update * linting update * linting update * linting update * tso-command new version * tso-command linting update * linting update * linting update * linting update' * linting removal * lint removal * linting testcases Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> * Update module_utils with an encode utility (#59) * initial commmit for encode_utils * support for string encoding conversion * Support MVS data set conversion and string conversion modification * Recursively copy to PDS fix * Linting correction * Linting updates * Linting updates 01 * fix ENOENT problem after shutil.move * updates with zos_encode * Merged branch and commit for zos_encode (#65) * Merged branch and commit for zos_encode * Lint error , trailing white space * z/OS Fetch Module (#7) * initial commit * begin updating examples * zos_fetch module * return rc, stdout, stderr as separate return parameters on failure * untrack non-fetch related files * add rest of the files * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * added unit and functional tests * check file existence when LISTDS returns invalid dataset name * added 'wait_s' parameter * clarified description of encoding parameter * make use_qaulifer false by default * remove is_uss and os_vsam parameters * remove data_set directory * remove is_uss parameter from examples * message return block * Fix YAML formatting * return code formatting * remove fail_on_missing from action plugin * fail if USS file does not exist * data sets are assumed to be cataloged * instead of entire dict, pass individual values * redundant code when fetching data set member * rename transfer_from_uss to fetch_non_partitioned_data_set * remove test cases for uncataloged datasets * import naming conflict * add __future__ imports * add __future__ imports * assuming all data sets are cataloged * assuming all data sets are cataloged * fix formatting issues * fix formatting issues * remove blank lines with whitespaces * change example member name data to DATA for accuracy. * continuation line unindented for visual indent * conform to PEP8 standard * avoid automatic field numbering for format * specify python interpreter * extract only data set name if member present * Remove newline character from error message * Update RETURN block with sample message * Remove blank line from return block * Update version_added to correspond to Ansible version * Check if temp datasets are being created successfully * Datasets.read() returns None for empty data sets * Use BetterArgParser to parse module args * Added parameter types * Remove trailing whitespaces * Avoid passing extra args from action plugin * Remove unused imports * Ignore lack of GPLv3 license complaint * catch ValueError thrown ny BetterArgParser * Blank line contains whitespace * Reduce line lengths to 80 * Add white space around string addition * Checking volume type no longer required * Check if src or dest is provided first * Fix indentation * Indent multiline string for better readability * Use DataSetUtils to get data set type * Use DataSetUtils to determine data set existence * Remove unit tests * Fix linting * Add test case for empty data sets * default params for _fail_json * Test case for missing source * Test case for insufficient write permission * Test case for insufficient desgt write permission * Catch exceptions while executing module * Use updated DataSetutils constructor * Updated functional tests * Fix linting errors * Use ansible fetch module to fetch USS files * Checking for USS type is unnecessary * Use sha256 instead of sha1 for checksum validation * Remove message return block * Missing uss file test case * Minor cleanup * Add stdout_lines and stderr_lines to module response * Update action plugin with stderr_lines and stdout_lines * YAML syntax issue * Corrected spelling errors * Inaccurate description of flat * No need to wrap error messages returned by DataSetUtils * Update doc regarding HFS and ZFS support * Handle encoding for USS files * Add encoding to doc * Calculate temp file size for VSAM storage * Preserve encoding using SFTP * Resolve PEP8 issues * Check for data set member existence * raw binary not supported in PY2 * Fix linting issues * Change doc to reflect USS absolute path * Prepend dataset hlq before parameter verification * src not declared before being referenced * Add test case using use_qualifier * Updated doc link for fetch L(..) and suboptions * Use tempfile.mkstemp() for temp data storage * Remove default for encoding * default encoding values are None * Create destination dirs when flat is false * Add test case for flat hostname * Fix spelling * exist_ok is not valid keyword for PY2 * Detect read-only file system * Handle sftp errors * Capture module_stderr in case of module failure * Ensure dest is a directory for PDS transfer * Ensure dest is a directory except when fetching a member * Fix data set member destination path * Default values for encoding is not needed * Remove unused imports * Update 'msg' description * Clean up temp files in case of module failure * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * enhanced zoau import error message (#66) * enhanced zoau import error message * satisfy pep8 witespace requirement * Added new playbook for latest module collection and a filter * fixed linting issues, added license * updated spacing to conform to formatting standards * updated shell to command where possible * Use encode util to convert encoding (#68) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update RST and MD doc files to reflect next release * Update group vars to reflect zoau 1.0.2 new env vars * Update encode short description to fit within the limits of doc length * Update makefile for more generic use by others * Update to fix RST lint error , short line * Rename sub-dir from plugin to filter, doc error * doc tree correction * Update release notes to correspond to beta release tag * Update sample playbooks with doc, configuration and new variables * Upate docs to highlight users to review sample playbooks and JCL before running * Update sample JCL to run on my systems with default values and notes * Add supporting doc for behavioral inventory parameters and ansible.cfg * Fix the md style URL in the index RST file * Update short descriptions for modules * Fix README.md link to IMS collection * Fix README.md link to collection artifacts * Linting updates * Add an example that requires a command to be escaped (#71) * refine backup_file & put it into encode_utils (#70) * refine backup_file & put it into encode_utils * correction for sanity test * refactor for security and usability, fix test case * move copy logic to utility * refactor zos_encode, improve backup functionality Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Update galaxy.yml to reflect the latest version and authors * Fix import name change resulting from refactor * updated usage of encode_utils to encode * Documentation updates, corrections and alterations (#75) * Documentation updates, corrections and alterations * ID review-round 2 * Update the user description in the doc Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Content/81/zos tso command docreview (#73) * ID review of zos-tso-commands * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * Fix indentation Co-authored-by: ddimatos <dimatos@gmail.com> * Fix extra space in doc * fixed zos_ping action plugin issues (#77) * Add out-of-box awx support (#80) * Added encoding supplemental doc (#78) * Added encoding supplemental doc * Removed references to modules not available and causing doc warnings * Fix Inline literal start-string without end-string. * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Add binary support copying across USS and MVS * Copy USS file in binary mode * fixed bug and cleaned up code (#82) * fixed bug and cleaned up code * fixed linting issues * Avoid specifying -B and -F for cp * Copy MVS source to MVS dest * Documentation web content updates (#84) * First round of review of static docs * docs/source/installation.rst * Update block to rendor with syntax highlights Code block was using 'yaml' when it was 'sh' , renders better if we have the correct code block type. * Incorporated Demetri's review comments Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Replace only the first occurance of 'rec' * Minor updates in install,quickstart, and req pages (#86) * Updated make file with comments and recover from failure * Update Backup util (mvs_file_backup) (#85) * Alternative mvs_file_backup * fix backup logic, add encode backup tests * add mvs backup tests, util fixes * fix test for new logic path Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Udpate the restructured text to reflect version release (#89) * minor fixes and additional tests for zos_encode (#88) * removed Jobs.submit usage due to invalid response (#92) * Update to galaxy.yml to exclude files * Update makefile to support rexx written modules * Generate and push zos_ping rst doc * Corrected clean up of a directory in makefile * Fixes spacing in RETURN values and formatting (#97) * Validate bk_dsn after assigning a value * Return backup path * Fix alloc command * Remove trailing whitespace * Fix job utility to work on all z/OS systems (#99) * Shinx and sphinx-versioning support and updates (#100) * Updated Makefile to support sphinx-versioning and renamed some of the operations * Update configuration to support sphinx-versioning and update options for sphinx * Update configuration with linting corrections * Correct and update the installation of a pre-release (#102) * Correct and update the installation of a pre-release * minor fixes to docs Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update data set utils (#104) * Update data set utils * Update function signatures * Module parameter not needed for encode util Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Change default encoding from UTF-8 to ISO8859-1 * Fetching non-existent VSAM * Update doc for encoding parameter * Update encoding to handle different charsets * Remove encoding default from argument spec * encoding validation and doc update * Fix typo * Bugfix/case conversion fix zos tso command (#108) * fixed issues with commands forced to uppercase * add support for single or multi commands, added relevant test cases * updated alias doc * Added index file to support shared doc site (#109) * Remove module parameter from EncodeUtils constructor * fix issue when no default encoding specified (#112) * fix issue when no default encoding specified * fix local job submission * Feature/26/ansible zos copy module (#105) * initial commit * begin updating examples * default value 'yes/no' -> 'true/false' * updated documentation * copy to seq data set * copy local to seq data set * add data set size to module response * functional test cases for copy * test case for sequential data set * clean up copy to ps function * copy to pdse * assuming all data sets are cataloged * copy to PDS * Copy to VSAM * Copy to PDS(E) member * Actual directory size instead of max file size * Copy from remote source * Update copy * Reduce line lengths to less than 80 * Use mvscmd cli * remote pdse copy * IEBCOPY to copy data sets * saving changes * Create remote data set if it doesn't exist * Reorganize remote copy transfer * Update return params * Avoid using global module * Remove unused imports * Copy to USS * Clean up in case of module failure * Back up USS file * Copy content to remote file * Copy to sequential data set * Copy local dir to PDS * Copy remote PDS to PDS * Update backup file * USS dest not accessible if not present * Explicitly tag destination file * Changed file mode, group and owner * Change common file args for remote src * Change file common args after copying to dest * Preserve local file mode * Convert encoding for remote src * Added comments * Use backup util to back up files and data sets * Add validate parameter * Copy VSAM to VSAM * Support for symlinks * Copy PDS/PDSE member to USS * Fix formatting * Module should fail for empty VSAM * Added function signatures for all test cases * User better_arg_parser to determine member type * Add binary support for copying to USS target * Copy to PS in binary mode * Correct object instantiations * Import copy util * Remove write_rc * Resolve import issues * Move vsam copy handler to main copy handler class * Copy local dir to USS dir * Infer member name when it isn't provided * Added test cases for local and remote copy * src_member evalutes to True * Comments for extract_dsname() * IEBCOPY fails for incompatible recfm * Copy PDS member to PS * Copy data set member to non-existent PS * Copy local symlink * Add test case for uss file backup * Fix mvs file backup * Functional tests complete * Fix linting issues * Remove trailing whitespace * Ignore GPLv3 license * Remove is_vsam parameter * Fix license information * Fail action plugin * Add test cases to ensure module failure * Remove unneded parameters * Refactor to use updated data_set util * I/O abend when copying PDSE member to non-existent PS * Updated doc, comments, linting * Fix linting issues * Remove duplicate key from dic * Avoid hard-coding tmp path name * Change backup_path parameter to backup_file * Copy to data set member partial * Error converting encoding for local dir * Fix tmp cleanup bug * Updated test cases for backup_file * plugins/modules/zos_copy.py * Reactor code for better readability * Fix pep8 issue * Cleanup error on module failure * plugins/modules/zos_copy.py * Doc update mode parameter * Use native shell instead of bash for functional testing * Return invocation parameter * Remove unnecessary imports * plugins/modules/zos_copy.py * Return backup_file on module failure Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Feature/1089/zos lineinfile (#111) * initial version for interface and options * add backup option * update lineinfile * make golden path work * fix the bugs so that method could support none vars * refactor based on new dsed * updated params and return * Initial check in for backup feature support * support all charsets; updated backup feature doc * clean up; used better_arg_parser * use latest data_set module_util * change the returned string to be loadable by json.loads (BOF/EOF) * uss test cases * dataset test cases; create a dataset/file with content instead of being dependent on existing source * uss test env set bug fix * added UTF-8 in test cases; simplified dataset names * use new dedicated ZOAU function for lineinfile * added assert for expected strings * encoding parameter from absent * Linting updates from pipeline tests * Linting updates and corrections * Linting udpates continued * Update sanity exemption file and linting corrections * Doc error, indentation off, breaking rest of doc parse * module arg and doc updates * Formatting updates * Further doc updates * regexp alias fix * updated docs; added comments; updated parameters * fix sanity checks * fix RETURN key type; lint error * code review changes * fixed linting issues * removed incorrectly added choices for insertafter/insertbefore * added negative test cases fo not supported datasets * moved choices to description; fixed test cases lint issue * changed the choices format in description * add escape if string was quoted * added test cases for escaped/unescaped quotation * used 'src' for target location; moved the params checking before taking backup * fixed indentation * Doc edits for lineinfile, encode, and fetch * remove relative import method * removed the default values for insertafter/insertbefore * renamed helper functions; updated pytest.ini for test cases; fixed TEST_INFO * remove jcl tmp directory if it already exists * run all test_* Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: behnam-ibm <behnam@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/1067/community doc (#113) * Correct doc link to IMS collection to point to Galaxy * Added testing doc to community guidlines * Clean/delete accidental file * Fix Sphinx warning coming from ping module reference to zos_ssh plugin undefined label: zos_ssh_module * Added better arg parse to new dev addition to community guides * Fixes broken site anchor to the YAML configuration * Fix MD style Link * Remove additional MD style links * Fix mailformed RST link * Fix more RST links * Python syntax fixes in samples * Update betterArgParse to BetterArgParser * Expand pipelining=true doc * Correct Unicode quotes to ASCII quotes * Statement of direction on zoau and python * doc review of community guidelines * Doc review of Community guidelines * Fix malformed RST table * Update group vars requirements needed by IBM Pypthon * Update doc with var settings and sample all.yml * Update environ vars * Added environ LANG doc * Added doc to clarify shell support * Added advanced group_vars sample and doc * Update docs to reflect IBM Python announcement Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Remove unnecessary description of PDSE transfer * Update version tag in doc generation conf * Add missing link reference * Doc review (#118) Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update IBM Python download link * Add system info to job query (#127) * Add system info to job query * Change sysname to system for job query Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Close file descriptor (#128) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/2298/zos raw (#120) * add utilities for zos_raw * updated DD statement to better handle concatenations * update parameter names * fix concat naming * refactor of zos_data_set, add new features * updated examples * added steplib dd type * fix steplib data set * update valid concat members * fix datasetdefinition for steplibs * update dd statement types * argument parsing working in zos_raw * module argument parsing updates * updated valid arguments for consistency * update tests for new zos_data_set format * zos_raw module working * module doc updates * updated function documentation * backup, reuse and replace working * updated documentation * cleaned up wrapper classes * updated existence checking, function doc updates * spelling fixes, added new parameters for dd_unix * add list of string support for StdinDefinition * properly handle space units * initial set of test cases, added aliases for program, rename args to parms to match jcl naming conventions * added remaining happy path data set DD tests, added input dd tests. Fixed small bug in raw related to invalid parameter options * update example playbooks to account for new zos_data_set arguments * added initial functional tests for unix file dd statements * added access group support, fixed status group * added remaining tests for dd_unix and dd_dummy * fixed bug on dd_concat contents or backup returned * add dd_concat test cases * update parms to parm to match jcl naming * added execution tests for zos_raw * add final test cases * add special type functions for ksds related params * update import handling for special cases * add VIO DD support * uncomment test cases * fix basic linting issues, default arguments * add support for directory blocks * fix some linting issues * Revert "fix some linting issues" This reverts commit 44d0e14. * linting fixes * module doc and linting updates * additional doc fixes for linting sanity tests * added examples * added catch for nonexistent deletion * updated docs for proper formatting * update ignore files for new module * added dd_output support, standardized dd names * removed unnecessary import wrappers * improved validation for encoding types. * added new example with dd_output type * Merge branch 'origin/dev' * Revert "Merge branch 'origin/dev'" This reverts commit 1befb8e. * fixes for python 2 controller support * modify encode utils import * add utf-8 encoding tag to unit tests * improve check mode support * fix mock of AnsibleModule for encoding check cases * fix unit test formatting for pep8 * fix test case helpers * comment out failing test case unrelated to zos_raw * update encoding conversion handling * update rexx variables to capture correct environment variables * update naming based on discussion * rename test case files, update test cases * update default for output data set * update test ignore files for rename * doc edits (#133) * doc edits * fix minor pep8 issues Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * add python 2 support * remove redundant code comments * add temp data set support, backwards compatibility fix with old interface, update docs and tests * update documentation for return_content * minor doc changes for clarity * improve documentation clarity * fix incorrect syntax for option name Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Update configuration vars in doc and yml (#131) * remove ZOAU Python API deps for zos_operator (#132) * remove ZOAU deps which break in 1.0.3 * add back shebang * remove unsafe shell usage * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/2516/release doc update (#136) * Update the version in galaxy.yml * edits to support doc gen * Updated for doc gen * updated all module doc for release * Update template to upper case boolean when no default present * update raw module doc to fomat as the indentation intended * updated module RST * add details to module enhancements * Update message around Rocket's python * Update Rocket Python Message * Reference Rocket Python configuration in requirements * Updated doc to reflect the latest dependencies * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Remove see also reference to external collections * updated zos_copy module RST * Sample playbook for latest version * Updated ZOAU PTF information with the latest PTF UI70435 * Updates to raw doc to handle the error - Definition list ends without a blank line * Update copy module docs * Update and add clarity around Rocket Python * Requirements order change Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update conf for v1.2.0-beta.1 * Fix E231: missing whitespace (#138) * Specify SFTP port for fetch and copy (#141) * Add sftp_port parameter to fetch and copy * Add sftp_port parameter to fetch and copy * Fix spelling * Missing positional arguments * Test case for negative port specification * Check for valid port range * Remove redundant assertions Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/dependency discovery (#142) * add host setup playbook * handle error cases * restructure and improve robustness of host-setup * add max_search_depth variable * change file extension for consistency * add documentation for host-setup.yaml * add in-file docs for tasks files * add important note for task files * Remove redundant data_set_util * Move error classes to the bottom * Rename backup_file to backup_name (#144) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/zos_copy record format (#147) * Update uss2mvs copy format * Default PS allocation should be FB * Use ZOAU Copy for sequential data sets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/148/accept all valid data set name characters (#149) * update regex for data set name * add test case for special characters * Add ignore_sftp_stderr parameter for fetch and copy * improve cleanup (#151) * change file extension * add /usr/sbin to default PATH * Use USS cp command to copy to PS * Bugfix: zos operator action query not returning all outstanding messages (#154) * remove ZOAU python API usage * add back ZOAU python API, update parsing regex * Changes to support single source doc site (#155) * Fix E231: missing whitespace * Changes to support single source doc site * Under indentation * Under indentation 2 * doc review * doc review of additional files * Update the shell scripts to create a single requirements RST Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * add verbose support and steplib fix (#156) * udpate maxdepth for RST to 1 * Content/3397/update docs (#158) * Fix E231: missing whitespace * Changes to support single source doc site (#145) * Changes to support single source doc site * Under indentation * Under indentation 2 * doc review * doc review of additional files * Update the shell scripts to create a single requirements RST Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Module known issues and release notes added * Updated doc and sample host config playbook doc * Fix use ASCII quotes instead of Unicode quotes * use ASCII quotes instead of Unicode quotes Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update galaxy to latest doc site and version * add FQCN to action plugins (#167) * add FQCN to action plugins * fix pep8 * Fixed release notes merge comment * Galaxy and release notes update to reflect bugfix Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Jake Jackson <jljacks93@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> Co-authored-by: zlbjlu <16587467+yourfuwa2015@users.noreply.github.com> Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: balkajbaf <62961336+balkajbaf@users.noreply.github.com> Co-authored-by: behnam-ibm <behnam@ibm.com>
ddimatos
added a commit
that referenced
this pull request
Aug 18, 2020
* Release v1.2.0 beta.2 (#163) * Initial collection commit for v0.0.1 (#1) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * added arg parser to module_utils (#3) * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * Updated job output for improved DD handling (#14) Fixes the job summary output being returned when return_output=false. * Security module updates (#15) * patched security holes * updated test cases * removed unnecessary import * updated execution bits * Update modules and documentation corrections (#17) * update documentation, remove unused imports * updated requirements * updated module imports, formatting, and spelling * doc update and formatting update * z/OS Ping Module Feature Delivered (#18) * added zos_ping, test, and test fixture update * Description completeness * Grammar completeness * Pulled repetitive code into functions * updated module docs * Minor spelling corrections * Repo updates to satisfy ansible-test sanity testing requirements (#19) * update formatting * formatting changes to appease ansible-test * update tests * fix string format use and quotes * update README * remove trailing whitespace * update group_vars trailing spaces * remove shebang in attempt to fix encoding issue * add back shebang lines * Added security updates and test cases (#20) * add new types for parser, update arg handling zos_job_submit * updated module argument parsing and security tests * uncommented security tests * updated README to reflect use of MIT license * update hash function * test and formatting updates * updated documentation to pass ansible-test * switch name back to normal.py, add appropriate ignores * update ignore files * added ignore for GPL * updated README, .gitignore and removed MIT code * remove broken tests * increase robustness of zos_job_query * make job query even more robust * Webdocs content for IBM z/OS Core Collection (#21) * Initial commit for reStructuredText content * Updates and clean up to the template * Updated grammar * Grammar edit * Grammar fixes * Text updates * Updates to restructured text, fixed link, renamed collection * Fix ibm_zos_core bold * Fixes ansible-test lint: docs/source/conf.py:51:161: E501: line too long (178 > 160 characters) * ansible doc changes to modules for better formatting * remove original_message return parameter (#23) * remove original_message return parameter * Fix reported linting errors from PR * Fix reported linting errors from PR * Fix reported linting errors from PR for zos_job_query.py * Module document updates, Makefile changes and jinja support for defau… (#24) * Module document updates, Makefile changes and jinja support for default:int * Doc updates per linting rules * Update mmodule doc and remove md files (#25) * Removed all MD files from docs directory * Remove README.md , its replaced by README.rst * Fix bolding for ibm_zos_core * Fix highlight for ibm_zos_core * Updated index.rst to correspond to the README.md updates * Updated README.rst to point to webdocs * Updated README.rst webdocs URL * Update the modules to their ansible-doc * Documentation updates (#26) * Updated galaxy.yml to reflect latest version 1.0 * Documentation copyright updates and formatting * Documentation udpates * Update z/OS data set module and add catalog functionality (#27) * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * catalog for VSAM working, bugs fixed, tests added * satisfy ansible-test requirements, minor arg validation updates * removed unit tests * raise min . ansible requirement due to security vulnerability * added documentation to VTOC * Update product name in webdocs (#28) * Update product name in webdocs * Fix trailing newline * Document updates include some template bugs * Integraded document review comments (#29) * Integrated document review comments * Incorrect break for bulleted line * Bug in Jinja template where property would tab to much (#31) * Updates to documentation and galaxy configuration (#32) * Fix to README.rst, duplicate wording (#34) * Remove duplicate action plugins link (#35) * testing readme rendering (#36) Ported the README.rst file to README.md to meet collection requirements. * Corrected README to point to MD, and long tag removed (#39) * Add two additional arg types to BetterArgHandler (#44) * Add two additional arg types * Remove os from os.path.isabs Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Data Set Utility for Common Data Set Related Tasks (#48) * DataSetUtils initial commit * LISTDS and LISTCAT output processing * Raise exception when data set is open for write * Fix LISTDS regex * Fix indentation * Add USS path support * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/832/fix non vsam non sms existence checking zos data set (#49) * updated catalog handling for non-vsam, non-sms * updated some method comment structures * bugfix/822/determine-data-set-type (#50) * Use mvscmd CLI instead of Python API * Fix linting errors * Remove unused variable Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Extend dynamic function support to the better arg parser (#51) * Dynamic argument support for dynamic functions. * improve default handling for reusability of BetterArgParser object. * add test cases for dynamic argument support * removed print statements from test cases * Updates to data-set-utils exceptions (#52) * Avoid raising exception for uncatalogued data sets * verify listcat output for uncatalogued datasets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Removed return block from documentation (#53) Removed message and changed, empty return block now * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * Fixed PATH (#55) * Update/973/add new supported data set characters betterargparser (#56) * updated valid data set types builtin type name fix * added test cases for data set types * Bugfix when checking if data set exists (#57) * Return False instead of None when data set doesn't exist * Test for membership should ne 'not in' Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Committing module zos_operator_action_query (#43) * Reviewd changes for feature zos_operator_action_query.py * Fix linting errors reported in PR * Linting corrections * fix the syntax error * fix lint errors * fix lint error * fix lint error * fix lint error * fix function test * Added zos_operator_action_query to the sanity exemption list for lack of GPLV3 Lic * Update author tag * Update the collection reference * add function test(need revise later) & doc update * updated regex and value capturing * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * updated docs, made environment agnostic test cases * added support for "*" and fixed splat handling * updated test cases * updated regex matching method, updates tests * Update short description Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Check for data set member existence (#58) * Check for data set member existence * Resolve linting issue Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix for job name capture group (#60) * Better ZOAU import missing messages (#61) * Better ZOAU import missing messages * added missing license and imports * Changed behavior on failure for zosoperator_action_query (#63) * changed behavior on failure * update tests for new expected output * Bugfix 1661 & 1704 fix query of job for different user zos_job_submit and zos_job_action_query (#64) * updates to fix job submission as other user * updated and optimized job utility * removed usage of ZOAU and updated jobs utility usage * update job_output return structure * improved error messages * add job log cleanup for test cases * Initial module zos_tso_command (#46) * Linting corrections * Update doc so html will generate * Lintig updates * Lintig updates v2 * Lintig updates v3 * Lintig updates v4 * Lintig updates v5 * Lintig updates v6 * Lintig updates v7 * Lintig updates v8 * Delete test_zos_tso_command_unit.py Remove the unit test as it only mocks things. * Documenation updates * Update errro message * zos-tso-command update * linting update * testcase linting updates * linting update * linting update * linting update * linting update * linting updates * linting update * new authorized long command handling method * linting update * test cases update * linting update * linting update * linting update * tso-command new version * tso-command linting update * linting update * linting update * linting update' * linting removal * lint removal * linting testcases Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> * Update module_utils with an encode utility (#59) * initial commmit for encode_utils * support for string encoding conversion * Support MVS data set conversion and string conversion modification * Recursively copy to PDS fix * Linting correction * Linting updates * Linting updates 01 * fix ENOENT problem after shutil.move * updates with zos_encode * Merged branch and commit for zos_encode (#65) * Merged branch and commit for zos_encode * Lint error , trailing white space * z/OS Fetch Module (#7) * initial commit * begin updating examples * zos_fetch module * return rc, stdout, stderr as separate return parameters on failure * untrack non-fetch related files * add rest of the files * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * added unit and functional tests * check file existence when LISTDS returns invalid dataset name * added 'wait_s' parameter * clarified description of encoding parameter * make use_qaulifer false by default * remove is_uss and os_vsam parameters * remove data_set directory * remove is_uss parameter from examples * message return block * Fix YAML formatting * return code formatting * remove fail_on_missing from action plugin * fail if USS file does not exist * data sets are assumed to be cataloged * instead of entire dict, pass individual values * redundant code when fetching data set member * rename transfer_from_uss to fetch_non_partitioned_data_set * remove test cases for uncataloged datasets * import naming conflict * add __future__ imports * add __future__ imports * assuming all data sets are cataloged * assuming all data sets are cataloged * fix formatting issues * fix formatting issues * remove blank lines with whitespaces * change example member name data to DATA for accuracy. * continuation line unindented for visual indent * conform to PEP8 standard * avoid automatic field numbering for format * specify python interpreter * extract only data set name if member present * Remove newline character from error message * Update RETURN block with sample message * Remove blank line from return block * Update version_added to correspond to Ansible version * Check if temp datasets are being created successfully * Datasets.read() returns None for empty data sets * Use BetterArgParser to parse module args * Added parameter types * Remove trailing whitespaces * Avoid passing extra args from action plugin * Remove unused imports * Ignore lack of GPLv3 license complaint * catch ValueError thrown ny BetterArgParser * Blank line contains whitespace * Reduce line lengths to 80 * Add white space around string addition * Checking volume type no longer required * Check if src or dest is provided first * Fix indentation * Indent multiline string for better readability * Use DataSetUtils to get data set type * Use DataSetUtils to determine data set existence * Remove unit tests * Fix linting * Add test case for empty data sets * default params for _fail_json * Test case for missing source * Test case for insufficient write permission * Test case for insufficient desgt write permission * Catch exceptions while executing module * Use updated DataSetutils constructor * Updated functional tests * Fix linting errors * Use ansible fetch module to fetch USS files * Checking for USS type is unnecessary * Use sha256 instead of sha1 for checksum validation * Remove message return block * Missing uss file test case * Minor cleanup * Add stdout_lines and stderr_lines to module response * Update action plugin with stderr_lines and stdout_lines * YAML syntax issue * Corrected spelling errors * Inaccurate description of flat * No need to wrap error messages returned by DataSetUtils * Update doc regarding HFS and ZFS support * Handle encoding for USS files * Add encoding to doc * Calculate temp file size for VSAM storage * Preserve encoding using SFTP * Resolve PEP8 issues * Check for data set member existence * raw binary not supported in PY2 * Fix linting issues * Change doc to reflect USS absolute path * Prepend dataset hlq before parameter verification * src not declared before being referenced * Add test case using use_qualifier * Updated doc link for fetch L(..) and suboptions * Use tempfile.mkstemp() for temp data storage * Remove default for encoding * default encoding values are None * Create destination dirs when flat is false * Add test case for flat hostname * Fix spelling * exist_ok is not valid keyword for PY2 * Detect read-only file system * Handle sftp errors * Capture module_stderr in case of module failure * Ensure dest is a directory for PDS transfer * Ensure dest is a directory except when fetching a member * Fix data set member destination path * Default values for encoding is not needed * Remove unused imports * Update 'msg' description * Clean up temp files in case of module failure * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * enhanced zoau import error message (#66) * enhanced zoau import error message * satisfy pep8 witespace requirement * Added new playbook for latest module collection and a filter * fixed linting issues, added license * updated spacing to conform to formatting standards * updated shell to command where possible * Use encode util to convert encoding (#68) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update RST and MD doc files to reflect next release * Update group vars to reflect zoau 1.0.2 new env vars * Update encode short description to fit within the limits of doc length * Update makefile for more generic use by others * Update to fix RST lint error , short line * Rename sub-dir from plugin to filter, doc error * doc tree correction * Update release notes to correspond to beta release tag * Update sample playbooks with doc, configuration and new variables * Upate docs to highlight users to review sample playbooks and JCL before running * Update sample JCL to run on my systems with default values and notes * Add supporting doc for behavioral inventory parameters and ansible.cfg * Fix the md style URL in the index RST file * Update short descriptions for modules * Fix README.md link to IMS collection * Fix README.md link to collection artifacts * Linting updates * Add an example that requires a command to be escaped (#71) * refine backup_file & put it into encode_utils (#70) * refine backup_file & put it into encode_utils * correction for sanity test * refactor for security and usability, fix test case * move copy logic to utility * refactor zos_encode, improve backup functionality Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Update galaxy.yml to reflect the latest version and authors * Fix import name change resulting from refactor * updated usage of encode_utils to encode * Documentation updates, corrections and alterations (#75) * Documentation updates, corrections and alterations * ID review-round 2 * Update the user description in the doc Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Content/81/zos tso command docreview (#73) * ID review of zos-tso-commands * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * Fix indentation Co-authored-by: ddimatos <dimatos@gmail.com> * Fix extra space in doc * fixed zos_ping action plugin issues (#77) * Add out-of-box awx support (#80) * Added encoding supplemental doc (#78) * Added encoding supplemental doc * Removed references to modules not available and causing doc warnings * Fix Inline literal start-string without end-string. * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Add binary support copying across USS and MVS * Copy USS file in binary mode * fixed bug and cleaned up code (#82) * fixed bug and cleaned up code * fixed linting issues * Avoid specifying -B and -F for cp * Copy MVS source to MVS dest * Documentation web content updates (#84) * First round of review of static docs * docs/source/installation.rst * Update block to rendor with syntax highlights Code block was using 'yaml' when it was 'sh' , renders better if we have the correct code block type. * Incorporated Demetri's review comments Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Replace only the first occurance of 'rec' * Minor updates in install,quickstart, and req pages (#86) * Updated make file with comments and recover from failure * Update Backup util (mvs_file_backup) (#85) * Alternative mvs_file_backup * fix backup logic, add encode backup tests * add mvs backup tests, util fixes * fix test for new logic path Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Udpate the restructured text to reflect version release (#89) * minor fixes and additional tests for zos_encode (#88) * removed Jobs.submit usage due to invalid response (#92) * Update to galaxy.yml to exclude files * Update makefile to support rexx written modules * Generate and push zos_ping rst doc * Corrected clean up of a directory in makefile * Fixes spacing in RETURN values and formatting (#97) * Validate bk_dsn after assigning a value * Return backup path * Fix alloc command * Remove trailing whitespace * Fix job utility to work on all z/OS systems (#99) * Shinx and sphinx-versioning support and updates (#100) * Updated Makefile to support sphinx-versioning and renamed some of the operations * Update configuration to support sphinx-versioning and update options for sphinx * Update configuration with linting corrections * Correct and update the installation of a pre-release (#102) * Correct and update the installation of a pre-release * minor fixes to docs Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update data set utils (#104) * Update data set utils * Update function signatures * Module parameter not needed for encode util Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Change default encoding from UTF-8 to ISO8859-1 * Fetching non-existent VSAM * Update doc for encoding parameter * Update encoding to handle different charsets * Remove encoding default from argument spec * encoding validation and doc update * Fix typo * Bugfix/case conversion fix zos tso command (#108) * fixed issues with commands forced to uppercase * add support for single or multi commands, added relevant test cases * updated alias doc * Added index file to support shared doc site (#109) * Remove module parameter from EncodeUtils constructor * fix issue when no default encoding specified (#112) * fix issue when no default encoding specified * fix local job submission * Feature/26/ansible zos copy module (#105) * initial commit * begin updating examples * default value 'yes/no' -> 'true/false' * updated documentation * copy to seq data set * copy local to seq data set * add data set size to module response * functional test cases for copy * test case for sequential data set * clean up copy to ps function * copy to pdse * assuming all data sets are cataloged * copy to PDS * Copy to VSAM * Copy to PDS(E) member * Actual directory size instead of max file size * Copy from remote source * Update copy * Reduce line lengths to less than 80 * Use mvscmd cli * remote pdse copy * IEBCOPY to copy data sets * saving changes * Create remote data set if it doesn't exist * Reorganize remote copy transfer * Update return params * Avoid using global module * Remove unused imports * Copy to USS * Clean up in case of module failure * Back up USS file * Copy content to remote file * Copy to sequential data set * Copy local dir to PDS * Copy remote PDS to PDS * Update backup file * USS dest not accessible if not present * Explicitly tag destination file * Changed file mode, group and owner * Change common file args for remote src * Change file common args after copying to dest * Preserve local file mode * Convert encoding for remote src * Added comments * Use backup util to back up files and data sets * Add validate parameter * Copy VSAM to VSAM * Support for symlinks * Copy PDS/PDSE member to USS * Fix formatting * Module should fail for empty VSAM * Added function signatures for all test cases * User better_arg_parser to determine member type * Add binary support for copying to USS target * Copy to PS in binary mode * Correct object instantiations * Import copy util * Remove write_rc * Resolve import issues * Move vsam copy handler to main copy handler class * Copy local dir to USS dir * Infer member name when it isn't provided * Added test cases for local and remote copy * src_member evalutes to True * Comments for extract_dsname() * IEBCOPY fails for incompatible recfm * Copy PDS member to PS * Copy data set member to non-existent PS * Copy local symlink * Add test case for uss file backup * Fix mvs file backup * Functional tests complete * Fix linting issues * Remove trailing whitespace * Ignore GPLv3 license * Remove is_vsam parameter * Fix license information * Fail action plugin * Add test cases to ensure module failure * Remove unneded parameters * Refactor to use updated data_set util * I/O abend when copying PDSE member to non-existent PS * Updated doc, comments, linting * Fix linting issues * Remove duplicate key from dic * Avoid hard-coding tmp path name * Change backup_path parameter to backup_file * Copy to data set member partial * Error converting encoding for local dir * Fix tmp cleanup bug * Updated test cases for backup_file * plugins/modules/zos_copy.py * Reactor code for better readability * Fix pep8 issue * Cleanup error on module failure * plugins/modules/zos_copy.py * Doc update mode parameter * Use native shell instead of bash for functional testing * Return invocation parameter * Remove unnecessary imports * plugins/modules/zos_copy.py * Return backup_file on module failure Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Feature/1089/zos lineinfile (#111) * initial version for interface and options * add backup option * update lineinfile * make golden path work * fix the bugs so that method could support none vars * refactor based on new dsed * updated params and return * Initial check in for backup feature support * support all charsets; updated backup feature doc * clean up; used better_arg_parser * use latest data_set module_util * change the returned string to be loadable by json.loads (BOF/EOF) * uss test cases * dataset test cases; create a dataset/file with content instead of being dependent on existing source * uss test env set bug fix * added UTF-8 in test cases; simplified dataset names * use new dedicated ZOAU function for lineinfile * added assert for expected strings * encoding parameter from absent * Linting updates from pipeline tests * Linting updates and corrections * Linting udpates continued * Update sanity exemption file and linting corrections * Doc error, indentation off, breaking rest of doc parse * module arg and doc updates * Formatting updates * Further doc updates * regexp alias fix * updated docs; added comments; updated parameters * fix sanity checks * fix RETURN key type; lint error * code review changes * fixed linting issues * removed incorrectly added choices for insertafter/insertbefore * added negative test cases fo not supported datasets * moved choices to description; fixed test cases lint issue * changed the choices format in description * add escape if string was quoted * added test cases for escaped/unescaped quotation * used 'src' for target location; moved the params checking before taking backup * fixed indentation * Doc edits for lineinfile, encode, and fetch * remove relative import method * removed the default values for insertafter/insertbefore * renamed helper functions; updated pytest.ini for test cases; fixed TEST_INFO * remove jcl tmp directory if it already exists * run all test_* Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: behnam-ibm <behnam@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/1067/community doc (#113) * Correct doc link to IMS collection to point to Galaxy * Added testing doc to community guidlines * Clean/delete accidental file * Fix Sphinx warning coming from ping module reference to zos_ssh plugin undefined label: zos_ssh_module * Added better arg parse to new dev addition to community guides * Fixes broken site anchor to the YAML configuration * Fix MD style Link * Remove additional MD style links * Fix mailformed RST link * Fix more RST links * Python syntax fixes in samples * Update betterArgParse to BetterArgParser * Expand pipelining=true doc * Correct Unicode quotes to ASCII quotes * Statement of direction on zoau and python * doc review of community guidelines * Doc review of Community guidelines * Fix malformed RST table * Update group vars requirements needed by IBM Pypthon * Update doc with var settings and sample all.yml * Update environ vars * Added environ LANG doc * Added doc to clarify shell support * Added advanced group_vars sample and doc * Update docs to reflect IBM Python announcement Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Remove unnecessary description of PDSE transfer * Update version tag in doc generation conf * Add missing link reference * Doc review (#118) Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update IBM Python download link * Add system info to job query (#127) * Add system info to job query * Change sysname to system for job query Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Close file descriptor (#128) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/2298/zos raw (#120) * add utilities for zos_raw * updated DD statement to better handle concatenations * update parameter names * fix concat naming * refactor of zos_data_set, add new features * updated examples * added steplib dd type * fix steplib data set * update valid concat members * fix datasetdefinition for steplibs * update dd statement types * argument parsing working in zos_raw * module argument parsing updates * updated valid arguments for consistency * update tests for new zos_data_set format * zos_raw module working * module doc updates * updated function documentation * backup, reuse and replace working * updated documentation * cleaned up wrapper classes * updated existence checking, function doc updates * spelling fixes, added new parameters for dd_unix * add list of string support for StdinDefinition * properly handle space units * initial set of test cases, added aliases for program, rename args to parms to match jcl naming conventions * added remaining happy path data set DD tests, added input dd tests. Fixed small bug in raw related to invalid parameter options * update example playbooks to account for new zos_data_set arguments * added initial functional tests for unix file dd statements * added access group support, fixed status group * added remaining tests for dd_unix and dd_dummy * fixed bug on dd_concat contents or backup returned * add dd_concat test cases * update parms to parm to match jcl naming * added execution tests for zos_raw * add final test cases * add special type functions for ksds related params * update import handling for special cases * add VIO DD support * uncomment test cases * fix basic linting issues, default arguments * add support for directory blocks * fix some linting issues * Revert "fix some linting issues" This reverts commit 44d0e14. * linting fixes * module doc and linting updates * additional doc fixes for linting sanity tests * added examples * added catch for nonexistent deletion * updated docs for proper formatting * update ignore files for new module * added dd_output support, standardized dd names * removed unnecessary import wrappers * improved validation for encoding types. * added new example with dd_output type * Merge branch 'origin/dev' * Revert "Merge branch 'origin/dev'" This reverts commit 1befb8e. * fixes for python 2 controller support * modify encode utils import * add utf-8 encoding tag to unit tests * improve check mode support * fix mock of AnsibleModule for encoding check cases * fix unit test formatting for pep8 * fix test case helpers * comment out failing test case unrelated to zos_raw * update encoding conversion handling * update rexx variables to capture correct environment variables * update naming based on discussion * rename test case files, update test cases * update default for output data set * update test ignore files for rename * doc edits (#133) * doc edits * fix minor pep8 issues Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * add python 2 support * remove redundant code comments * add temp data set support, backwards compatibility fix with old interface, update docs and tests * update documentation for return_content * minor doc changes for clarity * improve documentation clarity * fix incorrect syntax for option name Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Update configuration vars in doc and yml (#131) * remove ZOAU Python API deps for zos_operator (#132) * remove ZOAU deps which break in 1.0.3 * add back shebang * remove unsafe shell usage * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/2516/release doc update (#136) * Update the version in galaxy.yml * edits to support doc gen * Updated for doc gen * updated all module doc for release * Update template to upper case boolean when no default present * update raw module doc to fomat as the indentation intended * updated module RST * add details to module enhancements * Update message around Rocket's python * Update Rocket Python Message * Reference Rocket Python configuration in requirements * Updated doc to reflect the latest dependencies * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Remove see also reference to external collections * updated zos_copy module RST * Sample playbook for latest version * Updated ZOAU PTF information with the latest PTF UI70435 * Updates to raw doc to handle the error - Definition list ends without a blank line * Update copy module docs * Update and add clarity around Rocket Python * Requirements order change Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update conf for v1.2.0-beta.1 * Fix E231: missing whitespace * Changes to support single source doc site (#145) * Changes to support single source doc site * Under indentation * Under indentation 2 * doc review * doc review of additional files * Update the shell scripts to create a single requirements RST Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update/release v1.2.0 beta.2 (#162) * Fix E231: missing whitespace (#138) * Specify SFTP port for fetch and copy (#141) * Add sftp_port parameter to fetch and copy * Add sftp_port parameter to fetch and copy * Fix spelling * Missing positional arguments * Test case for negative port specification * Check for valid port range * Remove redundant assertions Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/dependency discovery (#142) * add host setup playbook * handle error cases * restructure and improve robustness of host-setup * add max_search_depth variable * change file extension for consistency * add documentation for host-setup.yaml * add in-file docs for tasks files * add important note for task files * Remove redundant data_set_util * Move error classes to the bottom * Rename backup_file to backup_name (#144) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/zos_copy record format (#147) * Update uss2mvs copy format * Default PS allocation should be FB * Use ZOAU Copy for sequential data sets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/148/accept all valid data set name characters (#149) * update regex for data set name * add test case for special characters * Add ignore_sftp_stderr parameter for fetch and copy * improve cleanup (#151) * change file extension * add /usr/sbin to default PATH * Use USS cp command to copy to PS * Bugfix: zos operator action query not returning all outstanding messages (#154) * remove ZOAU python API usage * add back ZOAU python API, update parsing regex * Changes to support single source doc site (#155) * Fix E231: missing whitespace * Changes to support single source doc site * Under indentation * Under indentation 2 * doc review * doc review of additional files * Update the shell scripts to create a single requirements RST Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * add verbose support and steplib fix (#156) * udpate maxdepth for RST to 1 * Content/3397/update docs (#157) * Module known issues and release notes added * Updated doc and sample host config playbook doc * Fix use ASCII quotes instead of Unicode quotes * use ASCII quotes instead of Unicode quotes * Update galaxy to latest doc site and version Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Jake Jackson <jljacks93@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> Co-authored-by: zlbjlu <16587467+yourfuwa2015@users.noreply.github.com> Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: balkajbaf <62961336+balkajbaf@users.noreply.github.com> Co-authored-by: behnam-ibm <behnam@ibm.com> * Doc generation updates to correct generation * Update Makefile to remove inclusive auto gen calls and fixed doc links * Update doc conf with added tags Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Jake Jackson <jljacks93@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> Co-authored-by: zlbjlu <16587467+yourfuwa2015@users.noreply.github.com> Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: balkajbaf <62961336+balkajbaf@users.noreply.github.com> Co-authored-by: behnam-ibm <behnam@ibm.com>
ddimatos
added a commit
that referenced
this pull request
Sep 3, 2020
* Release v1.2.0 beta.3 (#168) * Initial collection commit for v0.0.1 (#1) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * added arg parser to module_utils (#3) * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * Updated job output for improved DD handling (#14) Fixes the job summary output being returned when return_output=false. * Security module updates (#15) * patched security holes * updated test cases * removed unnecessary import * updated execution bits * Update modules and documentation corrections (#17) * update documentation, remove unused imports * updated requirements * updated module imports, formatting, and spelling * doc update and formatting update * z/OS Ping Module Feature Delivered (#18) * added zos_ping, test, and test fixture update * Description completeness * Grammar completeness * Pulled repetitive code into functions * updated module docs * Minor spelling corrections * Repo updates to satisfy ansible-test sanity testing requirements (#19) * update formatting * formatting changes to appease ansible-test * update tests * fix string format use and quotes * update README * remove trailing whitespace * update group_vars trailing spaces * remove shebang in attempt to fix encoding issue * add back shebang lines * Added security updates and test cases (#20) * add new types for parser, update arg handling zos_job_submit * updated module argument parsing and security tests * uncommented security tests * updated README to reflect use of MIT license * update hash function * test and formatting updates * updated documentation to pass ansible-test * switch name back to normal.py, add appropriate ignores * update ignore files * added ignore for GPL * updated README, .gitignore and removed MIT code * remove broken tests * increase robustness of zos_job_query * make job query even more robust * Webdocs content for IBM z/OS Core Collection (#21) * Initial commit for reStructuredText content * Updates and clean up to the template * Updated grammar * Grammar edit * Grammar fixes * Text updates * Updates to restructured text, fixed link, renamed collection * Fix ibm_zos_core bold * Fixes ansible-test lint: docs/source/conf.py:51:161: E501: line too long (178 > 160 characters) * ansible doc changes to modules for better formatting * remove original_message return parameter (#23) * remove original_message return parameter * Fix reported linting errors from PR * Fix reported linting errors from PR * Fix reported linting errors from PR for zos_job_query.py * Module document updates, Makefile changes and jinja support for defau… (#24) * Module document updates, Makefile changes and jinja support for default:int * Doc updates per linting rules * Update mmodule doc and remove md files (#25) * Removed all MD files from docs directory * Remove README.md , its replaced by README.rst * Fix bolding for ibm_zos_core * Fix highlight for ibm_zos_core * Updated index.rst to correspond to the README.md updates * Updated README.rst to point to webdocs * Updated README.rst webdocs URL * Update the modules to their ansible-doc * Documentation updates (#26) * Updated galaxy.yml to reflect latest version 1.0 * Documentation copyright updates and formatting * Documentation udpates * Update z/OS data set module and add catalog functionality (#27) * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * catalog for VSAM working, bugs fixed, tests added * satisfy ansible-test requirements, minor arg validation updates * removed unit tests * raise min . ansible requirement due to security vulnerability * added documentation to VTOC * Update product name in webdocs (#28) * Update product name in webdocs * Fix trailing newline * Document updates include some template bugs * Integraded document review comments (#29) * Integrated document review comments * Incorrect break for bulleted line * Bug in Jinja template where property would tab to much (#31) * Updates to documentation and galaxy configuration (#32) * Fix to README.rst, duplicate wording (#34) * Remove duplicate action plugins link (#35) * testing readme rendering (#36) Ported the README.rst file to README.md to meet collection requirements. * Corrected README to point to MD, and long tag removed (#39) * Add two additional arg types to BetterArgHandler (#44) * Add two additional arg types * Remove os from os.path.isabs Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Data Set Utility for Common Data Set Related Tasks (#48) * DataSetUtils initial commit * LISTDS and LISTCAT output processing * Raise exception when data set is open for write * Fix LISTDS regex * Fix indentation * Add USS path support * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/832/fix non vsam non sms existence checking zos data set (#49) * updated catalog handling for non-vsam, non-sms * updated some method comment structures * bugfix/822/determine-data-set-type (#50) * Use mvscmd CLI instead of Python API * Fix linting errors * Remove unused variable Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Extend dynamic function support to the better arg parser (#51) * Dynamic argument support for dynamic functions. * improve default handling for reusability of BetterArgParser object. * add test cases for dynamic argument support * removed print statements from test cases * Updates to data-set-utils exceptions (#52) * Avoid raising exception for uncatalogued data sets * verify listcat output for uncatalogued datasets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Removed return block from documentation (#53) Removed message and changed, empty return block now * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * Fixed PATH (#55) * Update/973/add new supported data set characters betterargparser (#56) * updated valid data set types builtin type name fix * added test cases for data set types * Bugfix when checking if data set exists (#57) * Return False instead of None when data set doesn't exist * Test for membership should ne 'not in' Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Committing module zos_operator_action_query (#43) * Reviewd changes for feature zos_operator_action_query.py * Fix linting errors reported in PR * Linting corrections * fix the syntax error * fix lint errors * fix lint error * fix lint error * fix lint error * fix function test * Added zos_operator_action_query to the sanity exemption list for lack of GPLV3 Lic * Update author tag * Update the collection reference * add function test(need revise later) & doc update * updated regex and value capturing * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * updated docs, made environment agnostic test cases * added support for "*" and fixed splat handling * updated test cases * updated regex matching method, updates tests * Update short description Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Check for data set member existence (#58) * Check for data set member existence * Resolve linting issue Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix for job name capture group (#60) * Better ZOAU import missing messages (#61) * Better ZOAU import missing messages * added missing license and imports * Changed behavior on failure for zosoperator_action_query (#63) * changed behavior on failure * update tests for new expected output * Bugfix 1661 & 1704 fix query of job for different user zos_job_submit and zos_job_action_query (#64) * updates to fix job submission as other user * updated and optimized job utility * removed usage of ZOAU and updated jobs utility usage * update job_output return structure * improved error messages * add job log cleanup for test cases * Initial module zos_tso_command (#46) * Linting corrections * Update doc so html will generate * Lintig updates * Lintig updates v2 * Lintig updates v3 * Lintig updates v4 * Lintig updates v5 * Lintig updates v6 * Lintig updates v7 * Lintig updates v8 * Delete test_zos_tso_command_unit.py Remove the unit test as it only mocks things. * Documenation updates * Update errro message * zos-tso-command update * linting update * testcase linting updates * linting update * linting update * linting update * linting update * linting updates * linting update * new authorized long command handling method * linting update * test cases update * linting update * linting update * linting update * tso-command new version * tso-command linting update * linting update * linting update * linting update' * linting removal * lint removal * linting testcases Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> * Update module_utils with an encode utility (#59) * initial commmit for encode_utils * support for string encoding conversion * Support MVS data set conversion and string conversion modification * Recursively copy to PDS fix * Linting correction * Linting updates * Linting updates 01 * fix ENOENT problem after shutil.move * updates with zos_encode * Merged branch and commit for zos_encode (#65) * Merged branch and commit for zos_encode * Lint error , trailing white space * z/OS Fetch Module (#7) * initial commit * begin updating examples * zos_fetch module * return rc, stdout, stderr as separate return parameters on failure * untrack non-fetch related files * add rest of the files * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * added unit and functional tests * check file existence when LISTDS returns invalid dataset name * added 'wait_s' parameter * clarified description of encoding parameter * make use_qaulifer false by default * remove is_uss and os_vsam parameters * remove data_set directory * remove is_uss parameter from examples * message return block * Fix YAML formatting * return code formatting * remove fail_on_missing from action plugin * fail if USS file does not exist * data sets are assumed to be cataloged * instead of entire dict, pass individual values * redundant code when fetching data set member * rename transfer_from_uss to fetch_non_partitioned_data_set * remove test cases for uncataloged datasets * import naming conflict * add __future__ imports * add __future__ imports * assuming all data sets are cataloged * assuming all data sets are cataloged * fix formatting issues * fix formatting issues * remove blank lines with whitespaces * change example member name data to DATA for accuracy. * continuation line unindented for visual indent * conform to PEP8 standard * avoid automatic field numbering for format * specify python interpreter * extract only data set name if member present * Remove newline character from error message * Update RETURN block with sample message * Remove blank line from return block * Update version_added to correspond to Ansible version * Check if temp datasets are being created successfully * Datasets.read() returns None for empty data sets * Use BetterArgParser to parse module args * Added parameter types * Remove trailing whitespaces * Avoid passing extra args from action plugin * Remove unused imports * Ignore lack of GPLv3 license complaint * catch ValueError thrown ny BetterArgParser * Blank line contains whitespace * Reduce line lengths to 80 * Add white space around string addition * Checking volume type no longer required * Check if src or dest is provided first * Fix indentation * Indent multiline string for better readability * Use DataSetUtils to get data set type * Use DataSetUtils to determine data set existence * Remove unit tests * Fix linting * Add test case for empty data sets * default params for _fail_json * Test case for missing source * Test case for insufficient write permission * Test case for insufficient desgt write permission * Catch exceptions while executing module * Use updated DataSetutils constructor * Updated functional tests * Fix linting errors * Use ansible fetch module to fetch USS files * Checking for USS type is unnecessary * Use sha256 instead of sha1 for checksum validation * Remove message return block * Missing uss file test case * Minor cleanup * Add stdout_lines and stderr_lines to module response * Update action plugin with stderr_lines and stdout_lines * YAML syntax issue * Corrected spelling errors * Inaccurate description of flat * No need to wrap error messages returned by DataSetUtils * Update doc regarding HFS and ZFS support * Handle encoding for USS files * Add encoding to doc * Calculate temp file size for VSAM storage * Preserve encoding using SFTP * Resolve PEP8 issues * Check for data set member existence * raw binary not supported in PY2 * Fix linting issues * Change doc to reflect USS absolute path * Prepend dataset hlq before parameter verification * src not declared before being referenced * Add test case using use_qualifier * Updated doc link for fetch L(..) and suboptions * Use tempfile.mkstemp() for temp data storage * Remove default for encoding * default encoding values are None * Create destination dirs when flat is false * Add test case for flat hostname * Fix spelling * exist_ok is not valid keyword for PY2 * Detect read-only file system * Handle sftp errors * Capture module_stderr in case of module failure * Ensure dest is a directory for PDS transfer * Ensure dest is a directory except when fetching a member * Fix data set member destination path * Default values for encoding is not needed * Remove unused imports * Update 'msg' description * Clean up temp files in case of module failure * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * enhanced zoau import error message (#66) * enhanced zoau import error message * satisfy pep8 witespace requirement * Added new playbook for latest module collection and a filter * fixed linting issues, added license * updated spacing to conform to formatting standards * updated shell to command where possible * Use encode util to convert encoding (#68) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update RST and MD doc files to reflect next release * Update group vars to reflect zoau 1.0.2 new env vars * Update encode short description to fit within the limits of doc length * Update makefile for more generic use by others * Update to fix RST lint error , short line * Rename sub-dir from plugin to filter, doc error * doc tree correction * Update release notes to correspond to beta release tag * Update sample playbooks with doc, configuration and new variables * Upate docs to highlight users to review sample playbooks and JCL before running * Update sample JCL to run on my systems with default values and notes * Add supporting doc for behavioral inventory parameters and ansible.cfg * Fix the md style URL in the index RST file * Update short descriptions for modules * Fix README.md link to IMS collection * Fix README.md link to collection artifacts * Linting updates * Add an example that requires a command to be escaped (#71) * refine backup_file & put it into encode_utils (#70) * refine backup_file & put it into encode_utils * correction for sanity test * refactor for security and usability, fix test case * move copy logic to utility * refactor zos_encode, improve backup functionality Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Update galaxy.yml to reflect the latest version and authors * Fix import name change resulting from refactor * updated usage of encode_utils to encode * Documentation updates, corrections and alterations (#75) * Documentation updates, corrections and alterations * ID review-round 2 * Update the user description in the doc Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Content/81/zos tso command docreview (#73) * ID review of zos-tso-commands * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * Fix indentation Co-authored-by: ddimatos <dimatos@gmail.com> * Fix extra space in doc * fixed zos_ping action plugin issues (#77) * Add out-of-box awx support (#80) * Added encoding supplemental doc (#78) * Added encoding supplemental doc * Removed references to modules not available and causing doc warnings * Fix Inline literal start-string without end-string. * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Add binary support copying across USS and MVS * Copy USS file in binary mode * fixed bug and cleaned up code (#82) * fixed bug and cleaned up code * fixed linting issues * Avoid specifying -B and -F for cp * Copy MVS source to MVS dest * Documentation web content updates (#84) * First round of review of static docs * docs/source/installation.rst * Update block to rendor with syntax highlights Code block was using 'yaml' when it was 'sh' , renders better if we have the correct code block type. * Incorporated Demetri's review comments Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Replace only the first occurance of 'rec' * Minor updates in install,quickstart, and req pages (#86) * Updated make file with comments and recover from failure * Update Backup util (mvs_file_backup) (#85) * Alternative mvs_file_backup * fix backup logic, add encode backup tests * add mvs backup tests, util fixes * fix test for new logic path Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Udpate the restructured text to reflect version release (#89) * minor fixes and additional tests for zos_encode (#88) * removed Jobs.submit usage due to invalid response (#92) * Update to galaxy.yml to exclude files * Update makefile to support rexx written modules * Generate and push zos_ping rst doc * Corrected clean up of a directory in makefile * Fixes spacing in RETURN values and formatting (#97) * Validate bk_dsn after assigning a value * Return backup path * Fix alloc command * Remove trailing whitespace * Fix job utility to work on all z/OS systems (#99) * Shinx and sphinx-versioning support and updates (#100) * Updated Makefile to support sphinx-versioning and renamed some of the operations * Update configuration to support sphinx-versioning and update options for sphinx * Update configuration with linting corrections * Correct and update the installation of a pre-release (#102) * Correct and update the installation of a pre-release * minor fixes to docs Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update data set utils (#104) * Update data set utils * Update function signatures * Module parameter not needed for encode util Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Change default encoding from UTF-8 to ISO8859-1 * Fetching non-existent VSAM * Update doc for encoding parameter * Update encoding to handle different charsets * Remove encoding default from argument spec * encoding validation and doc update * Fix typo * Bugfix/case conversion fix zos tso command (#108) * fixed issues with commands forced to uppercase * add support for single or multi commands, added relevant test cases * updated alias doc * Added index file to support shared doc site (#109) * Remove module parameter from EncodeUtils constructor * fix issue when no default encoding specified (#112) * fix issue when no default encoding specified * fix local job submission * Feature/26/ansible zos copy module (#105) * initial commit * begin updating examples * default value 'yes/no' -> 'true/false' * updated documentation * copy to seq data set * copy local to seq data set * add data set size to module response * functional test cases for copy * test case for sequential data set * clean up copy to ps function * copy to pdse * assuming all data sets are cataloged * copy to PDS * Copy to VSAM * Copy to PDS(E) member * Actual directory size instead of max file size * Copy from remote source * Update copy * Reduce line lengths to less than 80 * Use mvscmd cli * remote pdse copy * IEBCOPY to copy data sets * saving changes * Create remote data set if it doesn't exist * Reorganize remote copy transfer * Update return params * Avoid using global module * Remove unused imports * Copy to USS * Clean up in case of module failure * Back up USS file * Copy content to remote file * Copy to sequential data set * Copy local dir to PDS * Copy remote PDS to PDS * Update backup file * USS dest not accessible if not present * Explicitly tag destination file * Changed file mode, group and owner * Change common file args for remote src * Change file common args after copying to dest * Preserve local file mode * Convert encoding for remote src * Added comments * Use backup util to back up files and data sets * Add validate parameter * Copy VSAM to VSAM * Support for symlinks * Copy PDS/PDSE member to USS * Fix formatting * Module should fail for empty VSAM * Added function signatures for all test cases * User better_arg_parser to determine member type * Add binary support for copying to USS target * Copy to PS in binary mode * Correct object instantiations * Import copy util * Remove write_rc * Resolve import issues * Move vsam copy handler to main copy handler class * Copy local dir to USS dir * Infer member name when it isn't provided * Added test cases for local and remote copy * src_member evalutes to True * Comments for extract_dsname() * IEBCOPY fails for incompatible recfm * Copy PDS member to PS * Copy data set member to non-existent PS * Copy local symlink * Add test case for uss file backup * Fix mvs file backup * Functional tests complete * Fix linting issues * Remove trailing whitespace * Ignore GPLv3 license * Remove is_vsam parameter * Fix license information * Fail action plugin * Add test cases to ensure module failure * Remove unneded parameters * Refactor to use updated data_set util * I/O abend when copying PDSE member to non-existent PS * Updated doc, comments, linting * Fix linting issues * Remove duplicate key from dic * Avoid hard-coding tmp path name * Change backup_path parameter to backup_file * Copy to data set member partial * Error converting encoding for local dir * Fix tmp cleanup bug * Updated test cases for backup_file * plugins/modules/zos_copy.py * Reactor code for better readability * Fix pep8 issue * Cleanup error on module failure * plugins/modules/zos_copy.py * Doc update mode parameter * Use native shell instead of bash for functional testing * Return invocation parameter * Remove unnecessary imports * plugins/modules/zos_copy.py * Return backup_file on module failure Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Feature/1089/zos lineinfile (#111) * initial version for interface and options * add backup option * update lineinfile * make golden path work * fix the bugs so that method could support none vars * refactor based on new dsed * updated params and return * Initial check in for backup feature support * support all charsets; updated backup feature doc * clean up; used better_arg_parser * use latest data_set module_util * change the returned string to be loadable by json.loads (BOF/EOF) * uss test cases * dataset test cases; create a dataset/file with content instead of being dependent on existing source * uss test env set bug fix * added UTF-8 in test cases; simplified dataset names * use new dedicated ZOAU function for lineinfile * added assert for expected strings * encoding parameter from absent * Linting updates from pipeline tests * Linting updates and corrections * Linting udpates continued * Update sanity exemption file and linting corrections * Doc error, indentation off, breaking rest of doc parse * module arg and doc updates * Formatting updates * Further doc updates * regexp alias fix * updated docs; added comments; updated parameters * fix sanity checks * fix RETURN key type; lint error * code review changes * fixed linting issues * removed incorrectly added choices for insertafter/insertbefore * added negative test cases fo not supported datasets * moved choices to description; fixed test cases lint issue * changed the choices format in description * add escape if string was quoted * added test cases for escaped/unescaped quotation * used 'src' for target location; moved the params checking before taking backup * fixed indentation * Doc edits for lineinfile, encode, and fetch * remove relative import method * removed the default values for insertafter/insertbefore * renamed helper functions; updated pytest.ini for test cases; fixed TEST_INFO * remove jcl tmp directory if it already exists * run all test_* Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: behnam-ibm <behnam@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/1067/community doc (#113) * Correct doc link to IMS collection to point to Galaxy * Added testing doc to community guidlines * Clean/delete accidental file * Fix Sphinx warning coming from ping module reference to zos_ssh plugin undefined label: zos_ssh_module * Added better arg parse to new dev addition to community guides * Fixes broken site anchor to the YAML configuration * Fix MD style Link * Remove additional MD style links * Fix mailformed RST link * Fix more RST links * Python syntax fixes in samples * Update betterArgParse to BetterArgParser * Expand pipelining=true doc * Correct Unicode quotes to ASCII quotes * Statement of direction on zoau and python * doc review of community guidelines * Doc review of Community guidelines * Fix malformed RST table * Update group vars requirements needed by IBM Pypthon * Update doc with var settings and sample all.yml * Update environ vars * Added environ LANG doc * Added doc to clarify shell support * Added advanced group_vars sample and doc * Update docs to reflect IBM Python announcement Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Remove unnecessary description of PDSE transfer * Update version tag in doc generation conf * Add missing link reference * Doc review (#118) Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update IBM Python download link * Add system info to job query (#127) * Add system info to job query * Change sysname to system for job query Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Close file descriptor (#128) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/2298/zos raw (#120) * add utilities for zos_raw * updated DD statement to better handle concatenations * update parameter names * fix concat naming * refactor of zos_data_set, add new features * updated examples * added steplib dd type * fix steplib data set * update valid concat members * fix datasetdefinition for steplibs * update dd statement types * argument parsing working in zos_raw * module argument parsing updates * updated valid arguments for consistency * update tests for new zos_data_set format * zos_raw module working * module doc updates * updated function documentation * backup, reuse and replace working * updated documentation * cleaned up wrapper classes * updated existence checking, function doc updates * spelling fixes, added new parameters for dd_unix * add list of string support for StdinDefinition * properly handle space units * initial set of test cases, added aliases for program, rename args to parms to match jcl naming conventions * added remaining happy path data set DD tests, added input dd tests. Fixed small bug in raw related to invalid parameter options * update example playbooks to account for new zos_data_set arguments * added initial functional tests for unix file dd statements * added access group support, fixed status group * added remaining tests for dd_unix and dd_dummy * fixed bug on dd_concat contents or backup returned * add dd_concat test cases * update parms to parm to match jcl naming * added execution tests for zos_raw * add final test cases * add special type functions for ksds related params * update import handling for special cases * add VIO DD support * uncomment test cases * fix basic linting issues, default arguments * add support for directory blocks * fix some linting issues * Revert "fix some linting issues" This reverts commit 44d0e14. * linting fixes * module doc and linting updates * additional doc fixes for linting sanity tests * added examples * added catch for nonexistent deletion * updated docs for proper formatting * update ignore files for new module * added dd_output support, standardized dd names * removed unnecessary import wrappers * improved validation for encoding types. * added new example with dd_output type * Merge branch 'origin/dev' * Revert "Merge branch 'origin/dev'" This reverts commit 1befb8e. * fixes for python 2 controller support * modify encode utils import * add utf-8 encoding tag to unit tests * improve check mode support * fix mock of AnsibleModule for encoding check cases * fix unit test formatting for pep8 * fix test case helpers * comment out failing test case unrelated to zos_raw * update encoding conversion handling * update rexx variables to capture correct environment variables * update naming based on discussion * rename test case files, update test cases * update default for output data set * update test ignore files for rename * doc edits (#133) * doc edits * fix minor pep8 issues Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * add python 2 support * remove redundant code comments * add temp data set support, backwards compatibility fix with old interface, update docs and tests * update documentation for return_content * minor doc changes for clarity * improve documentation clarity * fix incorrect syntax for option name Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Update configuration vars in doc and yml (#131) * remove ZOAU Python API deps for zos_operator (#132) * remove ZOAU deps which break in 1.0.3 * add back shebang * remove unsafe shell usage * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/2516/release doc update (#136) * Update the version in galaxy.yml * edits to support doc gen * Updated for doc gen * updated all module doc for release * Update template to upper case boolean when no default present * update raw module doc to fomat as the indentation intended * updated module RST * add details to module enhancements * Update message around Rocket's python * Update Rocket Python Message * Reference Rocket Python configuration in requirements * Updated doc to reflect the latest dependencies * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Remove see also reference to external collections * updated zos_copy module RST * Sample playbook for latest version * Updated ZOAU PTF information with the latest PTF UI70435 * Updates to raw doc to handle the error - Definition list ends without a blank line * Update copy module docs * Update and add clarity around Rocket Python * Requirements order change Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update conf for v1.2.0-beta.1 * Fix E231: missing whitespace (#138) * Specify SFTP port for fetch and copy (#141) * Add sftp_port parameter to fetch and copy * Add sftp_port parameter to fetch and copy * Fix spelling * Missing positional arguments * Test case for negative port specification * Check for valid port range * Remove redundant assertions Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/dependency discovery (#142) * add host setup playbook * handle error cases * restructure and improve robustness of host-setup * add max_search_depth variable * change file extension for consistency * add documentation for host-setup.yaml * add in-file docs for tasks files * add important note for task files * Remove redundant data_set_util * Move error classes to the bottom * Rename backup_file to backup_name (#144) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/zos_copy record format (#147) * Update uss2mvs copy format * Default PS allocation should be FB * Use ZOAU Copy for sequential data sets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/148/accept all valid data set name characters (#149) * update regex for data set name * add test case for special characters * Add ignore_sftp_stderr parameter for fetch and copy * improve cleanup (#151) * change file extension * add /usr/sbin to default PATH * Use USS cp command to copy to PS * Bugfix: zos operator action query not returning all outstanding messages (#154) * remove ZOAU python API usage * add back ZOAU python API, update parsing regex * Changes to support single source doc site (#155) * Fix E231: missing whitespace * Changes to support single source doc site * Under indentation * Under indentation 2 * doc review * doc review of additional files * Update the shell scripts to create a single requirements RST Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * add verbose support and steplib fix (#156) * udpate maxdepth for RST to 1 * Content/3397/update docs (#158) * Fix E231: missing whitespace * Changes to support single source doc site (#145) * Changes to support single source doc site * Under indentation * Under indentation 2 * doc review * doc review of additional files * Update the shell scripts to create a single requirements RST Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Module known issues and release notes added * Updated doc and sample host config playbook doc * Fix use ASCII quotes instead of Unicode quotes * use ASCII quotes instead of Unicode quotes Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update galaxy to latest doc site and version * add FQCN to action plugins (#167) * add FQCN to action plugins * fix pep8 * Fixed release notes merge comment * Galaxy and release notes update to reflect bugfix Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Jake Jackson <jljacks93@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> Co-authored-by: zlbjlu <16587467+yourfuwa2015@users.noreply.github.com> Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: balkajbaf <62961336+balkajbaf@users.noreply.github.com> Co-authored-by: behnam-ibm <behnam@ibm.com> * add FQCN to action plugins (#167) * add FQCN to action plugins * fix pep8 * Doc generation updates to correct generation * Update Makefile to remove inclusive auto gen calls and fixed doc links * update comment in conf * Generated RST module doc updates * Update Galaxy to latest beta release * Update version doc config to pull next beta release * Fixes zos_job_submit doc where should duration_s should be wait_time_s * Fix collection requirement in sample playbook * update release notes to correctly reflect zoau supported dependency * Correct missing white space after comma Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Jake Jackson <jljacks93@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> Co-authored-by: zlbjlu <16587467+yourfuwa2015@users.noreply.github.com> Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: balkajbaf <62961336+balkajbaf@users.noreply.github.com> Co-authored-by: behnam-ibm <behnam@ibm.com>
ddimatos
added a commit
that referenced
this pull request
Sep 4, 2020
* Initial collection commit for v0.0.1 (#1) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * added arg parser to module_utils (#3) * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * Updated job output for improved DD handling (#14) Fixes the job summary output being returned when return_output=false. * Security module updates (#15) * patched security holes * updated test cases * removed unnecessary import * updated execution bits * Update modules and documentation corrections (#17) * update documentation, remove unused imports * updated requirements * updated module imports, formatting, and spelling * doc update and formatting update * z/OS Ping Module Feature Delivered (#18) * added zos_ping, test, and test fixture update * Description completeness * Grammar completeness * Pulled repetitive code into functions * updated module docs * Minor spelling corrections * Repo updates to satisfy ansible-test sanity testing requirements (#19) * update formatting * formatting changes to appease ansible-test * update tests * fix string format use and quotes * update README * remove trailing whitespace * update group_vars trailing spaces * remove shebang in attempt to fix encoding issue * add back shebang lines * Added security updates and test cases (#20) * add new types for parser, update arg handling zos_job_submit * updated module argument parsing and security tests * uncommented security tests * updated README to reflect use of MIT license * update hash function * test and formatting updates * updated documentation to pass ansible-test * switch name back to normal.py, add appropriate ignores * update ignore files * added ignore for GPL * updated README, .gitignore and removed MIT code * remove broken tests * increase robustness of zos_job_query * make job query even more robust * Webdocs content for IBM z/OS Core Collection (#21) * Initial commit for reStructuredText content * Updates and clean up to the template * Updated grammar * Grammar edit * Grammar fixes * Text updates * Updates to restructured text, fixed link, renamed collection * Fix ibm_zos_core bold * Fixes ansible-test lint: docs/source/conf.py:51:161: E501: line too long (178 > 160 characters) * ansible doc changes to modules for better formatting * remove original_message return parameter (#23) * remove original_message return parameter * Fix reported linting errors from PR * Fix reported linting errors from PR * Fix reported linting errors from PR for zos_job_query.py * Module document updates, Makefile changes and jinja support for defau… (#24) * Module document updates, Makefile changes and jinja support for default:int * Doc updates per linting rules * Update mmodule doc and remove md files (#25) * Removed all MD files from docs directory * Remove README.md , its replaced by README.rst * Fix bolding for ibm_zos_core * Fix highlight for ibm_zos_core * Updated index.rst to correspond to the README.md updates * Updated README.rst to point to webdocs * Updated README.rst webdocs URL * Update the modules to their ansible-doc * Documentation updates (#26) * Updated galaxy.yml to reflect latest version 1.0 * Documentation copyright updates and formatting * Documentation udpates * Update z/OS data set module and add catalog functionality (#27) * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * catalog for VSAM working, bugs fixed, tests added * satisfy ansible-test requirements, minor arg validation updates * removed unit tests * raise min . ansible requirement due to security vulnerability * added documentation to VTOC * Update product name in webdocs (#28) * Update product name in webdocs * Fix trailing newline * Document updates include some template bugs * Integraded document review comments (#29) * Integrated document review comments * Incorrect break for bulleted line * Bug in Jinja template where property would tab to much (#31) * Updates to documentation and galaxy configuration (#32) * Fix to README.rst, duplicate wording (#34) * Remove duplicate action plugins link (#35) * testing readme rendering (#36) Ported the README.rst file to README.md to meet collection requirements. * Corrected README to point to MD, and long tag removed (#39) * Add two additional arg types to BetterArgHandler (#44) * Add two additional arg types * Remove os from os.path.isabs Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Data Set Utility for Common Data Set Related Tasks (#48) * DataSetUtils initial commit * LISTDS and LISTCAT output processing * Raise exception when data set is open for write * Fix LISTDS regex * Fix indentation * Add USS path support * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/832/fix non vsam non sms existence checking zos data set (#49) * updated catalog handling for non-vsam, non-sms * updated some method comment structures * bugfix/822/determine-data-set-type (#50) * Use mvscmd CLI instead of Python API * Fix linting errors * Remove unused variable Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Extend dynamic function support to the better arg parser (#51) * Dynamic argument support for dynamic functions. * improve default handling for reusability of BetterArgParser object. * add test cases for dynamic argument support * removed print statements from test cases * Updates to data-set-utils exceptions (#52) * Avoid raising exception for uncatalogued data sets * verify listcat output for uncatalogued datasets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Removed return block from documentation (#53) Removed message and changed, empty return block now * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * Fixed PATH (#55) * Update/973/add new supported data set characters betterargparser (#56) * updated valid data set types builtin type name fix * added test cases for data set types * Bugfix when checking if data set exists (#57) * Return False instead of None when data set doesn't exist * Test for membership should ne 'not in' Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Committing module zos_operator_action_query (#43) * Reviewd changes for feature zos_operator_action_query.py * Fix linting errors reported in PR * Linting corrections * fix the syntax error * fix lint errors * fix lint error * fix lint error * fix lint error * fix function test * Added zos_operator_action_query to the sanity exemption list for lack of GPLV3 Lic * Update author tag * Update the collection reference * add function test(need revise later) & doc update * updated regex and value capturing * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * updated docs, made environment agnostic test cases * added support for "*" and fixed splat handling * updated test cases * updated regex matching method, updates tests * Update short description Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Check for data set member existence (#58) * Check for data set member existence * Resolve linting issue Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix for job name capture group (#60) * Better ZOAU import missing messages (#61) * Better ZOAU import missing messages * added missing license and imports * Changed behavior on failure for zosoperator_action_query (#63) * changed behavior on failure * update tests for new expected output * Bugfix 1661 & 1704 fix query of job for different user zos_job_submit and zos_job_action_query (#64) * updates to fix job submission as other user * updated and optimized job utility * removed usage of ZOAU and updated jobs utility usage * update job_output return structure * improved error messages * add job log cleanup for test cases * Initial module zos_tso_command (#46) * Linting corrections * Update doc so html will generate * Lintig updates * Lintig updates v2 * Lintig updates v3 * Lintig updates v4 * Lintig updates v5 * Lintig updates v6 * Lintig updates v7 * Lintig updates v8 * Delete test_zos_tso_command_unit.py Remove the unit test as it only mocks things. * Documenation updates * Update errro message * zos-tso-command update * linting update * testcase linting updates * linting update * linting update * linting update * linting update * linting updates * linting update * new authorized long command handling method * linting update * test cases update * linting update * linting update * linting update * tso-command new version * tso-command linting update * linting update * linting update * linting update' * linting removal * lint removal * linting testcases Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> * Update module_utils with an encode utility (#59) * initial commmit for encode_utils * support for string encoding conversion * Support MVS data set conversion and string conversion modification * Recursively copy to PDS fix * Linting correction * Linting updates * Linting updates 01 * fix ENOENT problem after shutil.move * updates with zos_encode * Merged branch and commit for zos_encode (#65) * Merged branch and commit for zos_encode * Lint error , trailing white space * z/OS Fetch Module (#7) * initial commit * begin updating examples * zos_fetch module * return rc, stdout, stderr as separate return parameters on failure * untrack non-fetch related files * add rest of the files * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * added unit and functional tests * check file existence when LISTDS returns invalid dataset name * added 'wait_s' parameter * clarified description of encoding parameter * make use_qaulifer false by default * remove is_uss and os_vsam parameters * remove data_set directory * remove is_uss parameter from examples * message return block * Fix YAML formatting * return code formatting * remove fail_on_missing from action plugin * fail if USS file does not exist * data sets are assumed to be cataloged * instead of entire dict, pass individual values * redundant code when fetching data set member * rename transfer_from_uss to fetch_non_partitioned_data_set * remove test cases for uncataloged datasets * import naming conflict * add __future__ imports * add __future__ imports * assuming all data sets are cataloged * assuming all data sets are cataloged * fix formatting issues * fix formatting issues * remove blank lines with whitespaces * change example member name data to DATA for accuracy. * continuation line unindented for visual indent * conform to PEP8 standard * avoid automatic field numbering for format * specify python interpreter * extract only data set name if member present * Remove newline character from error message * Update RETURN block with sample message * Remove blank line from return block * Update version_added to correspond to Ansible version * Check if temp datasets are being created successfully * Datasets.read() returns None for empty data sets * Use BetterArgParser to parse module args * Added parameter types * Remove trailing whitespaces * Avoid passing extra args from action plugin * Remove unused imports * Ignore lack of GPLv3 license complaint * catch ValueError thrown ny BetterArgParser * Blank line contains whitespace * Reduce line lengths to 80 * Add white space around string addition * Checking volume type no longer required * Check if src or dest is provided first * Fix indentation * Indent multiline string for better readability * Use DataSetUtils to get data set type * Use DataSetUtils to determine data set existence * Remove unit tests * Fix linting * Add test case for empty data sets * default params for _fail_json * Test case for missing source * Test case for insufficient write permission * Test case for insufficient desgt write permission * Catch exceptions while executing module * Use updated DataSetutils constructor * Updated functional tests * Fix linting errors * Use ansible fetch module to fetch USS files * Checking for USS type is unnecessary * Use sha256 instead of sha1 for checksum validation * Remove message return block * Missing uss file test case * Minor cleanup * Add stdout_lines and stderr_lines to module response * Update action plugin with stderr_lines and stdout_lines * YAML syntax issue * Corrected spelling errors * Inaccurate description of flat * No need to wrap error messages returned by DataSetUtils * Update doc regarding HFS and ZFS support * Handle encoding for USS files * Add encoding to doc * Calculate temp file size for VSAM storage * Preserve encoding using SFTP * Resolve PEP8 issues * Check for data set member existence * raw binary not supported in PY2 * Fix linting issues * Change doc to reflect USS absolute path * Prepend dataset hlq before parameter verification * src not declared before being referenced * Add test case using use_qualifier * Updated doc link for fetch L(..) and suboptions * Use tempfile.mkstemp() for temp data storage * Remove default for encoding * default encoding values are None * Create destination dirs when flat is false * Add test case for flat hostname * Fix spelling * exist_ok is not valid keyword for PY2 * Detect read-only file system * Handle sftp errors * Capture module_stderr in case of module failure * Ensure dest is a directory for PDS transfer * Ensure dest is a directory except when fetching a member * Fix data set member destination path * Default values for encoding is not needed * Remove unused imports * Update 'msg' description * Clean up temp files in case of module failure * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * enhanced zoau import error message (#66) * enhanced zoau import error message * satisfy pep8 witespace requirement * Added new playbook for latest module collection and a filter * fixed linting issues, added license * updated spacing to conform to formatting standards * updated shell to command where possible * Use encode util to convert encoding (#68) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update RST and MD doc files to reflect next release * Update group vars to reflect zoau 1.0.2 new env vars * Update encode short description to fit within the limits of doc length * Update makefile for more generic use by others * Update to fix RST lint error , short line * Rename sub-dir from plugin to filter, doc error * doc tree correction * Update release notes to correspond to beta release tag * Update sample playbooks with doc, configuration and new variables * Upate docs to highlight users to review sample playbooks and JCL before running * Update sample JCL to run on my systems with default values and notes * Add supporting doc for behavioral inventory parameters and ansible.cfg * Fix the md style URL in the index RST file * Update short descriptions for modules * Fix README.md link to IMS collection * Fix README.md link to collection artifacts * Linting updates * Add an example that requires a command to be escaped (#71) * refine backup_file & put it into encode_utils (#70) * refine backup_file & put it into encode_utils * correction for sanity test * refactor for security and usability, fix test case * move copy logic to utility * refactor zos_encode, improve backup functionality Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Update galaxy.yml to reflect the latest version and authors * Fix import name change resulting from refactor * updated usage of encode_utils to encode * Documentation updates, corrections and alterations (#75) * Documentation updates, corrections and alterations * ID review-round 2 * Update the user description in the doc Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Content/81/zos tso command docreview (#73) * ID review of zos-tso-commands * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * Fix indentation Co-authored-by: ddimatos <dimatos@gmail.com> * Fix extra space in doc * fixed zos_ping action plugin issues (#77) * Add out-of-box awx support (#80) * Added encoding supplemental doc (#78) * Added encoding supplemental doc * Removed references to modules not available and causing doc warnings * Fix Inline literal start-string without end-string. * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Add binary support copying across USS and MVS * Copy USS file in binary mode * fixed bug and cleaned up code (#82) * fixed bug and cleaned up code * fixed linting issues * Avoid specifying -B and -F for cp * Copy MVS source to MVS dest * Documentation web content updates (#84) * First round of review of static docs * docs/source/installation.rst * Update block to rendor with syntax highlights Code block was using 'yaml' when it was 'sh' , renders better if we have the correct code block type. * Incorporated Demetri's review comments Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Replace only the first occurance of 'rec' * Minor updates in install,quickstart, and req pages (#86) * Updated make file with comments and recover from failure * Update Backup util (mvs_file_backup) (#85) * Alternative mvs_file_backup * fix backup logic, add encode backup tests * add mvs backup tests, util fixes * fix test for new logic path Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Udpate the restructured text to reflect version release (#89) * minor fixes and additional tests for zos_encode (#88) * removed Jobs.submit usage due to invalid response (#92) * Update to galaxy.yml to exclude files * Update makefile to support rexx written modules * Generate and push zos_ping rst doc * Corrected clean up of a directory in makefile * Fixes spacing in RETURN values and formatting (#97) * Validate bk_dsn after assigning a value * Return backup path * Fix alloc command * Remove trailing whitespace * Fix job utility to work on all z/OS systems (#99) * Shinx and sphinx-versioning support and updates (#100) * Updated Makefile to support sphinx-versioning and renamed some of the operations * Update configuration to support sphinx-versioning and update options for sphinx * Update configuration with linting corrections * Correct and update the installation of a pre-release (#102) * Correct and update the installation of a pre-release * minor fixes to docs Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update data set utils (#104) * Update data set utils * Update function signatures * Module parameter not needed for encode util Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Change default encoding from UTF-8 to ISO8859-1 * Fetching non-existent VSAM * Update doc for encoding parameter * Update encoding to handle different charsets * Remove encoding default from argument spec * encoding validation and doc update * Fix typo * Bugfix/case conversion fix zos tso command (#108) * fixed issues with commands forced to uppercase * add support for single or multi commands, added relevant test cases * updated alias doc * Added index file to support shared doc site (#109) * Remove module parameter from EncodeUtils constructor * fix issue when no default encoding specified (#112) * fix issue when no default encoding specified * fix local job submission * Feature/26/ansible zos copy module (#105) * initial commit * begin updating examples * default value 'yes/no' -> 'true/false' * updated documentation * copy to seq data set * copy local to seq data set * add data set size to module response * functional test cases for copy * test case for sequential data set * clean up copy to ps function * copy to pdse * assuming all data sets are cataloged * copy to PDS * Copy to VSAM * Copy to PDS(E) member * Actual directory size instead of max file size * Copy from remote source * Update copy * Reduce line lengths to less than 80 * Use mvscmd cli * remote pdse copy * IEBCOPY to copy data sets * saving changes * Create remote data set if it doesn't exist * Reorganize remote copy transfer * Update return params * Avoid using global module * Remove unused imports * Copy to USS * Clean up in case of module failure * Back up USS file * Copy content to remote file * Copy to sequential data set * Copy local dir to PDS * Copy remote PDS to PDS * Update backup file * USS dest not accessible if not present * Explicitly tag destination file * Changed file mode, group and owner * Change common file args for remote src * Change file common args after copying to dest * Preserve local file mode * Convert encoding for remote src * Added comments * Use backup util to back up files and data sets * Add validate parameter * Copy VSAM to VSAM * Support for symlinks * Copy PDS/PDSE member to USS * Fix formatting * Module should fail for empty VSAM * Added function signatures for all test cases * User better_arg_parser to determine member type * Add binary support for copying to USS target * Copy to PS in binary mode * Correct object instantiations * Import copy util * Remove write_rc * Resolve import issues * Move vsam copy handler to main copy handler class * Copy local dir to USS dir * Infer member name when it isn't provided * Added test cases for local and remote copy * src_member evalutes to True * Comments for extract_dsname() * IEBCOPY fails for incompatible recfm * Copy PDS member to PS * Copy data set member to non-existent PS * Copy local symlink * Add test case for uss file backup * Fix mvs file backup * Functional tests complete * Fix linting issues * Remove trailing whitespace * Ignore GPLv3 license * Remove is_vsam parameter * Fix license information * Fail action plugin * Add test cases to ensure module failure * Remove unneded parameters * Refactor to use updated data_set util * I/O abend when copying PDSE member to non-existent PS * Updated doc, comments, linting * Fix linting issues * Remove duplicate key from dic * Avoid hard-coding tmp path name * Change backup_path parameter to backup_file * Copy to data set member partial * Error converting encoding for local dir * Fix tmp cleanup bug * Updated test cases for backup_file * plugins/modules/zos_copy.py * Reactor code for better readability * Fix pep8 issue * Cleanup error on module failure * plugins/modules/zos_copy.py * Doc update mode parameter * Use native shell instead of bash for functional testing * Return invocation parameter * Remove unnecessary imports * plugins/modules/zos_copy.py * Return backup_file on module failure Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Feature/1089/zos lineinfile (#111) * initial version for interface and options * add backup option * update lineinfile * make golden path work * fix the bugs so that method could support none vars * refactor based on new dsed * updated params and return * Initial check in for backup feature support * support all charsets; updated backup feature doc * clean up; used better_arg_parser * use latest data_set module_util * change the returned string to be loadable by json.loads (BOF/EOF) * uss test cases * dataset test cases; create a dataset/file with content instead of being dependent on existing source * uss test env set bug fix * added UTF-8 in test cases; simplified dataset names * use new dedicated ZOAU function for lineinfile * added assert for expected strings * encoding parameter from absent * Linting updates from pipeline tests * Linting updates and corrections * Linting udpates continued * Update sanity exemption file and linting corrections * Doc error, indentation off, breaking rest of doc parse * module arg and doc updates * Formatting updates * Further doc updates * regexp alias fix * updated docs; added comments; updated parameters * fix sanity checks * fix RETURN key type; lint error * code review changes * fixed linting issues * removed incorrectly added choices for insertafter/insertbefore * added negative test cases fo not supported datasets * moved choices to description; fixed test cases lint issue * changed the choices format in description * add escape if string was quoted * added test cases for escaped/unescaped quotation * used 'src' for target location; moved the params checking before taking backup * fixed indentation * Doc edits for lineinfile, encode, and fetch * remove relative import method * removed the default values for insertafter/insertbefore * renamed helper functions; updated pytest.ini for test cases; fixed TEST_INFO * remove jcl tmp directory if it already exists * run all test_* Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: behnam-ibm <behnam@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/1067/community doc (#113) * Correct doc link to IMS collection to point to Galaxy * Added testing doc to community guidlines * Clean/delete accidental file * Fix Sphinx warning coming from ping module reference to zos_ssh plugin undefined label: zos_ssh_module * Added better arg parse to new dev addition to community guides * Fixes broken site anchor to the YAML configuration * Fix MD style Link * Remove additional MD style links * Fix mailformed RST link * Fix more RST links * Python syntax fixes in samples * Update betterArgParse to BetterArgParser * Expand pipelining=true doc * Correct Unicode quotes to ASCII quotes * Statement of direction on zoau and python * doc review of community guidelines * Doc review of Community guidelines * Fix malformed RST table * Update group vars requirements needed by IBM Pypthon * Update doc with var settings and sample all.yml * Update environ vars * Added environ LANG doc * Added doc to clarify shell support * Added advanced group_vars sample and doc * Update docs to reflect IBM Python announcement Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Remove unnecessary description of PDSE transfer * Update version tag in doc generation conf * Add missing link reference * Doc review (#118) Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update IBM Python download link * Add system info to job query (#127) * Add system info to job query * Change sysname to system for job query Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Close file descriptor (#128) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/2298/zos raw (#120) * add utilities for zos_raw * updated DD statement to better handle concatenations * update parameter names * fix concat naming * refactor of zos_data_set, add new features * updated examples * added steplib dd type * fix steplib data set * update valid concat members * fix datasetdefinition for steplibs * update dd statement types * argument parsing working in zos_raw * module argument parsing updates * updated valid arguments for consistency * update tests for new zos_data_set format * zos_raw module working * module doc updates * updated function documentation * backup, reuse and replace working * updated documentation * cleaned up wrapper classes * updated existence checking, function doc updates * spelling fixes, added new parameters for dd_unix * add list of string support for StdinDefinition * properly handle space units * initial set of test cases, added aliases for program, rename args to parms to match jcl naming conventions * added remaining happy path data set DD tests, added input dd tests. Fixed small bug in raw related to invalid parameter options * update example playbooks to account for new zos_data_set arguments * added initial functional tests for unix file dd statements * added access group support, fixed status group * added remaining tests for dd_unix and dd_dummy * fixed bug on dd_concat contents or backup returned * add dd_concat test cases * update parms to parm to match jcl naming * added execution tests for zos_raw * add final test cases * add special type functions for ksds related params * update import handling for special cases * add VIO DD support * uncomment test cases * fix basic linting issues, default arguments * add support for directory blocks * fix some linting issues * Revert "fix some linting issues" This reverts commit 44d0e1457d76e8c5e5725b457f525d76afb1111f. * linting fixes * module doc and linting updates * additional doc fixes for linting sanity tests * added examples * added catch for nonexistent deletion * updated docs for proper formatting * update ignore files for new module * added dd_output support, standardized dd names * removed unnecessary import wrappers * improved validation for encoding types. * added new example with dd_output type * Merge branch 'origin/dev' * Revert "Merge branch 'origin/dev'" This reverts commit 1befb8e8f0a2448cb9408d94c3cf6a64a2ff9e4c. * fixes for python 2 controller support * modify encode utils import * add utf-8 encoding tag to unit tests * improve check mode support * fix mock of AnsibleModule for encoding check cases * fix unit test formatting for pep8 * fix test case helpers * comment out failing test case unrelated to zos_raw * update encoding conversion handling * update rexx variables to capture correct environment variables * update naming based on discussion * rename test case files, update test cases * update default for output data set * update test ignore files for rename * doc edits (#133) * doc edits * fix minor pep8 issues Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * add python 2 support * remove redundant code comments * add temp data set support, backwards compatibility fix with old interface, update docs and tests * update documentation for return_content * minor doc changes for clarity * improve documentation clarity * fix incorrect syntax for option name Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Update configuration vars in doc and yml (#131) * remove ZOAU Python API deps for zos_operator (#132) * remove ZOAU deps which break in 1.0.3 * add back shebang * remove unsafe shell usage * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/2516/release doc update (#136) * Update the version in galaxy.yml * edits to support doc gen * Updated for doc gen * updated all module doc for release * Update template to upper case boolean when no default present * update raw module doc to fomat as the indentation intended * updated module RST * add details to module enhancements * Update message around Rocket's python * Update Rocket Python Message * Reference Rocket Python configuration in requirements * Updated doc to reflect the latest dependencies * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Remove see also reference to external collections * updated zos_copy module RST * Sample playbook for latest version * Updated ZOAU PTF information with the latest PTF UI70435 * Updates to raw doc to handle the error - Definition list ends without a blank line * Update copy module docs * Update and add clarity around Rocket Python * Requirements order change Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update conf for v1.2.0-beta.1 * Fix E231: missing whitespace (#138) * Specify SFTP port for fetch and copy (#141) * Add sftp_port parameter to fetch and copy * Add sftp_port parameter to fetch and copy * Fix spelling * Missing positional arguments * Test case for negative port specification * Check for valid port range * Remove redundant assertions Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/dependency discovery (#142) * add host setup playbook * handle error cases * restructure and improve robustness of host-setup * add max_search_depth variable * change file extension for consistency * add documentation for host-setup.yaml * add in-file docs for tasks files * add important note for task files * Remove redundant data_set_util * Move error classes to the bottom * Rename backup_file to backup_name (#144) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/zos_copy record format (#147) * Update uss2mvs copy format * Default PS allocation should be FB * Use ZOAU Copy for sequential data sets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/148/accept all valid data set name characters (#149) * update regex for data set name * add test case for special characters * Add ignore_sftp_stderr parameter for fetch and copy * improve cleanup (#151) * change file extension * add /usr/sbin to default PATH * Use USS cp command to copy to PS * Bugfix: zos operator action query not returning all outstanding messages (#154) * remove ZOAU python API usage * add back ZOAU python API, update parsing regex * Changes to support single source doc site (#155) * Fix E231: missing whitespace * Changes to support single source doc site * Under indentation * Under indentation 2 * doc review * doc review of additional files * Update the shell scripts to create a single requirements RST Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * add verbose support and steplib fix (#156) * udpate maxdepth for RST to 1 * Use default discovered port for sftp * Content/3397/update docs (#158) * Fix E231: missing whitespace * Changes to support single source doc site (#145) * Changes to support single source doc site * Under indentation * Under indentation 2 * doc review * doc review of additional files * Update the shell scripts to create a single requirements RST Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Module known issues and release notes added * Updated doc and sample host config playbook doc * Fix use ASCII quotes instead of Unicode quotes * use ASCII quotes instead of Unicode quotes Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update galaxy to latest doc site and version * Release v1.2.0 beta.2 (#163) * Initial collection commit for v0.0.1 (#1) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * added arg parser to module_utils (#3) * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * Updated job output for improved DD handling (#14) Fixes the job summary output being returned when return_output=false. * Security module updates (#15) * patched security holes * updated test cases * removed unnecessary import * updated execution bits * Update modules and documentation corrections (#17) * update documentation, remove unused imports * updated requirements * updated module imports, formatting, and spelling * doc update and formatting update * z/OS Ping Module Feature Delivered (#18) * added zos_ping, test, and test fixture update * Description completeness * Grammar completeness * Pulled repetitive code into functions * updated module docs * Minor spelling corrections * Repo updates to satisfy ansible-test sanity testing requirements (#19) * update formatting * formatting changes to appease ansible-test * update tests * fix string format use and quotes * update README * remove trailing whitespace * update group_vars trailing spaces * remove shebang in attempt to fix encoding issue * add back shebang lines * Added security updates and test cases (#20) * add new types for parser, update arg handling zos_job_submit * updated module argument parsing and security tests * uncommented security tests * updated README to reflect use of MIT license * update hash function * test and formatting updates * updated documentation to pass ansible-test * switch name back to normal.py, add appropriate ignores * update ignore files * added ignore for GPL * updated README, .gitignore and removed MIT code * remove broken tests * increase robustness of zos_job_query * make job query even more robust * Webdocs content for IBM z/OS Core Collection (#21) * Initial commit for reStructuredText content * Updates and clean up to the template * Updated grammar * Grammar edit * Grammar fixes * Text updates * Updates to restructured text, fixed link, renamed collection * Fix ibm_zos_core bold * Fixes ansible-test lint: docs/source/conf.py:51:161: E501: line too long (178 > 160 characters) * ansible doc changes to modules for better formatting * remove original_message return parameter (#23) * remove original_message return parameter * Fix reported linting errors from PR * Fix reported linting errors from PR * Fix reported linting errors from PR for zos_job_query.py * Module document updates, Makefile changes and jinja support for defau… (#24) * Module document updates, Makefile changes and jinja support for default:int * Doc updates per linting rules * Update mmodule doc and remove md files (#25) * Removed all MD files from docs directory * Remove README.md , its replaced by README.rst * Fix bolding for ibm_zos_core * Fix highlight for ibm_zos_core * Updated index.rst to correspond to the README.md updates * Updated README.rst to point to webdocs * Updated README.rst webdocs URL * Update the modules to their ansible-doc * Documentation updates (#26) * Updated galaxy.yml to reflect latest version 1.0 * Documentation copyright updates and formatting * Documentation udpates * Update z/OS data set module and add catalog functionality (#27) * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * catalog for VSAM working, bugs fixed, tests added * satisfy ansible-test requirements, minor arg validation updates * removed unit tests * raise min . ansible requirement due to security vulnerability * added documentation to VTOC * Update product name in webdocs (#28) * Update product name in webdocs * Fix trailing newline * Document updates include some template bugs * Integraded document review comments (#29) * Integrated document review comments * Incorrect break for bulleted line * Bug in Jinja template where property would tab to much (#31) * Updates to documentation and galaxy configuration (#32) * Fix to README.rst, duplicate wording (#34) * Remove duplicate action plugins link (#35) * testing readme rendering (#36) Ported the README.rst file to README.md to meet collection requirements. * Corrected README to point to MD, and long tag removed (#39) * Add two additional arg types to BetterArgHandler (#44) * Add two additional arg types * Remove os from os.path.isabs Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Data Set Utility for Common Data Set Related Tasks (#48) * DataSetUtils initial commit * LISTDS and LISTCAT output processing * Raise exception when data set is open for write * Fix LISTDS regex * Fix indentation * Add USS path support * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/832/fix non vsam non sms existence checking zos data set (#49) * updated catalog handling for non-vsam, non-sms * updated some method comment structures * bugfix/822/determine-data-set-type (#50) * Use mvscmd CLI instead of Python API * Fix linting errors * Remove unused variable Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Extend dynamic function support to the better arg parser (#51) * Dynamic argument support for dynamic functions. * improve default handling for reusability of BetterArgParser object. * add test cases for dynamic argument support * removed print statements from test cases * Updates to data-set-utils exceptions (#52) * Avoid raising exception for uncatalogued data sets * verify listcat output for uncatalogued datasets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Removed return block from documentation (#53) Removed message and changed, empty return block now * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * Fixed PATH (#55) * Update/973/add new supported data set characters betterargparser (#56) * updated valid data set types builtin type name fix * added test cases for data set types * Bugfix when checking if data set exists (#57) * Return False instead of None when data set doesn't exist * Test for membership should ne 'not in' Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Committing module zos_operator_action_query (#43) * Reviewd changes for feature zos_operator_action_query.py * Fix linting errors reported in PR * Linting corrections * fix the syntax error * fix lint errors * fix lint error * fix lint error * fix lint error * fix function test * Added zos_operator_action_query to the sanity exemption list for lack of GPLV3 Lic * Update author tag * Update the collection reference * add function test(need revise later) & doc update * updated regex and value capturing * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * updated docs, made environment agnostic test cases * added support for "*" and fixed splat handling * updated test cases * updated regex matching method, updates tests * Update short description Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Check for data set member existence (#58) * Check for data set member existence * Resolve linting issue Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix for job name capture group (#60) * Better ZOAU import missing messages (#61) * Better ZOAU import missing messages * added missing license and imports * Changed behavior on failure for zosoperator_action_query (#63) * changed behavior on failure * update tests for new expected output * Bugfix 1661 & 1704 fix query of job for different user zos_job_submit and zos_job_action_query (#64) * updates to fix job submission as other user * updated and optimized job utility * removed usage of ZOAU and updated jobs utility usage * update job_output return structure * improved error messages * add job log cleanup for test cases * Initial module zos_tso_command (#46) * Linting corrections * Update doc so html will generate * Lintig updates * Lintig updates v2 * Lintig updates v3 * Lintig updates v4 * Lintig updates v5 * Lintig updates v6 * Lintig updates v7 * Lintig updates v8 * Delete test_zos_tso_command_unit.py Remove the unit test as it only mocks things. * Documenation updates * Update errro message * zos-tso-command update * linting update * testcase linting updates * linting update * linting update * linting update * linting update * linting updates * linting update * new authorized long command handling method * linting update * test cases update * linting update * linting update * linting update * tso-command new version * tso-command linting update * linting update * linting update * linting update' * linting removal * lint removal * linting testcases Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> * Update module_utils with an encode utility (#59) * initial commmit for encode_utils * support for string encoding conversion * Support MVS data set conversion and string conversion modification * Recursively copy to PDS fix * Linting correction * Linting updates * Linting updates 01 * fix ENOENT problem after shutil.move * updates with zos_encode * Merged branch and commit for zos_encode (#65) * Merged branch and commit for zos_encode * Lint error , trailing white space * z/OS Fetch Module (#7) * initial commit * begin updating examples * zos_fetch module * return rc, stdout, stderr as separate return parameters on failure * untrack non-fetch related files * add rest of the files * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * added unit and functional tests * check file existence when LISTDS returns invalid dataset name * added 'wait_s' parameter * clarified description of encoding parameter * make use_qaulifer false by default * remove is_uss and os_vsam parameters * remove data_set directory * remove is_uss parameter from examples * message return block * Fix YAML formatting * return code formatting * remove fail_on_missing from action plugin * fail if USS file does not exist * data sets are assumed to be cataloged * instead of entire dict, pass individual values * redundant code when fetching data set member * rename transfer_from_uss to fetch_non_partitioned_data_set * remove test cases for uncataloged datasets * import naming conflict * add __future__ imports * add __future__ imports * assuming all data sets are cataloged * assuming all data sets are cataloged * fix formatting issues * fix formatting issues * remove blank lines with whitespaces * change example member name data to DATA for accuracy. * continuation line unindented for visual indent * conform to PEP8 standard * avoid automatic field numbering for format * specify python interpreter * extract only data set name if member present * Remove newline character from error message * Update RETURN block with sample message * Remove blank line from return block * Update version_added to correspond to Ansible version * Check if temp datasets are being created successfully * Datasets.read() returns None for empty data sets * Use BetterArgParser to parse module args * Added parameter types * Remove trailing whitespaces * Avoid passing extra args from action plugin * Remove unused imports * Ignore lack of GPLv3 license complaint * catch ValueError thrown ny BetterArgParser * Blank line contains whitespace * Reduce line lengths to 80 * Add white space around string addition * Checking volume type no longer required * Check if src or dest is provided first * Fix indentation * Indent multiline string for better readability * Use DataSetUtils to get data set type * Use DataSetUtils to determine data set existence * Remove unit tests * Fix linting * Add test case for empty data sets * default params for _fail_json * Test case for missing source * Test case for insufficient write permission * Test case for insufficient desgt write permission * Catch exceptions while executing module * Use updated DataSetutils constructor * Updated functional tests * Fix linting errors * Use ansible fetch module to fetch USS files * Checking for USS type is unnecessary * Use sha256 instead of sha1 for checksum validation * Remove message return block * Missing uss file test case * Minor cleanup * Add stdout_lines and stderr_lines to module response * Update action plugin with stderr_lines and stdout_lines * YAML syntax issue * Corrected spelling errors * Inaccurate description of flat * No need to wrap error messages returned by DataSetUtils * Update doc regarding HFS and ZFS support * Handle encoding for USS files * Add encoding to doc * Calculate temp file size for VSAM storage * Preserve encoding using SFTP * Resolve PEP8 issues * Check for data set member existence * raw binary not supported in PY2 * Fix linting issues * Change doc to reflect USS absolute path * Prepend dataset hlq before parameter verification * src not declared before being referenced * Add test case using use_qualifier * Updated doc link for fetch L(..) and suboptions * Use tempfile.mkstemp() for temp data storage * Remove default for encoding * default encoding values are None * Create destination dirs when flat is false * Add test case for flat hostname * Fix spelling * exist_ok is not valid keyword for PY2 * Detect read-only file system * Handle sftp errors * Capture module_stderr in case of module failure * Ensure dest is a directory for PDS transfer * Ensure dest is a directory except when fetching a member * Fix data set member destination path * Default values for encoding is not needed * Remove unused imports * Update 'msg' description * Clean up temp files in case of module failure * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * enhanced zoau import error message (#66) * enhanced zoau import error message * satisfy pep8 witespace requirement * Added new playbook for latest module collection and a filter * fixed linting issues, added license * updated spacing to conform to formatting standards * updated shell to command where possible * Use encode util to convert encoding (#68) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update RST and MD doc files to reflect next release * Update group vars to reflect zoau 1.0.2 new env vars * Update encode short description to fit within the limits of doc length * Update makefile for more generic use by others * Update to fix RST lint error , short line * Rename sub-dir from plugin to filter, doc error * doc tree correction * Update release notes to correspond to beta release tag * Update sample playbooks with doc, configuration and new variables * Upate docs to highlight users to review sample playbooks and JCL before running * Update sample JCL to run on my systems with default values and notes * Add supporting doc for behavioral inventory parameters and ansible.cfg * Fix the md style URL in the index RST file * Update short descriptions for modules * Fix README.md link to IMS collection * Fix README.md link to collection artifacts * Linting updates * Add an example that requires a command to be escaped (#71) * refine backup_file & put it into encode_utils (#70) * refine backup_file & put it into encode_utils * correction for sanity test * refactor for security and usability, fix test case * move copy logic to utility * refactor zos_encode, improve backup functionality Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Update galaxy.yml to reflect the latest version and authors * Fix import name change resulting from refactor * updated usage of encode_utils to encode * Documentation updates, corrections and alterations (#75) * Documentation updates, corrections and alterations * ID review-round 2 * Update the user description in the doc Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Content/81/zos tso command docreview (#73) * ID review of zos-tso-commands * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * Fix indentation Co-authored-by: ddimatos …
ddimatos
added a commit
that referenced
this pull request
Sep 17, 2020
* Release v1.2.0 beta.4 (#185) * Initial collection commit for v0.0.1 (#1) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * added arg parser to module_utils (#3) * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * Updated job output for improved DD handling (#14) Fixes the job summary output being returned when return_output=false. * Security module updates (#15) * patched security holes * updated test cases * removed unnecessary import * updated execution bits * Update modules and documentation corrections (#17) * update documentation, remove unused imports * updated requirements * updated module imports, formatting, and spelling * doc update and formatting update * z/OS Ping Module Feature Delivered (#18) * added zos_ping, test, and test fixture update * Description completeness * Grammar completeness * Pulled repetitive code into functions * updated module docs * Minor spelling corrections * Repo updates to satisfy ansible-test sanity testing requirements (#19) * update formatting * formatting changes to appease ansible-test * update tests * fix string format use and quotes * update README * remove trailing whitespace * update group_vars trailing spaces * remove shebang in attempt to fix encoding issue * add back shebang lines * Added security updates and test cases (#20) * add new types for parser, update arg handling zos_job_submit * updated module argument parsing and security tests * uncommented security tests * updated README to reflect use of MIT license * update hash function * test and formatting updates * updated documentation to pass ansible-test * switch name back to normal.py, add appropriate ignores * update ignore files * added ignore for GPL * updated README, .gitignore and removed MIT code * remove broken tests * increase robustness of zos_job_query * make job query even more robust * Webdocs content for IBM z/OS Core Collection (#21) * Initial commit for reStructuredText content * Updates and clean up to the template * Updated grammar * Grammar edit * Grammar fixes * Text updates * Updates to restructured text, fixed link, renamed collection * Fix ibm_zos_core bold * Fixes ansible-test lint: docs/source/conf.py:51:161: E501: line too long (178 > 160 characters) * ansible doc changes to modules for better formatting * remove original_message return parameter (#23) * remove original_message return parameter * Fix reported linting errors from PR * Fix reported linting errors from PR * Fix reported linting errors from PR for zos_job_query.py * Module document updates, Makefile changes and jinja support for defau… (#24) * Module document updates, Makefile changes and jinja support for default:int * Doc updates per linting rules * Update mmodule doc and remove md files (#25) * Removed all MD files from docs directory * Remove README.md , its replaced by README.rst * Fix bolding for ibm_zos_core * Fix highlight for ibm_zos_core * Updated index.rst to correspond to the README.md updates * Updated README.rst to point to webdocs * Updated README.rst webdocs URL * Update the modules to their ansible-doc * Documentation updates (#26) * Updated galaxy.yml to reflect latest version 1.0 * Documentation copyright updates and formatting * Documentation udpates * Update z/OS data set module and add catalog functionality (#27) * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * catalog for VSAM working, bugs fixed, tests added * satisfy ansible-test requirements, minor arg validation updates * removed unit tests * raise min . ansible requirement due to security vulnerability * added documentation to VTOC * Update product name in webdocs (#28) * Update product name in webdocs * Fix trailing newline * Document updates include some template bugs * Integraded document review comments (#29) * Integrated document review comments * Incorrect break for bulleted line * Bug in Jinja template where property would tab to much (#31) * Updates to documentation and galaxy configuration (#32) * Fix to README.rst, duplicate wording (#34) * Remove duplicate action plugins link (#35) * testing readme rendering (#36) Ported the README.rst file to README.md to meet collection requirements. * Corrected README to point to MD, and long tag removed (#39) * Add two additional arg types to BetterArgHandler (#44) * Add two additional arg types * Remove os from os.path.isabs Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Data Set Utility for Common Data Set Related Tasks (#48) * DataSetUtils initial commit * LISTDS and LISTCAT output processing * Raise exception when data set is open for write * Fix LISTDS regex * Fix indentation * Add USS path support * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/832/fix non vsam non sms existence checking zos data set (#49) * updated catalog handling for non-vsam, non-sms * updated some method comment structures * bugfix/822/determine-data-set-type (#50) * Use mvscmd CLI instead of Python API * Fix linting errors * Remove unused variable Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Extend dynamic function support to the better arg parser (#51) * Dynamic argument support for dynamic functions. * improve default handling for reusability of BetterArgParser object. * add test cases for dynamic argument support * removed print statements from test cases * Updates to data-set-utils exceptions (#52) * Avoid raising exception for uncatalogued data sets * verify listcat output for uncatalogued datasets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Removed return block from documentation (#53) Removed message and changed, empty return block now * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * Fixed PATH (#55) * Update/973/add new supported data set characters betterargparser (#56) * updated valid data set types builtin type name fix * added test cases for data set types * Bugfix when checking if data set exists (#57) * Return False instead of None when data set doesn't exist * Test for membership should ne 'not in' Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Committing module zos_operator_action_query (#43) * Reviewd changes for feature zos_operator_action_query.py * Fix linting errors reported in PR * Linting corrections * fix the syntax error * fix lint errors * fix lint error * fix lint error * fix lint error * fix function test * Added zos_operator_action_query to the sanity exemption list for lack of GPLV3 Lic * Update author tag * Update the collection reference * add function test(need revise later) & doc update * updated regex and value capturing * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * updated docs, made environment agnostic test cases * added support for "*" and fixed splat handling * updated test cases * updated regex matching method, updates tests * Update short description Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Check for data set member existence (#58) * Check for data set member existence * Resolve linting issue Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix for job name capture group (#60) * Better ZOAU import missing messages (#61) * Better ZOAU import missing messages * added missing license and imports * Changed behavior on failure for zosoperator_action_query (#63) * changed behavior on failure * update tests for new expected output * Bugfix 1661 & 1704 fix query of job for different user zos_job_submit and zos_job_action_query (#64) * updates to fix job submission as other user * updated and optimized job utility * removed usage of ZOAU and updated jobs utility usage * update job_output return structure * improved error messages * add job log cleanup for test cases * Initial module zos_tso_command (#46) * Linting corrections * Update doc so html will generate * Lintig updates * Lintig updates v2 * Lintig updates v3 * Lintig updates v4 * Lintig updates v5 * Lintig updates v6 * Lintig updates v7 * Lintig updates v8 * Delete test_zos_tso_command_unit.py Remove the unit test as it only mocks things. * Documenation updates * Update errro message * zos-tso-command update * linting update * testcase linting updates * linting update * linting update * linting update * linting update * linting updates * linting update * new authorized long command handling method * linting update * test cases update * linting update * linting update * linting update * tso-command new version * tso-command linting update * linting update * linting update * linting update' * linting removal * lint removal * linting testcases Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> * Update module_utils with an encode utility (#59) * initial commmit for encode_utils * support for string encoding conversion * Support MVS data set conversion and string conversion modification * Recursively copy to PDS fix * Linting correction * Linting updates * Linting updates 01 * fix ENOENT problem after shutil.move * updates with zos_encode * Merged branch and commit for zos_encode (#65) * Merged branch and commit for zos_encode * Lint error , trailing white space * z/OS Fetch Module (#7) * initial commit * begin updating examples * zos_fetch module * return rc, stdout, stderr as separate return parameters on failure * untrack non-fetch related files * add rest of the files * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * added unit and functional tests * check file existence when LISTDS returns invalid dataset name * added 'wait_s' parameter * clarified description of encoding parameter * make use_qaulifer false by default * remove is_uss and os_vsam parameters * remove data_set directory * remove is_uss parameter from examples * message return block * Fix YAML formatting * return code formatting * remove fail_on_missing from action plugin * fail if USS file does not exist * data sets are assumed to be cataloged * instead of entire dict, pass individual values * redundant code when fetching data set member * rename transfer_from_uss to fetch_non_partitioned_data_set * remove test cases for uncataloged datasets * import naming conflict * add __future__ imports * add __future__ imports * assuming all data sets are cataloged * assuming all data sets are cataloged * fix formatting issues * fix formatting issues * remove blank lines with whitespaces * change example member name data to DATA for accuracy. * continuation line unindented for visual indent * conform to PEP8 standard * avoid automatic field numbering for format * specify python interpreter * extract only data set name if member present * Remove newline character from error message * Update RETURN block with sample message * Remove blank line from return block * Update version_added to correspond to Ansible version * Check if temp datasets are being created successfully * Datasets.read() returns None for empty data sets * Use BetterArgParser to parse module args * Added parameter types * Remove trailing whitespaces * Avoid passing extra args from action plugin * Remove unused imports * Ignore lack of GPLv3 license complaint * catch ValueError thrown ny BetterArgParser * Blank line contains whitespace * Reduce line lengths to 80 * Add white space around string addition * Checking volume type no longer required * Check if src or dest is provided first * Fix indentation * Indent multiline string for better readability * Use DataSetUtils to get data set type * Use DataSetUtils to determine data set existence * Remove unit tests * Fix linting * Add test case for empty data sets * default params for _fail_json * Test case for missing source * Test case for insufficient write permission * Test case for insufficient desgt write permission * Catch exceptions while executing module * Use updated DataSetutils constructor * Updated functional tests * Fix linting errors * Use ansible fetch module to fetch USS files * Checking for USS type is unnecessary * Use sha256 instead of sha1 for checksum validation * Remove message return block * Missing uss file test case * Minor cleanup * Add stdout_lines and stderr_lines to module response * Update action plugin with stderr_lines and stdout_lines * YAML syntax issue * Corrected spelling errors * Inaccurate description of flat * No need to wrap error messages returned by DataSetUtils * Update doc regarding HFS and ZFS support * Handle encoding for USS files * Add encoding to doc * Calculate temp file size for VSAM storage * Preserve encoding using SFTP * Resolve PEP8 issues * Check for data set member existence * raw binary not supported in PY2 * Fix linting issues * Change doc to reflect USS absolute path * Prepend dataset hlq before parameter verification * src not declared before being referenced * Add test case using use_qualifier * Updated doc link for fetch L(..) and suboptions * Use tempfile.mkstemp() for temp data storage * Remove default for encoding * default encoding values are None * Create destination dirs when flat is false * Add test case for flat hostname * Fix spelling * exist_ok is not valid keyword for PY2 * Detect read-only file system * Handle sftp errors * Capture module_stderr in case of module failure * Ensure dest is a directory for PDS transfer * Ensure dest is a directory except when fetching a member * Fix data set member destination path * Default values for encoding is not needed * Remove unused imports * Update 'msg' description * Clean up temp files in case of module failure * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * enhanced zoau import error message (#66) * enhanced zoau import error message * satisfy pep8 witespace requirement * Added new playbook for latest module collection and a filter * fixed linting issues, added license * updated spacing to conform to formatting standards * updated shell to command where possible * Use encode util to convert encoding (#68) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update RST and MD doc files to reflect next release * Update group vars to reflect zoau 1.0.2 new env vars * Update encode short description to fit within the limits of doc length * Update makefile for more generic use by others * Update to fix RST lint error , short line * Rename sub-dir from plugin to filter, doc error * doc tree correction * Update release notes to correspond to beta release tag * Update sample playbooks with doc, configuration and new variables * Upate docs to highlight users to review sample playbooks and JCL before running * Update sample JCL to run on my systems with default values and notes * Add supporting doc for behavioral inventory parameters and ansible.cfg * Fix the md style URL in the index RST file * Update short descriptions for modules * Fix README.md link to IMS collection * Fix README.md link to collection artifacts * Linting updates * Add an example that requires a command to be escaped (#71) * refine backup_file & put it into encode_utils (#70) * refine backup_file & put it into encode_utils * correction for sanity test * refactor for security and usability, fix test case * move copy logic to utility * refactor zos_encode, improve backup functionality Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Update galaxy.yml to reflect the latest version and authors * Fix import name change resulting from refactor * updated usage of encode_utils to encode * Documentation updates, corrections and alterations (#75) * Documentation updates, corrections and alterations * ID review-round 2 * Update the user description in the doc Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Content/81/zos tso command docreview (#73) * ID review of zos-tso-commands * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * Fix indentation Co-authored-by: ddimatos <dimatos@gmail.com> * Fix extra space in doc * fixed zos_ping action plugin issues (#77) * Add out-of-box awx support (#80) * Added encoding supplemental doc (#78) * Added encoding supplemental doc * Removed references to modules not available and causing doc warnings * Fix Inline literal start-string without end-string. * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Add binary support copying across USS and MVS * Copy USS file in binary mode * fixed bug and cleaned up code (#82) * fixed bug and cleaned up code * fixed linting issues * Avoid specifying -B and -F for cp * Copy MVS source to MVS dest * Documentation web content updates (#84) * First round of review of static docs * docs/source/installation.rst * Update block to rendor with syntax highlights Code block was using 'yaml' when it was 'sh' , renders better if we have the correct code block type. * Incorporated Demetri's review comments Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Replace only the first occurance of 'rec' * Minor updates in install,quickstart, and req pages (#86) * Updated make file with comments and recover from failure * Update Backup util (mvs_file_backup) (#85) * Alternative mvs_file_backup * fix backup logic, add encode backup tests * add mvs backup tests, util fixes * fix test for new logic path Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Udpate the restructured text to reflect version release (#89) * minor fixes and additional tests for zos_encode (#88) * removed Jobs.submit usage due to invalid response (#92) * Update to galaxy.yml to exclude files * Update makefile to support rexx written modules * Generate and push zos_ping rst doc * Corrected clean up of a directory in makefile * Fixes spacing in RETURN values and formatting (#97) * Validate bk_dsn after assigning a value * Return backup path * Fix alloc command * Remove trailing whitespace * Fix job utility to work on all z/OS systems (#99) * Shinx and sphinx-versioning support and updates (#100) * Updated Makefile to support sphinx-versioning and renamed some of the operations * Update configuration to support sphinx-versioning and update options for sphinx * Update configuration with linting corrections * Correct and update the installation of a pre-release (#102) * Correct and update the installation of a pre-release * minor fixes to docs Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update data set utils (#104) * Update data set utils * Update function signatures * Module parameter not needed for encode util Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Change default encoding from UTF-8 to ISO8859-1 * Fetching non-existent VSAM * Update doc for encoding parameter * Update encoding to handle different charsets * Remove encoding default from argument spec * encoding validation and doc update * Fix typo * Bugfix/case conversion fix zos tso command (#108) * fixed issues with commands forced to uppercase * add support for single or multi commands, added relevant test cases * updated alias doc * Added index file to support shared doc site (#109) * Remove module parameter from EncodeUtils constructor * fix issue when no default encoding specified (#112) * fix issue when no default encoding specified * fix local job submission * Feature/26/ansible zos copy module (#105) * initial commit * begin updating examples * default value 'yes/no' -> 'true/false' * updated documentation * copy to seq data set * copy local to seq data set * add data set size to module response * functional test cases for copy * test case for sequential data set * clean up copy to ps function * copy to pdse * assuming all data sets are cataloged * copy to PDS * Copy to VSAM * Copy to PDS(E) member * Actual directory size instead of max file size * Copy from remote source * Update copy * Reduce line lengths to less than 80 * Use mvscmd cli * remote pdse copy * IEBCOPY to copy data sets * saving changes * Create remote data set if it doesn't exist * Reorganize remote copy transfer * Update return params * Avoid using global module * Remove unused imports * Copy to USS * Clean up in case of module failure * Back up USS file * Copy content to remote file * Copy to sequential data set * Copy local dir to PDS * Copy remote PDS to PDS * Update backup file * USS dest not accessible if not present * Explicitly tag destination file * Changed file mode, group and owner * Change common file args for remote src * Change file common args after copying to dest * Preserve local file mode * Convert encoding for remote src * Added comments * Use backup util to back up files and data sets * Add validate parameter * Copy VSAM to VSAM * Support for symlinks * Copy PDS/PDSE member to USS * Fix formatting * Module should fail for empty VSAM * Added function signatures for all test cases * User better_arg_parser to determine member type * Add binary support for copying to USS target * Copy to PS in binary mode * Correct object instantiations * Import copy util * Remove write_rc * Resolve import issues * Move vsam copy handler to main copy handler class * Copy local dir to USS dir * Infer member name when it isn't provided * Added test cases for local and remote copy * src_member evalutes to True * Comments for extract_dsname() * IEBCOPY fails for incompatible recfm * Copy PDS member to PS * Copy data set member to non-existent PS * Copy local symlink * Add test case for uss file backup * Fix mvs file backup * Functional tests complete * Fix linting issues * Remove trailing whitespace * Ignore GPLv3 license * Remove is_vsam parameter * Fix license information * Fail action plugin * Add test cases to ensure module failure * Remove unneded parameters * Refactor to use updated data_set util * I/O abend when copying PDSE member to non-existent PS * Updated doc, comments, linting * Fix linting issues * Remove duplicate key from dic * Avoid hard-coding tmp path name * Change backup_path parameter to backup_file * Copy to data set member partial * Error converting encoding for local dir * Fix tmp cleanup bug * Updated test cases for backup_file * plugins/modules/zos_copy.py * Reactor code for better readability * Fix pep8 issue * Cleanup error on module failure * plugins/modules/zos_copy.py * Doc update mode parameter * Use native shell instead of bash for functional testing * Return invocation parameter * Remove unnecessary imports * plugins/modules/zos_copy.py * Return backup_file on module failure Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Feature/1089/zos lineinfile (#111) * initial version for interface and options * add backup option * update lineinfile * make golden path work * fix the bugs so that method could support none vars * refactor based on new dsed * updated params and return * Initial check in for backup feature support * support all charsets; updated backup feature doc * clean up; used better_arg_parser * use latest data_set module_util * change the returned string to be loadable by json.loads (BOF/EOF) * uss test cases * dataset test cases; create a dataset/file with content instead of being dependent on existing source * uss test env set bug fix * added UTF-8 in test cases; simplified dataset names * use new dedicated ZOAU function for lineinfile * added assert for expected strings * encoding parameter from absent * Linting updates from pipeline tests * Linting updates and corrections * Linting udpates continued * Update sanity exemption file and linting corrections * Doc error, indentation off, breaking rest of doc parse * module arg and doc updates * Formatting updates * Further doc updates * regexp alias fix * updated docs; added comments; updated parameters * fix sanity checks * fix RETURN key type; lint error * code review changes * fixed linting issues * removed incorrectly added choices for insertafter/insertbefore * added negative test cases fo not supported datasets * moved choices to description; fixed test cases lint issue * changed the choices format in description * add escape if string was quoted * added test cases for escaped/unescaped quotation * used 'src' for target location; moved the params checking before taking backup * fixed indentation * Doc edits for lineinfile, encode, and fetch * remove relative import method * removed the default values for insertafter/insertbefore * renamed helper functions; updated pytest.ini for test cases; fixed TEST_INFO * remove jcl tmp directory if it already exists * run all test_* Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: behnam-ibm <behnam@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/1067/community doc (#113) * Correct doc link to IMS collection to point to Galaxy * Added testing doc to community guidlines * Clean/delete accidental file * Fix Sphinx warning coming from ping module reference to zos_ssh plugin undefined label: zos_ssh_module * Added better arg parse to new dev addition to community guides * Fixes broken site anchor to the YAML configuration * Fix MD style Link * Remove additional MD style links * Fix mailformed RST link * Fix more RST links * Python syntax fixes in samples * Update betterArgParse to BetterArgParser * Expand pipelining=true doc * Correct Unicode quotes to ASCII quotes * Statement of direction on zoau and python * doc review of community guidelines * Doc review of Community guidelines * Fix malformed RST table * Update group vars requirements needed by IBM Pypthon * Update doc with var settings and sample all.yml * Update environ vars * Added environ LANG doc * Added doc to clarify shell support * Added advanced group_vars sample and doc * Update docs to reflect IBM Python announcement Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Remove unnecessary description of PDSE transfer * Update version tag in doc generation conf * Add missing link reference * Doc review (#118) Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update IBM Python download link * Add system info to job query (#127) * Add system info to job query * Change sysname to system for job query Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Close file descriptor (#128) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/2298/zos raw (#120) * add utilities for zos_raw * updated DD statement to better handle concatenations * update parameter names * fix concat naming * refactor of zos_data_set, add new features * updated examples * added steplib dd type * fix steplib data set * update valid concat members * fix datasetdefinition for steplibs * update dd statement types * argument parsing working in zos_raw * module argument parsing updates * updated valid arguments for consistency * update tests for new zos_data_set format * zos_raw module working * module doc updates * updated function documentation * backup, reuse and replace working * updated documentation * cleaned up wrapper classes * updated existence checking, function doc updates * spelling fixes, added new parameters for dd_unix * add list of string support for StdinDefinition * properly handle space units * initial set of test cases, added aliases for program, rename args to parms to match jcl naming conventions * added remaining happy path data set DD tests, added input dd tests. Fixed small bug in raw related to invalid parameter options * update example playbooks to account for new zos_data_set arguments * added initial functional tests for unix file dd statements * added access group support, fixed status group * added remaining tests for dd_unix and dd_dummy * fixed bug on dd_concat contents or backup returned * add dd_concat test cases * update parms to parm to match jcl naming * added execution tests for zos_raw * add final test cases * add special type functions for ksds related params * update import handling for special cases * add VIO DD support * uncomment test cases * fix basic linting issues, default arguments * add support for directory blocks * fix some linting issues * Revert "fix some linting issues" This reverts commit 44d0e1457d76e8c5e5725b457f525d76afb1111f. * linting fixes * module doc and linting updates * additional doc fixes for linting sanity tests * added examples * added catch for nonexistent deletion * updated docs for proper formatting * update ignore files for new module * added dd_output support, standardized dd names * removed unnecessary import wrappers * improved validation for encoding types. * added new example with dd_output type * Merge branch 'origin/dev' * Revert "Merge branch 'origin/dev'" This reverts commit 1befb8e8f0a2448cb9408d94c3cf6a64a2ff9e4c. * fixes for python 2 controller support * modify encode utils import * add utf-8 encoding tag to unit tests * improve check mode support * fix mock of AnsibleModule for encoding check cases * fix unit test formatting for pep8 * fix test case helpers * comment out failing test case unrelated to zos_raw * update encoding conversion handling * update rexx variables to capture correct environment variables * update naming based on discussion * rename test case files, update test cases * update default for output data set * update test ignore files for rename * doc edits (#133) * doc edits * fix minor pep8 issues Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * add python 2 support * remove redundant code comments * add temp data set support, backwards compatibility fix with old interface, update docs and tests * update documentation for return_content * minor doc changes for clarity * improve documentation clarity * fix incorrect syntax for option name Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Update configuration vars in doc and yml (#131) * remove ZOAU Python API deps for zos_operator (#132) * remove ZOAU deps which break in 1.0.3 * add back shebang * remove unsafe shell usage * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/2516/release doc update (#136) * Update the version in galaxy.yml * edits to support doc gen * Updated for doc gen * updated all module doc for release * Update template to upper case boolean when no default present * update raw module doc to fomat as the indentation intended * updated module RST * add details to module enhancements * Update message around Rocket's python * Update Rocket Python Message * Reference Rocket Python configuration in requirements * Updated doc to reflect the latest dependencies * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Remove see also reference to external collections * updated zos_copy module RST * Sample playbook for latest version * Updated ZOAU PTF information with the latest PTF UI70435 * Updates to raw doc to handle the error - Definition list ends without a blank line * Update copy module docs * Update and add clarity around Rocket Python * Requirements order change Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update conf for v1.2.0-beta.1 * Fix E231: missing whitespace (#138) * Specify SFTP port for fetch and copy (#141) * Add sftp_port parameter to fetch and copy * Add sftp_port parameter to fetch and copy * Fix spelling * Missing positional arguments * Test case for negative port specification * Check for valid port range * Remove redundant assertions Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/dependency discovery (#142) * add host setup playbook * handle error cases * restructure and improve robustness of host-setup * add max_search_depth variable * change file extension for consistency * add documentation for host-setup.yaml * add in-file docs for tasks files * add important note for task files * Remove redundant data_set_util * Move error classes to the bottom * Rename backup_file to backup_name (#144) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/zos_copy record format (#147) * Update uss2mvs copy format * Default PS allocation should be FB * Use ZOAU Copy for sequential data sets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/148/accept all valid data set name characters (#149) * update regex for data set name * add test case for special characters * Add ignore_sftp_stderr parameter for fetch and copy * improve cleanup (#151) * change file extension * add /usr/sbin to default PATH * Use USS cp command to copy to PS * Bugfix: zos operator action query not returning all outstanding messages (#154) * remove ZOAU python API usage * add back ZOAU python API, update parsing regex * Changes to support single source doc site (#155) * Fix E231: missing whitespace * Changes to support single source doc site * Under indentation * Under indentation 2 * doc review * doc review of additional files * Update the shell scripts to create a single requirements RST Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * add verbose support and steplib fix (#156) * udpate maxdepth for RST to 1 * Use default discovered port for sftp * Content/3397/update docs (#158) * Fix E231: missing whitespace * Changes to support single source doc site (#145) * Changes to support single source doc site * Under indentation * Under indentation 2 * doc review * doc review of additional files * Update the shell scripts to create a single requirements RST Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Module known issues and release notes added * Updated doc and sample host config playbook doc * Fix use ASCII quotes instead of Unicode quotes * use ASCII quotes instead of Unicode quotes Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update galaxy to latest doc site and version * Release v1.2.0 beta.2 (#163) * Initial collection commit for v0.0.1 (#1) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * added arg parser to module_utils (#3) * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * Updated job output for improved DD handling (#14) Fixes the job summary output being returned when return_output=false. * Security module updates (#15) * patched security holes * updated test cases * removed unnecessary import * updated execution bits * Update modules and documentation corrections (#17) * update documentation, remove unused imports * updated requirements * updated module imports, formatting, and spelling * doc update and formatting update * z/OS Ping Module Feature Delivered (#18) * added zos_ping, test, and test fixture update * Description completeness * Grammar completeness * Pulled repetitive code into functions * updated module docs * Minor spelling corrections * Repo updates to satisfy ansible-test sanity testing requirements (#19) * update formatting * formatting changes to appease ansible-test * update tests * fix string format use and quotes * update README * remove trailing whitespace * update group_vars trailing spaces * remove shebang in attempt to fix encoding issue * add back shebang lines * Added security updates and test cases (#20) * add new types for parser, update arg handling zos_job_submit * updated module argument parsing and security tests * uncommented security tests * updated README to reflect use of MIT license * update hash function * test and formatting updates * updated documentation to pass ansible-test * switch name back to normal.py, add appropriate ignores * update ignore files * added ignore for GPL * updated README, .gitignore and removed MIT code * remove broken tests * increase robustness of zos_job_query * make job query even more robust * Webdocs content for IBM z/OS Core Collection (#21) * Initial commit for reStructuredText content * Updates and clean up to the template * Updated grammar * Grammar edit * Grammar fixes * Text updates * Updates to restructured text, fixed link, renamed collection * Fix ibm_zos_core bold * Fixes ansible-test lint: docs/source/conf.py:51:161: E501: line too long (178 > 160 characters) * ansible doc changes to modules for better formatting * remove original_message return parameter (#23) * remove original_message return parameter * Fix reported linting errors from PR * Fix reported linting errors from PR * Fix reported linting errors from PR for zos_job_query.py * Module document updates, Makefile changes and jinja support for defau… (#24) * Module document updates, Makefile changes and jinja support for default:int * Doc updates per linting rules * Update mmodule doc and remove md files (#25) * Removed all MD files from docs directory * Remove README.md , its replaced by README.rst * Fix bolding for ibm_zos_core * Fix highlight for ibm_zos_core * Updated index.rst to correspond to the README.md updates * Updated README.rst to point to webdocs * Updated README.rst webdocs URL * Update the modules to their ansible-doc * Documentation updates (#26) * Updated galaxy.yml to reflect latest version 1.0 * Documentation copyright updates and formatting * Documentation udpates * Update z/OS data set module and add catalog functionality (#27) * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * catalog for VSAM working, bugs fixed, tests added * satisfy ansible-test requirements, minor arg validation updates * removed unit tests * raise min . ansible requirement due to security vulnerability * added documentation to VTOC * Update product name in webdocs (#28) * Update product name in webdocs * Fix trailing newline * Document updates include some template bugs * Integraded document review comments (#29) * Integrated document review comments * Incorrect break for bulleted line * Bug in Jinja template where property would tab to much (#31) * Updates to documentation and galaxy configuration (#32) * Fix to README.rst, duplicate wording (#34) * Remove duplicate action plugins link (#35) * testing readme rendering (#36) Ported the README.rst file to README.md to meet collection requirements. * Corrected README to point to MD, and long tag removed (#39) * Add two additional arg types to BetterArgHandler (#44) * Add two additional arg types * Remove os from os.path.isabs Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Data Set Utility for Common Data Set Related Tasks (#48) * DataSetUtils initial commit * LISTDS and LISTCAT output processing * Raise exception when data set is open for write * Fix LISTDS regex * Fix indentation * Add USS path support * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/832/fix non vsam non sms existence checking zos data set (#49) * updated catalog handling for non-vsam, non-sms * updated some method comment structures * bugfix/822/determine-data-set-type (#50) * Use mvscmd CLI instead of Python API * Fix linting errors * Remove unused variable Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Extend dynamic function support to the better arg parser (#51) * Dynamic argument support for dynamic functions. * improve default handling for reusability of BetterArgParser object. * add test cases for dynamic argument support * removed print statements from test cases * Updates to data-set-utils exceptions (#52) * Avoid raising exception for uncatalogued data sets * verify listcat output for uncatalogued datasets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Removed return block from documentation (#53) Removed message and changed, empty return block now * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * Fixed PATH (#55) * Update/973/add new supported data set characters betterargparser (#56) * updated valid data set types builtin type name fix * added test cases for data set types * Bugfix when checking if data set exists (#57) * Return False instead of None when data set doesn't exist * Test for membership should ne 'not in' Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Committing module zos_operator_action_query (#43) * Reviewd changes for feature zos_operator_action_query.py * Fix linting errors reported in PR * Linting corrections * fix the syntax error * fix lint errors * fix lint error * fix lint error * fix lint error * fix function test * Added zos_operator_action_query to the sanity exemption list for lack of GPLV3 Lic * Update author tag * Update the collection reference * add function test(need revise later) & doc update * updated regex and value capturing * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * updated docs, made environment agnostic test cases * added support for "*" and fixed splat handling * updated test cases * updated regex matching method, updates tests * Update short description Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Check for data set member existence (#58) * Check for data set member existence * Resolve linting issue Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix for job name capture group (#60) * Better ZOAU import missing messages (#61) * Better ZOAU import missing messages * added missing license and imports * Changed behavior on failure for zosoperator_action_query (#63) * changed behavior on failure * update tests for new expected output * Bugfix 1661 & 1704 fix query of job for different user zos_job_submit and zos_job_action_query (#64) * updates to fix job submission as other user * updated and optimized job utility * removed usage of ZOAU and updated jobs utility usage * update job_output return structure * improved error messages * add job log cleanup for test cases * Initial module zos_tso_command (#46) * Linting corrections * Update doc so html will generate * Lintig updates * Lintig updates v2 * Lintig updates v3 * Lintig updates v4 * Lintig updates v5 * Lintig updates v6 * Lintig updates v7 * Lintig updates v8 * Delete test_zos_tso_command_unit.py Remove the unit test as it only mocks things. * Documenation updates * Update errro message * zos-tso-command update * linting update * testcase linting updates * linting update * linting update * linting update * linting update * linting updates * linting update * new authorized long command handling method * linting update * test cases update * linting update * linting update * linting update * tso-command new version * tso-command linting update * linting update * linting update * linting update' * linting removal * lint removal * linting testcases Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> * Update module_utils with an encode utility (#59) * initial commmit for encode_utils * support for string encoding conversion * Support MVS data set conversion and string conversion modification * Recursively copy to PDS fix * Linting correction * Linting updates * Linting updates 01 * fix ENOENT problem after shutil.move * updates with zos_encode * Merged branch and commit for zos_encode (#65) * Merged branch and commit for zos_encode * Lint error , trailing white space * z/OS Fetch Module (#7) * initial commit * begin updating examples * zos_fetch module * return rc, stdout, stderr as separate return parameters on failure * untrack non-fetch related files * add rest of the files * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * added unit and functional tests * check file existence when LISTDS returns invalid dataset name * added 'wait_s' parameter * clarified description of encoding parameter * make use_qaulifer false by default * remove is_uss and os_vsam parameters * remove data_set directory * remove is_uss parameter from examples * message return block * Fix YAML formatting * return code formatting * remove fail_on_missing from action plugin * fail if USS file does not exist * data sets are assumed to be cataloged * instead of entire dict, pass individual values * redundant code when fetching data set member * rename transfer_from_uss to fetch_non_partitioned_data_set * remove test cases for uncataloged datasets * import naming conflict * add __future__ imports * add __future__ imports * assuming all data sets are cataloged * assuming all data sets are cataloged * fix formatting issues * fix formatting issues * remove blank lines with whitespaces * change example member name data to DATA for accuracy. * continuation line unindented for visual indent * conform to PEP8 standard * avoid automatic field numbering for format * specify python interpreter * extract only data set name if member present * Remove newline character from error message * Update RETURN block with sample message * Remove blank line from return block * Update version_added to correspond to Ansible version * Check if temp datasets are being created successfully * Datasets.read() returns None for empty data sets * Use BetterArgParser to parse module args * Added parameter types * Remove trailing whitespaces * Avoid passing extra args from action plugin * Remove unused imports * Ignore lack of GPLv3 license complaint * catch ValueError thrown ny BetterArgParser * Blank line contains whitespace * Reduce line lengths to 80 * Add white space around string addition * Checking volume type no longer required * Check if src or dest is provided first * Fix indentation * Indent multiline string for better readability * Use DataSetUtils to get data set type * Use DataSetUtils to determine data set existence * Remove unit tests * Fix linting * Add test case for empty data sets * default params for _fail_json * Test case for missing source * Test case for insufficient write permission * Test case for insufficient desgt write permission * Catch exceptions while executing module * Use updated DataSetutils constructor * Updated functional tests * Fix linting errors * Use ansible fetch module to fetch USS files * Checking for USS type is unnecessary * Use sha256 instead of sha1 for checksum validation * Remove message return block * Missing uss file test case * Minor cleanup * Add stdout_lines and stderr_lines to module response * Update action plugin with stderr_lines and stdout_lines * YAML syntax issue * Corrected spelling errors * Inaccurate description of flat * No need to wrap error messages returned by DataSetUtils * Update doc regarding HFS and ZFS support * Handle encoding for USS files * Add encoding to doc * Calculate temp file size for VSAM storage * Preserve encoding using SFTP * Resolve PEP8 issues * Check for data set member existence * raw binary not supported in PY2 * Fix linting issues * Change doc to reflect USS absolute path * Prepend dataset hlq before parameter verification * src not declared before being referenced * Add test case using use_qualifier * Updated doc link for fetch L(..) and suboptions * Use tempfile.mkstemp() for temp data storage * Remove default for encoding * default encoding values are None * Create destination dirs when flat is false * Add test case for flat hostname * Fix spelling * exist_ok is not valid keyword for PY2 * Detect read-only file system * Handle sftp errors * Capture module_stderr in case of module failure * Ensure dest is a directory for PDS transfer * Ensure dest is a directory except when fetching a member * Fix data set member destination path * Default values for encoding is not needed * Remove unused imports * Update 'msg' description * Clean up temp files in case of module failure * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * enhanced zoau import error message (#66) * enhanced zoau import error message * satisfy pep8 witespace requirement * Added new playbook for latest module collection and a filter * fixed linting issues, added license * updated spacing to conform to formatting standards * updated shell to command where possible * Use encode util to convert encoding (#68) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update RST and MD doc files to reflect next release * Update group vars to reflect zoau 1.0.2 new env vars * Update encode short description to fit within the limits of doc length * Update makefile for more generic use by others * Update to fix RST lint error , short line * Rename sub-dir from plugin to filter, doc error * doc tree correction * Update release notes to correspond to beta release tag * Update sample playbooks with doc, configuration and new variables * Upate docs to highlight users to review sample playbooks and JCL before running * Update sample JCL to run on my systems with default values and notes * Add supporting doc for behavioral inventory parameters and ansible.cfg * Fix the md style URL in the index RST file * Update short descriptions for modules * Fix README.md link to IMS collection * Fix README.md link to collection artifacts * Linting updates * Add an example that requires a command to be escaped (#71) * refine backup_file & put it into encode_utils (#70) * refine backup_file & put it into encode_utils * correction for sanity test * refactor for security and usability, fix test case * move copy logic to utility * refactor zos_encode, improve backup functionality Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Update galaxy.yml to reflect the latest version and authors * Fix import name change resulting from refactor * updated usage of encode_utils to encode * Documentation updates, corrections and alterations (#75) * Documentation updates, corrections and alterations * ID review-round 2 * Update the user description in the doc Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Content/81/zos tso command docreview (#73) * ID review of zos-tso-commands * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugi…
ddimatos
added a commit
that referenced
this pull request
Oct 6, 2020
* original blockinfile with few tweaks to work on USS files * initial check in for dataset support * changed zos_blockinfile to use ZOAU dmod * initial check-in for zos_blockinfile functional test cases * added block replace test cases; cleaned up doc * added RETURN doc * lint fix; examples; func test env setup update * doc review of zos_blockinfile * adapted to new API; renamed backup_file to backup_name * Release v1.2.0 beta.4 (#185) * Initial collection commit for v0.0.1 (#1) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * added arg parser to module_utils (#3) * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * Updated job output for improved DD handling (#14) Fixes the job summary output being returned when return_output=false. * Security module updates (#15) * patched security holes * updated test cases * removed unnecessary import * updated execution bits * Update modules and documentation corrections (#17) * update documentation, remove unused imports * updated requirements * updated module imports, formatting, and spelling * doc update and formatting update * z/OS Ping Module Feature Delivered (#18) * added zos_ping, test, and test fixture update * Description completeness * Grammar completeness * Pulled repetitive code into functions * updated module docs * Minor spelling corrections * Repo updates to satisfy ansible-test sanity testing requirements (#19) * update formatting * formatting changes to appease ansible-test * update tests * fix string format use and quotes * update README * remove trailing whitespace * update group_vars trailing spaces * remove shebang in attempt to fix encoding issue * add back shebang lines * Added security updates and test cases (#20) * add new types for parser, update arg handling zos_job_submit * updated module argument parsing and security tests * uncommented security tests * updated README to reflect use of MIT license * update hash function * test and formatting updates * updated documentation to pass ansible-test * switch name back to normal.py, add appropriate ignores * update ignore files * added ignore for GPL * updated README, .gitignore and removed MIT code * remove broken tests * increase robustness of zos_job_query * make job query even more robust * Webdocs content for IBM z/OS Core Collection (#21) * Initial commit for reStructuredText content * Updates and clean up to the template * Updated grammar * Grammar edit * Grammar fixes * Text updates * Updates to restructured text, fixed link, renamed collection * Fix ibm_zos_core bold * Fixes ansible-test lint: docs/source/conf.py:51:161: E501: line too long (178 > 160 characters) * ansible doc changes to modules for better formatting * remove original_message return parameter (#23) * remove original_message return parameter * Fix reported linting errors from PR * Fix reported linting errors from PR * Fix reported linting errors from PR for zos_job_query.py * Module document updates, Makefile changes and jinja support for defau… (#24) * Module document updates, Makefile changes and jinja support for default:int * Doc updates per linting rules * Update mmodule doc and remove md files (#25) * Removed all MD files from docs directory * Remove README.md , its replaced by README.rst * Fix bolding for ibm_zos_core * Fix highlight for ibm_zos_core * Updated index.rst to correspond to the README.md updates * Updated README.rst to point to webdocs * Updated README.rst webdocs URL * Update the modules to their ansible-doc * Documentation updates (#26) * Updated galaxy.yml to reflect latest version 1.0 * Documentation copyright updates and formatting * Documentation udpates * Update z/OS data set module and add catalog functionality (#27) * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * catalog for VSAM working, bugs fixed, tests added * satisfy ansible-test requirements, minor arg validation updates * removed unit tests * raise min . ansible requirement due to security vulnerability * added documentation to VTOC * Update product name in webdocs (#28) * Update product name in webdocs * Fix trailing newline * Document updates include some template bugs * Integraded document review comments (#29) * Integrated document review comments * Incorrect break for bulleted line * Bug in Jinja template where property would tab to much (#31) * Updates to documentation and galaxy configuration (#32) * Fix to README.rst, duplicate wording (#34) * Remove duplicate action plugins link (#35) * testing readme rendering (#36) Ported the README.rst file to README.md to meet collection requirements. * Corrected README to point to MD, and long tag removed (#39) * Add two additional arg types to BetterArgHandler (#44) * Add two additional arg types * Remove os from os.path.isabs Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Data Set Utility for Common Data Set Related Tasks (#48) * DataSetUtils initial commit * LISTDS and LISTCAT output processing * Raise exception when data set is open for write * Fix LISTDS regex * Fix indentation * Add USS path support * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/832/fix non vsam non sms existence checking zos data set (#49) * updated catalog handling for non-vsam, non-sms * updated some method comment structures * bugfix/822/determine-data-set-type (#50) * Use mvscmd CLI instead of Python API * Fix linting errors * Remove unused variable Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Extend dynamic function support to the better arg parser (#51) * Dynamic argument support for dynamic functions. * improve default handling for reusability of BetterArgParser object. * add test cases for dynamic argument support * removed print statements from test cases * Updates to data-set-utils exceptions (#52) * Avoid raising exception for uncatalogued data sets * verify listcat output for uncatalogued datasets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Removed return block from documentation (#53) Removed message and changed, empty return block now * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * Fixed PATH (#55) * Update/973/add new supported data set characters betterargparser (#56) * updated valid data set types builtin type name fix * added test cases for data set types * Bugfix when checking if data set exists (#57) * Return False instead of None when data set doesn't exist * Test for membership should ne 'not in' Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Committing module zos_operator_action_query (#43) * Reviewd changes for feature zos_operator_action_query.py * Fix linting errors reported in PR * Linting corrections * fix the syntax error * fix lint errors * fix lint error * fix lint error * fix lint error * fix function test * Added zos_operator_action_query to the sanity exemption list for lack of GPLV3 Lic * Update author tag * Update the collection reference * add function test(need revise later) & doc update * updated regex and value capturing * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * updated docs, made environment agnostic test cases * added support for "*" and fixed splat handling * updated test cases * updated regex matching method, updates tests * Update short description Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Check for data set member existence (#58) * Check for data set member existence * Resolve linting issue Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix for job name capture group (#60) * Better ZOAU import missing messages (#61) * Better ZOAU import missing messages * added missing license and imports * Changed behavior on failure for zosoperator_action_query (#63) * changed behavior on failure * update tests for new expected output * Bugfix 1661 & 1704 fix query of job for different user zos_job_submit and zos_job_action_query (#64) * updates to fix job submission as other user * updated and optimized job utility * removed usage of ZOAU and updated jobs utility usage * update job_output return structure * improved error messages * add job log cleanup for test cases * Initial module zos_tso_command (#46) * Linting corrections * Update doc so html will generate * Lintig updates * Lintig updates v2 * Lintig updates v3 * Lintig updates v4 * Lintig updates v5 * Lintig updates v6 * Lintig updates v7 * Lintig updates v8 * Delete test_zos_tso_command_unit.py Remove the unit test as it only mocks things. * Documenation updates * Update errro message * zos-tso-command update * linting update * testcase linting updates * linting update * linting update * linting update * linting update * linting updates * linting update * new authorized long command handling method * linting update * test cases update * linting update * linting update * linting update * tso-command new version * tso-command linting update * linting update * linting update * linting update' * linting removal * lint removal * linting testcases Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> * Update module_utils with an encode utility (#59) * initial commmit for encode_utils * support for string encoding conversion * Support MVS data set conversion and string conversion modification * Recursively copy to PDS fix * Linting correction * Linting updates * Linting updates 01 * fix ENOENT problem after shutil.move * updates with zos_encode * Merged branch and commit for zos_encode (#65) * Merged branch and commit for zos_encode * Lint error , trailing white space * z/OS Fetch Module (#7) * initial commit * begin updating examples * zos_fetch module * return rc, stdout, stderr as separate return parameters on failure * untrack non-fetch related files * add rest of the files * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * added unit and functional tests * check file existence when LISTDS returns invalid dataset name * added 'wait_s' parameter * clarified description of encoding parameter * make use_qaulifer false by default * remove is_uss and os_vsam parameters * remove data_set directory * remove is_uss parameter from examples * message return block * Fix YAML formatting * return code formatting * remove fail_on_missing from action plugin * fail if USS file does not exist * data sets are assumed to be cataloged * instead of entire dict, pass individual values * redundant code when fetching data set member * rename transfer_from_uss to fetch_non_partitioned_data_set * remove test cases for uncataloged datasets * import naming conflict * add __future__ imports * add __future__ imports * assuming all data sets are cataloged * assuming all data sets are cataloged * fix formatting issues * fix formatting issues * remove blank lines with whitespaces * change example member name data to DATA for accuracy. * continuation line unindented for visual indent * conform to PEP8 standard * avoid automatic field numbering for format * specify python interpreter * extract only data set name if member present * Remove newline character from error message * Update RETURN block with sample message * Remove blank line from return block * Update version_added to correspond to Ansible version * Check if temp datasets are being created successfully * Datasets.read() returns None for empty data sets * Use BetterArgParser to parse module args * Added parameter types * Remove trailing whitespaces * Avoid passing extra args from action plugin * Remove unused imports * Ignore lack of GPLv3 license complaint * catch ValueError thrown ny BetterArgParser * Blank line contains whitespace * Reduce line lengths to 80 * Add white space around string addition * Checking volume type no longer required * Check if src or dest is provided first * Fix indentation * Indent multiline string for better readability * Use DataSetUtils to get data set type * Use DataSetUtils to determine data set existence * Remove unit tests * Fix linting * Add test case for empty data sets * default params for _fail_json * Test case for missing source * Test case for insufficient write permission * Test case for insufficient desgt write permission * Catch exceptions while executing module * Use updated DataSetutils constructor * Updated functional tests * Fix linting errors * Use ansible fetch module to fetch USS files * Checking for USS type is unnecessary * Use sha256 instead of sha1 for checksum validation * Remove message return block * Missing uss file test case * Minor cleanup * Add stdout_lines and stderr_lines to module response * Update action plugin with stderr_lines and stdout_lines * YAML syntax issue * Corrected spelling errors * Inaccurate description of flat * No need to wrap error messages returned by DataSetUtils * Update doc regarding HFS and ZFS support * Handle encoding for USS files * Add encoding to doc * Calculate temp file size for VSAM storage * Preserve encoding using SFTP * Resolve PEP8 issues * Check for data set member existence * raw binary not supported in PY2 * Fix linting issues * Change doc to reflect USS absolute path * Prepend dataset hlq before parameter verification * src not declared before being referenced * Add test case using use_qualifier * Updated doc link for fetch L(..) and suboptions * Use tempfile.mkstemp() for temp data storage * Remove default for encoding * default encoding values are None * Create destination dirs when flat is false * Add test case for flat hostname * Fix spelling * exist_ok is not valid keyword for PY2 * Detect read-only file system * Handle sftp errors * Capture module_stderr in case of module failure * Ensure dest is a directory for PDS transfer * Ensure dest is a directory except when fetching a member * Fix data set member destination path * Default values for encoding is not needed * Remove unused imports * Update 'msg' description * Clean up temp files in case of module failure * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * enhanced zoau import error message (#66) * enhanced zoau import error message * satisfy pep8 witespace requirement * Added new playbook for latest module collection and a filter * fixed linting issues, added license * updated spacing to conform to formatting standards * updated shell to command where possible * Use encode util to convert encoding (#68) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update RST and MD doc files to reflect next release * Update group vars to reflect zoau 1.0.2 new env vars * Update encode short description to fit within the limits of doc length * Update makefile for more generic use by others * Update to fix RST lint error , short line * Rename sub-dir from plugin to filter, doc error * doc tree correction * Update release notes to correspond to beta release tag * Update sample playbooks with doc, configuration and new variables * Upate docs to highlight users to review sample playbooks and JCL before running * Update sample JCL to run on my systems with default values and notes * Add supporting doc for behavioral inventory parameters and ansible.cfg * Fix the md style URL in the index RST file * Update short descriptions for modules * Fix README.md link to IMS collection * Fix README.md link to collection artifacts * Linting updates * Add an example that requires a command to be escaped (#71) * refine backup_file & put it into encode_utils (#70) * refine backup_file & put it into encode_utils * correction for sanity test * refactor for security and usability, fix test case * move copy logic to utility * refactor zos_encode, improve backup functionality Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Update galaxy.yml to reflect the latest version and authors * Fix import name change resulting from refactor * updated usage of encode_utils to encode * Documentation updates, corrections and alterations (#75) * Documentation updates, corrections and alterations * ID review-round 2 * Update the user description in the doc Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Content/81/zos tso command docreview (#73) * ID review of zos-tso-commands * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * Fix indentation Co-authored-by: ddimatos <dimatos@gmail.com> * Fix extra space in doc * fixed zos_ping action plugin issues (#77) * Add out-of-box awx support (#80) * Added encoding supplemental doc (#78) * Added encoding supplemental doc * Removed references to modules not available and causing doc warnings * Fix Inline literal start-string without end-string. * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Add binary support copying across USS and MVS * Copy USS file in binary mode * fixed bug and cleaned up code (#82) * fixed bug and cleaned up code * fixed linting issues * Avoid specifying -B and -F for cp * Copy MVS source to MVS dest * Documentation web content updates (#84) * First round of review of static docs * docs/source/installation.rst * Update block to rendor with syntax highlights Code block was using 'yaml' when it was 'sh' , renders better if we have the correct code block type. * Incorporated Demetri's review comments Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Replace only the first occurance of 'rec' * Minor updates in install,quickstart, and req pages (#86) * Updated make file with comments and recover from failure * Update Backup util (mvs_file_backup) (#85) * Alternative mvs_file_backup * fix backup logic, add encode backup tests * add mvs backup tests, util fixes * fix test for new logic path Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Udpate the restructured text to reflect version release (#89) * minor fixes and additional tests for zos_encode (#88) * removed Jobs.submit usage due to invalid response (#92) * Update to galaxy.yml to exclude files * Update makefile to support rexx written modules * Generate and push zos_ping rst doc * Corrected clean up of a directory in makefile * Fixes spacing in RETURN values and formatting (#97) * Validate bk_dsn after assigning a value * Return backup path * Fix alloc command * Remove trailing whitespace * Fix job utility to work on all z/OS systems (#99) * Shinx and sphinx-versioning support and updates (#100) * Updated Makefile to support sphinx-versioning and renamed some of the operations * Update configuration to support sphinx-versioning and update options for sphinx * Update configuration with linting corrections * Correct and update the installation of a pre-release (#102) * Correct and update the installation of a pre-release * minor fixes to docs Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update data set utils (#104) * Update data set utils * Update function signatures * Module parameter not needed for encode util Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Change default encoding from UTF-8 to ISO8859-1 * Fetching non-existent VSAM * Update doc for encoding parameter * Update encoding to handle different charsets * Remove encoding default from argument spec * encoding validation and doc update * Fix typo * Bugfix/case conversion fix zos tso command (#108) * fixed issues with commands forced to uppercase * add support for single or multi commands, added relevant test cases * updated alias doc * Added index file to support shared doc site (#109) * Remove module parameter from EncodeUtils constructor * fix issue when no default encoding specified (#112) * fix issue when no default encoding specified * fix local job submission * Feature/26/ansible zos copy module (#105) * initial commit * begin updating examples * default value 'yes/no' -> 'true/false' * updated documentation * copy to seq data set * copy local to seq data set * add data set size to module response * functional test cases for copy * test case for sequential data set * clean up copy to ps function * copy to pdse * assuming all data sets are cataloged * copy to PDS * Copy to VSAM * Copy to PDS(E) member * Actual directory size instead of max file size * Copy from remote source * Update copy * Reduce line lengths to less than 80 * Use mvscmd cli * remote pdse copy * IEBCOPY to copy data sets * saving changes * Create remote data set if it doesn't exist * Reorganize remote copy transfer * Update return params * Avoid using global module * Remove unused imports * Copy to USS * Clean up in case of module failure * Back up USS file * Copy content to remote file * Copy to sequential data set * Copy local dir to PDS * Copy remote PDS to PDS * Update backup file * USS dest not accessible if not present * Explicitly tag destination file * Changed file mode, group and owner * Change common file args for remote src * Change file common args after copying to dest * Preserve local file mode * Convert encoding for remote src * Added comments * Use backup util to back up files and data sets * Add validate parameter * Copy VSAM to VSAM * Support for symlinks * Copy PDS/PDSE member to USS * Fix formatting * Module should fail for empty VSAM * Added function signatures for all test cases * User better_arg_parser to determine member type * Add binary support for copying to USS target * Copy to PS in binary mode * Correct object instantiations * Import copy util * Remove write_rc * Resolve import issues * Move vsam copy handler to main copy handler class * Copy local dir to USS dir * Infer member name when it isn't provided * Added test cases for local and remote copy * src_member evalutes to True * Comments for extract_dsname() * IEBCOPY fails for incompatible recfm * Copy PDS member to PS * Copy data set member to non-existent PS * Copy local symlink * Add test case for uss file backup * Fix mvs file backup * Functional tests complete * Fix linting issues * Remove trailing whitespace * Ignore GPLv3 license * Remove is_vsam parameter * Fix license information * Fail action plugin * Add test cases to ensure module failure * Remove unneded parameters * Refactor to use updated data_set util * I/O abend when copying PDSE member to non-existent PS * Updated doc, comments, linting * Fix linting issues * Remove duplicate key from dic * Avoid hard-coding tmp path name * Change backup_path parameter to backup_file * Copy to data set member partial * Error converting encoding for local dir * Fix tmp cleanup bug * Updated test cases for backup_file * plugins/modules/zos_copy.py * Reactor code for better readability * Fix pep8 issue * Cleanup error on module failure * plugins/modules/zos_copy.py * Doc update mode parameter * Use native shell instead of bash for functional testing * Return invocation parameter * Remove unnecessary imports * plugins/modules/zos_copy.py * Return backup_file on module failure Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Feature/1089/zos lineinfile (#111) * initial version for interface and options * add backup option * update lineinfile * make golden path work * fix the bugs so that method could support none vars * refactor based on new dsed * updated params and return * Initial check in for backup feature support * support all charsets; updated backup feature doc * clean up; used better_arg_parser * use latest data_set module_util * change the returned string to be loadable by json.loads (BOF/EOF) * uss test cases * dataset test cases; create a dataset/file with content instead of being dependent on existing source * uss test env set bug fix * added UTF-8 in test cases; simplified dataset names * use new dedicated ZOAU function for lineinfile * added assert for expected strings * encoding parameter from absent * Linting updates from pipeline tests * Linting updates and corrections * Linting udpates continued * Update sanity exemption file and linting corrections * Doc error, indentation off, breaking rest of doc parse * module arg and doc updates * Formatting updates * Further doc updates * regexp alias fix * updated docs; added comments; updated parameters * fix sanity checks * fix RETURN key type; lint error * code review changes * fixed linting issues * removed incorrectly added choices for insertafter/insertbefore * added negative test cases fo not supported datasets * moved choices to description; fixed test cases lint issue * changed the choices format in description * add escape if string was quoted * added test cases for escaped/unescaped quotation * used 'src' for target location; moved the params checking before taking backup * fixed indentation * Doc edits for lineinfile, encode, and fetch * remove relative import method * removed the default values for insertafter/insertbefore * renamed helper functions; updated pytest.ini for test cases; fixed TEST_INFO * remove jcl tmp directory if it already exists * run all test_* Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: behnam-ibm <behnam@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/1067/community doc (#113) * Correct doc link to IMS collection to point to Galaxy * Added testing doc to community guidlines * Clean/delete accidental file * Fix Sphinx warning coming from ping module reference to zos_ssh plugin undefined label: zos_ssh_module * Added better arg parse to new dev addition to community guides * Fixes broken site anchor to the YAML configuration * Fix MD style Link * Remove additional MD style links * Fix mailformed RST link * Fix more RST links * Python syntax fixes in samples * Update betterArgParse to BetterArgParser * Expand pipelining=true doc * Correct Unicode quotes to ASCII quotes * Statement of direction on zoau and python * doc review of community guidelines * Doc review of Community guidelines * Fix malformed RST table * Update group vars requirements needed by IBM Pypthon * Update doc with var settings and sample all.yml * Update environ vars * Added environ LANG doc * Added doc to clarify shell support * Added advanced group_vars sample and doc * Update docs to reflect IBM Python announcement Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Remove unnecessary description of PDSE transfer * Update version tag in doc generation conf * Add missing link reference * Doc review (#118) Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update IBM Python download link * Add system info to job query (#127) * Add system info to job query * Change sysname to system for job query Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Close file descriptor (#128) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/2298/zos raw (#120) * add utilities for zos_raw * updated DD statement to better handle concatenations * update parameter names * fix concat naming * refactor of zos_data_set, add new features * updated examples * added steplib dd type * fix steplib data set * update valid concat members * fix datasetdefinition for steplibs * update dd statement types * argument parsing working in zos_raw * module argument parsing updates * updated valid arguments for consistency * update tests for new zos_data_set format * zos_raw module working * module doc updates * updated function documentation * backup, reuse and replace working * updated documentation * cleaned up wrapper classes * updated existence checking, function doc updates * spelling fixes, added new parameters for dd_unix * add list of string support for StdinDefinition * properly handle space units * initial set of test cases, added aliases for program, rename args to parms to match jcl naming conventions * added remaining happy path data set DD tests, added input dd tests. Fixed small bug in raw related to invalid parameter options * update example playbooks to account for new zos_data_set arguments * added initial functional tests for unix file dd statements * added access group support, fixed status group * added remaining tests for dd_unix and dd_dummy * fixed bug on dd_concat contents or backup returned * add dd_concat test cases * update parms to parm to match jcl naming * added execution tests for zos_raw * add final test cases * add special type functions for ksds related params * update import handling for special cases * add VIO DD support * uncomment test cases * fix basic linting issues, default arguments * add support for directory blocks * fix some linting issues * Revert "fix some linting issues" This reverts commit 44d0e1457d76e8c5e5725b457f525d76afb1111f. * linting fixes * module doc and linting updates * additional doc fixes for linting sanity tests * added examples * added catch for nonexistent deletion * updated docs for proper formatting * update ignore files for new module * added dd_output support, standardized dd names * removed unnecessary import wrappers * improved validation for encoding types. * added new example with dd_output type * Merge branch 'origin/dev' * Revert "Merge branch 'origin/dev'" This reverts commit 1befb8e8f0a2448cb9408d94c3cf6a64a2ff9e4c. * fixes for python 2 controller support * modify encode utils import * add utf-8 encoding tag to unit tests * improve check mode support * fix mock of AnsibleModule for encoding check cases * fix unit test formatting for pep8 * fix test case helpers * comment out failing test case unrelated to zos_raw * update encoding conversion handling * update rexx variables to capture correct environment variables * update naming based on discussion * rename test case files, update test cases * update default for output data set * update test ignore files for rename * doc edits (#133) * doc edits * fix minor pep8 issues Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * add python 2 support * remove redundant code comments * add temp data set support, backwards compatibility fix with old interface, update docs and tests * update documentation for return_content * minor doc changes for clarity * improve documentation clarity * fix incorrect syntax for option name Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Update configuration vars in doc and yml (#131) * remove ZOAU Python API deps for zos_operator (#132) * remove ZOAU deps which break in 1.0.3 * add back shebang * remove unsafe shell usage * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/2516/release doc update (#136) * Update the version in galaxy.yml * edits to support doc gen * Updated for doc gen * updated all module doc for release * Update template to upper case boolean when no default present * update raw module doc to fomat as the indentation intended * updated module RST * add details to module enhancements * Update message around Rocket's python * Update Rocket Python Message * Reference Rocket Python configuration in requirements * Updated doc to reflect the latest dependencies * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Remove see also reference to external collections * updated zos_copy module RST * Sample playbook for latest version * Updated ZOAU PTF information with the latest PTF UI70435 * Updates to raw doc to handle the error - Definition list ends without a blank line * Update copy module docs * Update and add clarity around Rocket Python * Requirements order change Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update conf for v1.2.0-beta.1 * Fix E231: missing whitespace (#138) * Specify SFTP port for fetch and copy (#141) * Add sftp_port parameter to fetch and copy * Add sftp_port parameter to fetch and copy * Fix spelling * Missing positional arguments * Test case for negative port specification * Check for valid port range * Remove redundant assertions Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/dependency discovery (#142) * add host setup playbook * handle error cases * restructure and improve robustness of host-setup * add max_search_depth variable * change file extension for consistency * add documentation for host-setup.yaml * add in-file docs for tasks files * add important note for task files * Remove redundant data_set_util * Move error classes to the bottom * Rename backup_file to backup_name (#144) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/zos_copy record format (#147) * Update uss2mvs copy format * Default PS allocation should be FB * Use ZOAU Copy for sequential data sets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/148/accept all valid data set name characters (#149) * update regex for data set name * add test case for special characters * Add ignore_sftp_stderr parameter for fetch and copy * improve cleanup (#151) * change file extension * add /usr/sbin to default PATH * Use USS cp command to copy to PS * Bugfix: zos operator action query not returning all outstanding messages (#154) * remove ZOAU python API usage * add back ZOAU python API, update parsing regex * Changes to support single source doc site (#155) * Fix E231: missing whitespace * Changes to support single source doc site * Under indentation * Under indentation 2 * doc review * doc review of additional files * Update the shell scripts to create a single requirements RST Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * add verbose support and steplib fix (#156) * udpate maxdepth for RST to 1 * Use default discovered port for sftp * Content/3397/update docs (#158) * Fix E231: missing whitespace * Changes to support single source doc site (#145) * Changes to support single source doc site * Under indentation * Under indentation 2 * doc review * doc review of additional files * Update the shell scripts to create a single requirements RST Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Module known issues and release notes added * Updated doc and sample host config playbook doc * Fix use ASCII quotes instead of Unicode quotes * use ASCII quotes instead of Unicode quotes Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update galaxy to latest doc site and version * Release v1.2.0 beta.2 (#163) * Initial collection commit for v0.0.1 (#1) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * added arg parser to module_utils (#3) * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * Updated job output for improved DD handling (#14) Fixes the job summary output being returned when return_output=false. * Security module updates (#15) * patched security holes * updated test cases * removed unnecessary import * updated execution bits * Update modules and documentation corrections (#17) * update documentation, remove unused imports * updated requirements * updated module imports, formatting, and spelling * doc update and formatting update * z/OS Ping Module Feature Delivered (#18) * added zos_ping, test, and test fixture update * Description completeness * Grammar completeness * Pulled repetitive code into functions * updated module docs * Minor spelling corrections * Repo updates to satisfy ansible-test sanity testing requirements (#19) * update formatting * formatting changes to appease ansible-test * update tests * fix string format use and quotes * update README * remove trailing whitespace * update group_vars trailing spaces * remove shebang in attempt to fix encoding issue * add back shebang lines * Added security updates and test cases (#20) * add new types for parser, update arg handling zos_job_submit * updated module argument parsing and security tests * uncommented security tests * updated README to reflect use of MIT license * update hash function * test and formatting updates * updated documentation to pass ansible-test * switch name back to normal.py, add appropriate ignores * update ignore files * added ignore for GPL * updated README, .gitignore and removed MIT code * remove broken tests * increase robustness of zos_job_query * make job query even more robust * Webdocs content for IBM z/OS Core Collection (#21) * Initial commit for reStructuredText content * Updates and clean up to the template * Updated grammar * Grammar edit * Grammar fixes * Text updates * Updates to restructured text, fixed link, renamed collection * Fix ibm_zos_core bold * Fixes ansible-test lint: docs/source/conf.py:51:161: E501: line too long (178 > 160 characters) * ansible doc changes to modules for better formatting * remove original_message return parameter (#23) * remove original_message return parameter * Fix reported linting errors from PR * Fix reported linting errors from PR * Fix reported linting errors from PR for zos_job_query.py * Module document updates, Makefile changes and jinja support for defau… (#24) * Module document updates, Makefile changes and jinja support for default:int * Doc updates per linting rules * Update mmodule doc and remove md files (#25) * Removed all MD files from docs directory * Remove README.md , its replaced by README.rst * Fix bolding for ibm_zos_core * Fix highlight for ibm_zos_core * Updated index.rst to correspond to the README.md updates * Updated README.rst to point to webdocs * Updated README.rst webdocs URL * Update the modules to their ansible-doc * Documentation updates (#26) * Updated galaxy.yml to reflect latest version 1.0 * Documentation copyright updates and formatting * Documentation udpates * Update z/OS data set module and add catalog functionality (#27) * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * catalog for VSAM working, bugs fixed, tests added * satisfy ansible-test requirements, minor arg validation updates * removed unit tests * raise min . ansible requirement due to security vulnerability * added documentation to VTOC * Update product name in webdocs (#28) * Update product name in webdocs * Fix trailing newline * Document updates include some template bugs * Integraded document review comments (#29) * Integrated document review comments * Incorrect break for bulleted line * Bug in Jinja template where property would tab to much (#31) * Updates to documentation and galaxy configuration (#32) * Fix to README.rst, duplicate wording (#34) * Remove duplicate action plugins link (#35) * testing readme rendering (#36) Ported the README.rst file to README.md to meet collection requirements. * Corrected README to point to MD, and long tag removed (#39) * Add two additional arg types to BetterArgHandler (#44) * Add two additional arg types * Remove os from os.path.isabs Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Data Set Utility for Common Data Set Related Tasks (#48) * DataSetUtils initial commit * LISTDS and LISTCAT output processing * Raise exception when data set is open for write * Fix LISTDS regex * Fix indentation * Add USS path support * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/832/fix non vsam non sms existence checking zos data set (#49) * updated catalog handling for non-vsam, non-sms * updated some method comment structures * bugfix/822/determine-data-set-type (#50) * Use mvscmd CLI instead of Python API * Fix linting errors * Remove unused variable Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Extend dynamic function support to the better arg parser (#51) * Dynamic argument support for dynamic functions. * improve default handling for reusability of BetterArgParser object. * add test cases for dynamic argument support * removed print statements from test cases * Updates to data-set-utils exceptions (#52) * Avoid raising exception for uncatalogued data sets * verify listcat output for uncatalogued datasets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Removed return block from documentation (#53) Removed message and changed, empty return block now * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * Fixed PATH (#55) * Update/973/add new supported data set characters betterargparser (#56) * updated valid data set types builtin type name fix * added test cases for data set types * Bugfix when checking if data set exists (#57) * Return False instead of None when data set doesn't exist * Test for membership should ne 'not in' Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Committing module zos_operator_action_query (#43) * Reviewd changes for feature zos_operator_action_query.py * Fix linting errors reported in PR * Linting corrections * fix the syntax error * fix lint errors * fix lint error * fix lint error * fix lint error * fix function test * Added zos_operator_action_query to the sanity exemption list for lack of GPLV3 Lic * Update author tag * Update the collection reference * add function test(need revise later) & doc update * updated regex and value capturing * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * updated docs, made environment agnostic test cases * added support for "*" and fixed splat handling * updated test cases * updated regex matching method, updates tests * Update short description Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Check for data set member existence (#58) * Check for data set member existence * Resolve linting issue Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix for job name capture group (#60) * Better ZOAU import missing messages (#61) * Better ZOAU import missing messages * added missing license and imports * Changed behavior on failure for zosoperator_action_query (#63) * changed behavior on failure * update tests for new expected output * Bugfix 1661 & 1704 fix query of job for different user zos_job_submit and zos_job_action_query (#64) * updates to fix job submission as other user * updated and optimized job utility * removed usage of ZOAU and updated jobs utility usage * update job_output return structure * improved error messages * add job log cleanup for test cases * Initial module zos_tso_command (#46) * Linting corrections * Update doc so html will generate * Lintig updates * Lintig updates v2 * Lintig updates v3 * Lintig updates v4 * Lintig updates v5 * Lintig updates v6 * Lintig updates v7 * Lintig updates v8 * Delete test_zos_tso_command_unit.py Remove the unit test as it only mocks things. * Documenation updates * Update errro message * zos-tso-command update * linting update * testcase linting updates * linting update * linting update * linting update * linting update * linting updates * linting update * new authorized long command handling method * linting update * test cases update * linting update * linting update * linting update * tso-command new version * tso-command linting update * linting update * linting update * linting update' * linting removal * lint removal * linting testcases Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> * Update module_utils with an encode utility (#59) * initial commmit for encode_utils * support for string encoding conversion * Support MVS data set conversion and string conversion modification * Recursively copy to PDS fix * Linting correction * Linting updates * Linting updates 01 * fix ENOENT problem after shutil.move * updates with zos_encode * Merged branch and commit for zos_encode (#65) * Merged branch and commit for zos_encode * Lint error , trailing white space * z/OS Fetch Module (#7) * initial commit * begin updating examples * zos_fetch module * return rc, stdout, stderr as separate return parameters on failure * untrack non-fetch related files * add rest of the files * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * added unit and functional tests * check file existence when LISTDS returns invalid dataset name * added 'wait_s' parameter * clarified description of encoding parameter * make use_qaulifer false by default * remove is_uss and os_vsam parameters * remove data_set directory * remove is_uss parameter from examples * message return block * Fix YAML formatting * return code formatting * remove fail_on_missing from action plugin * fail if USS file does not exist * data sets are assumed to be cataloged * instead of entire dict, pass individual values * redundant code when fetching data set member * rename transfer_from_uss to fetch_non_partitioned_data_set * remove test cases for uncataloged datasets * import naming conflict * add __future__ imports * add __future__ imports * assuming all data sets are cataloged * assuming all data sets are cataloged * fix formatting issues * fix formatting issues * remove blank lines with whitespaces * change example member name data to DATA for accuracy. * continuation line unindented for visual indent * conform to PEP8 standard * avoid automatic field numbering for format * specify python interpreter * extract only data set name if member present * Remove newline character from error message * Update RETURN block with sample message * Remove blank line from return block * Update version_added to correspond to Ansible version * Check if temp datasets are being created successfully * Datasets.read() returns None for empty data sets * Use BetterArgParser to parse module args * Added parameter types * Remove trailing whitespaces * Avoid passing extra args from action plugin * Remove unused imports * Ignore lack of GPLv3 license complaint * catch ValueError thrown ny BetterArgParser * Blank line contains whitespace * Reduce line lengths to 80 * Add white space around string addition * Checking volume type no longer required * Check if src or dest is provided first * Fix indentation * Indent multiline string for better readability * Use DataSetUtils to get data set type * Use DataSetUtils to determine data set existence * Remove unit tests * Fix linting * Add test case for empty data sets * default params for _fail_json * Test case for missing source * Test case for insufficient write permission * Test case for insufficient desgt write permission * Catch exceptions while executing module * Use updated DataSetutils constructor * Updated functional tests * Fix linting errors * Use ansible fetch module to fetch USS files * Checking for USS type is unnecessary * Use sha256 instead of sha1 for checksum validation * Remove message return block * Missing uss file test case * Minor cleanup * Add stdout_lines and stderr_lines to module response * Update action plugin with stderr_lines and stdout_lines * YAML syntax issue * Corrected spelling errors * Inaccurate description of flat * No need to wrap error messages returned by DataSetUtils * Update doc regarding HFS and ZFS support * Handle encoding for USS files * Add encoding to doc * Calculate temp file size for VSAM storage * Preserve encoding using SFTP * Resolve PEP8 issues * Check for data set member existence * raw binary not supported in PY2 * Fix linting issues * Change doc to reflect USS absolute path * Prepend dataset hlq before parameter verification * src not declared before being referenced * Add test case using use_qualifier * Updated doc link for fetch L(..) and suboptions * Use tempfile.mkstemp() for temp data storage * Remove default for encoding * default encoding values are None * Create destination dirs when flat is false * Add test case for flat hostname * Fix spelling * exist_ok is not valid keyword for PY2 * Detect read-only file system * Handle sftp errors * Capture module_stderr in case of module failure * Ensure dest is a directory for PDS transfer * Ensure dest is a directory except when fetching a member * Fix data set member destination path * Default values for encoding is not needed * Remove unused imports * Update 'msg' description * Clean up temp files in case of module failure * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * enhanced zoau import error message (#66) * enhanced zoau import error message * satisfy pep8 witespace requirement * Added new playbook for latest module collection and a filter * fixed linting issues, added license * updated spacing to conform to formatting standards * updated shell to command where possible * Use encode util to convert encoding (#68) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update RST and MD doc files to reflect next release * Update group vars to reflect zoau 1.0.2 new env vars * Update encode short description to fit within the limits of doc length * Update makefile for more generic use by others * Update to fix RST lint error , short line * Rename sub-dir from plugin to filter, doc error * doc tree correction * Update release notes to correspond to beta release tag * Update sample playbooks with doc, configuration and new variables * Upate docs to highlight users to review sample playbooks and JCL before running * Update sample JCL to run on my systems with default values and notes * Add supporting doc for behavioral inventory parameters and ansible.cfg * Fix the md style URL in the index RST file * Update short descriptions for modules * Fix README.md link to IMS collection * Fix README.md link to collection artifacts * Linting updates * Add an example that requires a command to be escaped (#71) * refine backup_file & put it into encode_utils (#70) * refine backup_file & put it into encode_utils * correction for sanity test * refactor for security and usability, fix test case * move copy logic to utility * refactor zos_encode, improve backup functionality Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Update galaxy.yml to reflect the latest version and authors * Fix import name change resulting from refactor * updated usage of encode_utils to encode * Documentation updates, corrections…
blakeinate
added a commit
that referenced
this pull request
Oct 27, 2020
* Begin doc * age and age_stamp has been documented * Patterns and paths parameter * Parameter definitions complete * Start Examples block * Examples block * Added return block * Initial dev structure * Variable initialization complete * Added wrappers for ZOAU commands * Added filters * data_set and pds filter complete * Added VSAM filter * Return members in module response * Additional return parameters on doc * Add docstrings * Fix pds filter * Code reorganization * Code re-org * Test case for single search string * Test case implemented for single string search * Functional tests complete * Updated test case * Added additional data set types * Support for creation date * Return number of data sets examined * Search in stderr * Validate module doc * doc review of zos_find * Update content * Fix incorrect member list * Merge master into dev which diverged during release v1.2.1 (#189) * Release v1.2.0 beta.4 (#185) * Initial collection commit for v0.0.1 (#1) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * added arg parser to module_utils (#3) * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * Updated job output for improved DD handling (#14) Fixes the job summary output being returned when return_output=false. * Security module updates (#15) * patched security holes * updated test cases * removed unnecessary import * updated execution bits * Update modules and documentation corrections (#17) * update documentation, remove unused imports * updated requirements * updated module imports, formatting, and spelling * doc update and formatting update * z/OS Ping Module Feature Delivered (#18) * added zos_ping, test, and test fixture update * Description completeness * Grammar completeness * Pulled repetitive code into functions * updated module docs * Minor spelling corrections * Repo updates to satisfy ansible-test sanity testing requirements (#19) * update formatting * formatting changes to appease ansible-test * update tests * fix string format use and quotes * update README * remove trailing whitespace * update group_vars trailing spaces * remove shebang in attempt to fix encoding issue * add back shebang lines * Added security updates and test cases (#20) * add new types for parser, update arg handling zos_job_submit * updated module argument parsing and security tests * uncommented security tests * updated README to reflect use of MIT license * update hash function * test and formatting updates * updated documentation to pass ansible-test * switch name back to normal.py, add appropriate ignores * update ignore files * added ignore for GPL * updated README, .gitignore and removed MIT code * remove broken tests * increase robustness of zos_job_query * make job query even more robust * Webdocs content for IBM z/OS Core Collection (#21) * Initial commit for reStructuredText content * Updates and clean up to the template * Updated grammar * Grammar edit * Grammar fixes * Text updates * Updates to restructured text, fixed link, renamed collection * Fix ibm_zos_core bold * Fixes ansible-test lint: docs/source/conf.py:51:161: E501: line too long (178 > 160 characters) * ansible doc changes to modules for better formatting * remove original_message return parameter (#23) * remove original_message return parameter * Fix reported linting errors from PR * Fix reported linting errors from PR * Fix reported linting errors from PR for zos_job_query.py * Module document updates, Makefile changes and jinja support for defau… (#24) * Module document updates, Makefile changes and jinja support for default:int * Doc updates per linting rules * Update mmodule doc and remove md files (#25) * Removed all MD files from docs directory * Remove README.md , its replaced by README.rst * Fix bolding for ibm_zos_core * Fix highlight for ibm_zos_core * Updated index.rst to correspond to the README.md updates * Updated README.rst to point to webdocs * Updated README.rst webdocs URL * Update the modules to their ansible-doc * Documentation updates (#26) * Updated galaxy.yml to reflect latest version 1.0 * Documentation copyright updates and formatting * Documentation udpates * Update z/OS data set module and add catalog functionality (#27) * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * catalog for VSAM working, bugs fixed, tests added * satisfy ansible-test requirements, minor arg validation updates * removed unit tests * raise min . ansible requirement due to security vulnerability * added documentation to VTOC * Update product name in webdocs (#28) * Update product name in webdocs * Fix trailing newline * Document updates include some template bugs * Integraded document review comments (#29) * Integrated document review comments * Incorrect break for bulleted line * Bug in Jinja template where property would tab to much (#31) * Updates to documentation and galaxy configuration (#32) * Fix to README.rst, duplicate wording (#34) * Remove duplicate action plugins link (#35) * testing readme rendering (#36) Ported the README.rst file to README.md to meet collection requirements. * Corrected README to point to MD, and long tag removed (#39) * Add two additional arg types to BetterArgHandler (#44) * Add two additional arg types * Remove os from os.path.isabs Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Data Set Utility for Common Data Set Related Tasks (#48) * DataSetUtils initial commit * LISTDS and LISTCAT output processing * Raise exception when data set is open for write * Fix LISTDS regex * Fix indentation * Add USS path support * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/832/fix non vsam non sms existence checking zos data set (#49) * updated catalog handling for non-vsam, non-sms * updated some method comment structures * bugfix/822/determine-data-set-type (#50) * Use mvscmd CLI instead of Python API * Fix linting errors * Remove unused variable Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Extend dynamic function support to the better arg parser (#51) * Dynamic argument support for dynamic functions. * improve default handling for reusability of BetterArgParser object. * add test cases for dynamic argument support * removed print statements from test cases * Updates to data-set-utils exceptions (#52) * Avoid raising exception for uncatalogued data sets * verify listcat output for uncatalogued datasets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Removed return block from documentation (#53) Removed message and changed, empty return block now * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * Fixed PATH (#55) * Update/973/add new supported data set characters betterargparser (#56) * updated valid data set types builtin type name fix * added test cases for data set types * Bugfix when checking if data set exists (#57) * Return False instead of None when data set doesn't exist * Test for membership should ne 'not in' Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Committing module zos_operator_action_query (#43) * Reviewd changes for feature zos_operator_action_query.py * Fix linting errors reported in PR * Linting corrections * fix the syntax error * fix lint errors * fix lint error * fix lint error * fix lint error * fix function test * Added zos_operator_action_query to the sanity exemption list for lack of GPLV3 Lic * Update author tag * Update the collection reference * add function test(need revise later) & doc update * updated regex and value capturing * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * updated docs, made environment agnostic test cases * added support for "*" and fixed splat handling * updated test cases * updated regex matching method, updates tests * Update short description Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Check for data set member existence (#58) * Check for data set member existence * Resolve linting issue Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix for job name capture group (#60) * Better ZOAU import missing messages (#61) * Better ZOAU import missing messages * added missing license and imports * Changed behavior on failure for zosoperator_action_query (#63) * changed behavior on failure * update tests for new expected output * Bugfix 1661 & 1704 fix query of job for different user zos_job_submit and zos_job_action_query (#64) * updates to fix job submission as other user * updated and optimized job utility * removed usage of ZOAU and updated jobs utility usage * update job_output return structure * improved error messages * add job log cleanup for test cases * Initial module zos_tso_command (#46) * Linting corrections * Update doc so html will generate * Lintig updates * Lintig updates v2 * Lintig updates v3 * Lintig updates v4 * Lintig updates v5 * Lintig updates v6 * Lintig updates v7 * Lintig updates v8 * Delete test_zos_tso_command_unit.py Remove the unit test as it only mocks things. * Documenation updates * Update errro message * zos-tso-command update * linting update * testcase linting updates * linting update * linting update * linting update * linting update * linting updates * linting update * new authorized long command handling method * linting update * test cases update * linting update * linting update * linting update * tso-command new version * tso-command linting update * linting update * linting update * linting update' * linting removal * lint removal * linting testcases Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> * Update module_utils with an encode utility (#59) * initial commmit for encode_utils * support for string encoding conversion * Support MVS data set conversion and string conversion modification * Recursively copy to PDS fix * Linting correction * Linting updates * Linting updates 01 * fix ENOENT problem after shutil.move * updates with zos_encode * Merged branch and commit for zos_encode (#65) * Merged branch and commit for zos_encode * Lint error , trailing white space * z/OS Fetch Module (#7) * initial commit * begin updating examples * zos_fetch module * return rc, stdout, stderr as separate return parameters on failure * untrack non-fetch related files * add rest of the files * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * added unit and functional tests * check file existence when LISTDS returns invalid dataset name * added 'wait_s' parameter * clarified description of encoding parameter * make use_qaulifer false by default * remove is_uss and os_vsam parameters * remove data_set directory * remove is_uss parameter from examples * message return block * Fix YAML formatting * return code formatting * remove fail_on_missing from action plugin * fail if USS file does not exist * data sets are assumed to be cataloged * instead of entire dict, pass individual values * redundant code when fetching data set member * rename transfer_from_uss to fetch_non_partitioned_data_set * remove test cases for uncataloged datasets * import naming conflict * add __future__ imports * add __future__ imports * assuming all data sets are cataloged * assuming all data sets are cataloged * fix formatting issues * fix formatting issues * remove blank lines with whitespaces * change example member name data to DATA for accuracy. * continuation line unindented for visual indent * conform to PEP8 standard * avoid automatic field numbering for format * specify python interpreter * extract only data set name if member present * Remove newline character from error message * Update RETURN block with sample message * Remove blank line from return block * Update version_added to correspond to Ansible version * Check if temp datasets are being created successfully * Datasets.read() returns None for empty data sets * Use BetterArgParser to parse module args * Added parameter types * Remove trailing whitespaces * Avoid passing extra args from action plugin * Remove unused imports * Ignore lack of GPLv3 license complaint * catch ValueError thrown ny BetterArgParser * Blank line contains whitespace * Reduce line lengths to 80 * Add white space around string addition * Checking volume type no longer required * Check if src or dest is provided first * Fix indentation * Indent multiline string for better readability * Use DataSetUtils to get data set type * Use DataSetUtils to determine data set existence * Remove unit tests * Fix linting * Add test case for empty data sets * default params for _fail_json * Test case for missing source * Test case for insufficient write permission * Test case for insufficient desgt write permission * Catch exceptions while executing module * Use updated DataSetutils constructor * Updated functional tests * Fix linting errors * Use ansible fetch module to fetch USS files * Checking for USS type is unnecessary * Use sha256 instead of sha1 for checksum validation * Remove message return block * Missing uss file test case * Minor cleanup * Add stdout_lines and stderr_lines to module response * Update action plugin with stderr_lines and stdout_lines * YAML syntax issue * Corrected spelling errors * Inaccurate description of flat * No need to wrap error messages returned by DataSetUtils * Update doc regarding HFS and ZFS support * Handle encoding for USS files * Add encoding to doc * Calculate temp file size for VSAM storage * Preserve encoding using SFTP * Resolve PEP8 issues * Check for data set member existence * raw binary not supported in PY2 * Fix linting issues * Change doc to reflect USS absolute path * Prepend dataset hlq before parameter verification * src not declared before being referenced * Add test case using use_qualifier * Updated doc link for fetch L(..) and suboptions * Use tempfile.mkstemp() for temp data storage * Remove default for encoding * default encoding values are None * Create destination dirs when flat is false * Add test case for flat hostname * Fix spelling * exist_ok is not valid keyword for PY2 * Detect read-only file system * Handle sftp errors * Capture module_stderr in case of module failure * Ensure dest is a directory for PDS transfer * Ensure dest is a directory except when fetching a member * Fix data set member destination path * Default values for encoding is not needed * Remove unused imports * Update 'msg' description * Clean up temp files in case of module failure * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * enhanced zoau import error message (#66) * enhanced zoau import error message * satisfy pep8 witespace requirement * Added new playbook for latest module collection and a filter * fixed linting issues, added license * updated spacing to conform to formatting standards * updated shell to command where possible * Use encode util to convert encoding (#68) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update RST and MD doc files to reflect next release * Update group vars to reflect zoau 1.0.2 new env vars * Update encode short description to fit within the limits of doc length * Update makefile for more generic use by others * Update to fix RST lint error , short line * Rename sub-dir from plugin to filter, doc error * doc tree correction * Update release notes to correspond to beta release tag * Update sample playbooks with doc, configuration and new variables * Upate docs to highlight users to review sample playbooks and JCL before running * Update sample JCL to run on my systems with default values and notes * Add supporting doc for behavioral inventory parameters and ansible.cfg * Fix the md style URL in the index RST file * Update short descriptions for modules * Fix README.md link to IMS collection * Fix README.md link to collection artifacts * Linting updates * Add an example that requires a command to be escaped (#71) * refine backup_file & put it into encode_utils (#70) * refine backup_file & put it into encode_utils * correction for sanity test * refactor for security and usability, fix test case * move copy logic to utility * refactor zos_encode, improve backup functionality Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: ddimatos <dimatos@gmail.com> * Update galaxy.yml to reflect the latest version and authors * Fix import name change resulting from refactor * updated usage of encode_utils to encode * Documentation updates, corrections and alterations (#75) * Documentation updates, corrections and alterations * ID review-round 2 * Update the user description in the doc Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Content/81/zos tso command docreview (#73) * ID review of zos-tso-commands * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * plugins/modules/zos_fetch.py * Fix indentation Co-authored-by: ddimatos <dimatos@gmail.com> * Fix extra space in doc * fixed zos_ping action plugin issues (#77) * Add out-of-box awx support (#80) * Added encoding supplemental doc (#78) * Added encoding supplemental doc * Removed references to modules not available and causing doc warnings * Fix Inline literal start-string without end-string. * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst * docs/source/supplementary.rst Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Add binary support copying across USS and MVS * Copy USS file in binary mode * fixed bug and cleaned up code (#82) * fixed bug and cleaned up code * fixed linting issues * Avoid specifying -B and -F for cp * Copy MVS source to MVS dest * Documentation web content updates (#84) * First round of review of static docs * docs/source/installation.rst * Update block to rendor with syntax highlights Code block was using 'yaml' when it was 'sh' , renders better if we have the correct code block type. * Incorporated Demetri's review comments Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Replace only the first occurance of 'rec' * Minor updates in install,quickstart, and req pages (#86) * Updated make file with comments and recover from failure * Update Backup util (mvs_file_backup) (#85) * Alternative mvs_file_backup * fix backup logic, add encode backup tests * add mvs backup tests, util fixes * fix test for new logic path Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Udpate the restructured text to reflect version release (#89) * minor fixes and additional tests for zos_encode (#88) * removed Jobs.submit usage due to invalid response (#92) * Update to galaxy.yml to exclude files * Update makefile to support rexx written modules * Generate and push zos_ping rst doc * Corrected clean up of a directory in makefile * Fixes spacing in RETURN values and formatting (#97) * Validate bk_dsn after assigning a value * Return backup path * Fix alloc command * Remove trailing whitespace * Fix job utility to work on all z/OS systems (#99) * Shinx and sphinx-versioning support and updates (#100) * Updated Makefile to support sphinx-versioning and renamed some of the operations * Update configuration to support sphinx-versioning and update options for sphinx * Update configuration with linting corrections * Correct and update the installation of a pre-release (#102) * Correct and update the installation of a pre-release * minor fixes to docs Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update data set utils (#104) * Update data set utils * Update function signatures * Module parameter not needed for encode util Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Change default encoding from UTF-8 to ISO8859-1 * Fetching non-existent VSAM * Update doc for encoding parameter * Update encoding to handle different charsets * Remove encoding default from argument spec * encoding validation and doc update * Fix typo * Bugfix/case conversion fix zos tso command (#108) * fixed issues with commands forced to uppercase * add support for single or multi commands, added relevant test cases * updated alias doc * Added index file to support shared doc site (#109) * Remove module parameter from EncodeUtils constructor * fix issue when no default encoding specified (#112) * fix issue when no default encoding specified * fix local job submission * Feature/26/ansible zos copy module (#105) * initial commit * begin updating examples * default value 'yes/no' -> 'true/false' * updated documentation * copy to seq data set * copy local to seq data set * add data set size to module response * functional test cases for copy * test case for sequential data set * clean up copy to ps function * copy to pdse * assuming all data sets are cataloged * copy to PDS * Copy to VSAM * Copy to PDS(E) member * Actual directory size instead of max file size * Copy from remote source * Update copy * Reduce line lengths to less than 80 * Use mvscmd cli * remote pdse copy * IEBCOPY to copy data sets * saving changes * Create remote data set if it doesn't exist * Reorganize remote copy transfer * Update return params * Avoid using global module * Remove unused imports * Copy to USS * Clean up in case of module failure * Back up USS file * Copy content to remote file * Copy to sequential data set * Copy local dir to PDS * Copy remote PDS to PDS * Update backup file * USS dest not accessible if not present * Explicitly tag destination file * Changed file mode, group and owner * Change common file args for remote src * Change file common args after copying to dest * Preserve local file mode * Convert encoding for remote src * Added comments * Use backup util to back up files and data sets * Add validate parameter * Copy VSAM to VSAM * Support for symlinks * Copy PDS/PDSE member to USS * Fix formatting * Module should fail for empty VSAM * Added function signatures for all test cases * User better_arg_parser to determine member type * Add binary support for copying to USS target * Copy to PS in binary mode * Correct object instantiations * Import copy util * Remove write_rc * Resolve import issues * Move vsam copy handler to main copy handler class * Copy local dir to USS dir * Infer member name when it isn't provided * Added test cases for local and remote copy * src_member evalutes to True * Comments for extract_dsname() * IEBCOPY fails for incompatible recfm * Copy PDS member to PS * Copy data set member to non-existent PS * Copy local symlink * Add test case for uss file backup * Fix mvs file backup * Functional tests complete * Fix linting issues * Remove trailing whitespace * Ignore GPLv3 license * Remove is_vsam parameter * Fix license information * Fail action plugin * Add test cases to ensure module failure * Remove unneded parameters * Refactor to use updated data_set util * I/O abend when copying PDSE member to non-existent PS * Updated doc, comments, linting * Fix linting issues * Remove duplicate key from dic * Avoid hard-coding tmp path name * Change backup_path parameter to backup_file * Copy to data set member partial * Error converting encoding for local dir * Fix tmp cleanup bug * Updated test cases for backup_file * plugins/modules/zos_copy.py * Reactor code for better readability * Fix pep8 issue * Cleanup error on module failure * plugins/modules/zos_copy.py * Doc update mode parameter * Use native shell instead of bash for functional testing * Return invocation parameter * Remove unnecessary imports * plugins/modules/zos_copy.py * Return backup_file on module failure Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Feature/1089/zos lineinfile (#111) * initial version for interface and options * add backup option * update lineinfile * make golden path work * fix the bugs so that method could support none vars * refactor based on new dsed * updated params and return * Initial check in for backup feature support * support all charsets; updated backup feature doc * clean up; used better_arg_parser * use latest data_set module_util * change the returned string to be loadable by json.loads (BOF/EOF) * uss test cases * dataset test cases; create a dataset/file with content instead of being dependent on existing source * uss test env set bug fix * added UTF-8 in test cases; simplified dataset names * use new dedicated ZOAU function for lineinfile * added assert for expected strings * encoding parameter from absent * Linting updates from pipeline tests * Linting updates and corrections * Linting udpates continued * Update sanity exemption file and linting corrections * Doc error, indentation off, breaking rest of doc parse * module arg and doc updates * Formatting updates * Further doc updates * regexp alias fix * updated docs; added comments; updated parameters * fix sanity checks * fix RETURN key type; lint error * code review changes * fixed linting issues * removed incorrectly added choices for insertafter/insertbefore * added negative test cases fo not supported datasets * moved choices to description; fixed test cases lint issue * changed the choices format in description * add escape if string was quoted * added test cases for escaped/unescaped quotation * used 'src' for target location; moved the params checking before taking backup * fixed indentation * Doc edits for lineinfile, encode, and fetch * remove relative import method * removed the default values for insertafter/insertbefore * renamed helper functions; updated pytest.ini for test cases; fixed TEST_INFO * remove jcl tmp directory if it already exists * run all test_* Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: behnam-ibm <behnam@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/1067/community doc (#113) * Correct doc link to IMS collection to point to Galaxy * Added testing doc to community guidlines * Clean/delete accidental file * Fix Sphinx warning coming from ping module reference to zos_ssh plugin undefined label: zos_ssh_module * Added better arg parse to new dev addition to community guides * Fixes broken site anchor to the YAML configuration * Fix MD style Link * Remove additional MD style links * Fix mailformed RST link * Fix more RST links * Python syntax fixes in samples * Update betterArgParse to BetterArgParser * Expand pipelining=true doc * Correct Unicode quotes to ASCII quotes * Statement of direction on zoau and python * doc review of community guidelines * Doc review of Community guidelines * Fix malformed RST table * Update group vars requirements needed by IBM Pypthon * Update doc with var settings and sample all.yml * Update environ vars * Added environ LANG doc * Added doc to clarify shell support * Added advanced group_vars sample and doc * Update docs to reflect IBM Python announcement Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Remove unnecessary description of PDSE transfer * Update version tag in doc generation conf * Add missing link reference * Doc review (#118) Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update IBM Python download link * Add system info to job query (#127) * Add system info to job query * Change sysname to system for job query Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Close file descriptor (#128) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/2298/zos raw (#120) * add utilities for zos_raw * updated DD statement to better handle concatenations * update parameter names * fix concat naming * refactor of zos_data_set, add new features * updated examples * added steplib dd type * fix steplib data set * update valid concat members * fix datasetdefinition for steplibs * update dd statement types * argument parsing working in zos_raw * module argument parsing updates * updated valid arguments for consistency * update tests for new zos_data_set format * zos_raw module working * module doc updates * updated function documentation * backup, reuse and replace working * updated documentation * cleaned up wrapper classes * updated existence checking, function doc updates * spelling fixes, added new parameters for dd_unix * add list of string support for StdinDefinition * properly handle space units * initial set of test cases, added aliases for program, rename args to parms to match jcl naming conventions * added remaining happy path data set DD tests, added input dd tests. Fixed small bug in raw related to invalid parameter options * update example playbooks to account for new zos_data_set arguments * added initial functional tests for unix file dd statements * added access group support, fixed status group * added remaining tests for dd_unix and dd_dummy * fixed bug on dd_concat contents or backup returned * add dd_concat test cases * update parms to parm to match jcl naming * added execution tests for zos_raw * add final test cases * add special type functions for ksds related params * update import handling for special cases * add VIO DD support * uncomment test cases * fix basic linting issues, default arguments * add support for directory blocks * fix some linting issues * Revert "fix some linting issues" This reverts commit 44d0e1457d76e8c5e5725b457f525d76afb1111f. * linting fixes * module doc and linting updates * additional doc fixes for linting sanity tests * added examples * added catch for nonexistent deletion * updated docs for proper formatting * update ignore files for new module * added dd_output support, standardized dd names * removed unnecessary import wrappers * improved validation for encoding types. * added new example with dd_output type * Merge branch 'origin/dev' * Revert "Merge branch 'origin/dev'" This reverts commit 1befb8e8f0a2448cb9408d94c3cf6a64a2ff9e4c. * fixes for python 2 controller support * modify encode utils import * add utf-8 encoding tag to unit tests * improve check mode support * fix mock of AnsibleModule for encoding check cases * fix unit test formatting for pep8 * fix test case helpers * comment out failing test case unrelated to zos_raw * update encoding conversion handling * update rexx variables to capture correct environment variables * update naming based on discussion * rename test case files, update test cases * update default for output data set * update test ignore files for rename * doc edits (#133) * doc edits * fix minor pep8 issues Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * add python 2 support * remove redundant code comments * add temp data set support, backwards compatibility fix with old interface, update docs and tests * update documentation for return_content * minor doc changes for clarity * improve documentation clarity * fix incorrect syntax for option name Co-authored-by: RadhaV2020 <63616898+RadhaV2020@users.noreply.github.com> * Update configuration vars in doc and yml (#131) * remove ZOAU Python API deps for zos_operator (#132) * remove ZOAU deps which break in 1.0.3 * add back shebang * remove unsafe shell usage * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Content/2516/release doc update (#136) * Update the version in galaxy.yml * edits to support doc gen * Updated for doc gen * updated all module doc for release * Update template to upper case boolean when no default present * update raw module doc to fomat as the indentation intended * updated module RST * add details to module enhancements * Update message around Rocket's python * Update Rocket Python Message * Reference Rocket Python configuration in requirements * Updated doc to reflect the latest dependencies * upload helper wrapper and fix usage (#135) * upload helper wrapper and fix usage * fix whitespace issue * fix unit tests for raw * zos_copy Update (#134) * pathlib not suported in py 2.7 * Doc update * Memebers being deleted during pds backup * Migrate some utils to module utils * Generate a temp member name * Doc update * Use ZOAU API to delete members * Fix PEP8 issues * Delete members from existing PDS * Check for member delete return code * Follow symlink for tmp * Update cleanup process * Ignore backup request for empty PDS * Make 'is_empty()' generic * local dir endswith forward slash * Avoid allocating dest during PDS copy * Escape special characters in member name * Escape special character for src and dest * Add 'model_ds' parameter * Add model_ds support for PS * Fix allocation issues for PS * Specify BLKSIZE when allocating model * Check for dest existence from action plugin * Doc update * data set util update * Change force default on action plugin * Remove _fail_action() from action plugin * Update vtoc usage * Fix bug in action plugin * Use DataSet API for zos_encode * Update data set API for fetch Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Remove see also reference to external collections * updated zos_copy module RST * Sample playbook for latest version * Updated ZOAU PTF information with the latest PTF UI70435 * Updates to raw doc to handle the error - Definition list ends without a blank line * Update copy module docs * Update and add clarity around Rocket Python * Requirements order change Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> Co-authored-by: Asif Mahmud <amahmud@uci.edu> Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update conf for v1.2.0-beta.1 * Fix E231: missing whitespace (#138) * Specify SFTP port for fetch and copy (#141) * Add sftp_port parameter to fetch and copy * Add sftp_port parameter to fetch and copy * Fix spelling * Missing positional arguments * Test case for negative port specification * Check for valid port range * Remove redundant assertions Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Feature/dependency discovery (#142) * add host setup playbook * handle error cases * restructure and improve robustness of host-setup * add max_search_depth variable * change file extension for consistency * add documentation for host-setup.yaml * add in-file docs for tasks files * add important note for task files * Remove redundant data_set_util * Move error classes to the bottom * Rename backup_file to backup_name (#144) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/zos_copy record format (#147) * Update uss2mvs copy format * Default PS allocation should be FB * Use ZOAU Copy for sequential data sets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/148/accept all valid data set name characters (#149) * update regex for data set name * add test case for special characters * Add ignore_sftp_stderr parameter for fetch and copy * improve cleanup (#151) * change file extension * add /usr/sbin to default PATH * Use USS cp command to copy to PS * Bugfix: zos operator action query not returning all outstanding messages (#154) * remove ZOAU python API usage * add back ZOAU python API, update parsing regex * Changes to support single source doc site (#155) * Fix E231: missing whitespace * Changes to support single source doc site * Under indentation * Under indentation 2 * doc review * doc review of additional files * Update the shell scripts to create a single requirements RST Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * add verbose support and steplib fix (#156) * udpate maxdepth for RST to 1 * Use default discovered port for sftp * Content/3397/update docs (#158) * Fix E231: missing whitespace * Changes to support single source doc site (#145) * Changes to support single source doc site * Under indentation * Under indentation 2 * doc review * doc review of additional files * Update the shell scripts to create a single requirements RST Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Module known issues and release notes added * Updated doc and sample host config playbook doc * Fix use ASCII quotes instead of Unicode quotes * use ASCII quotes instead of Unicode quotes Co-authored-by: Radha Varadachari <radha.varadachari@ibm.com> * Update galaxy to latest doc site and version * Release v1.2.0 beta.2 (#163) * Initial collection commit for v0.0.1 (#1) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * added arg parser to module_utils (#3) * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * Updated job output for improved DD handling (#14) Fixes the job summary output being returned when return_output=false. * Security module updates (#15) * patched security holes * updated test cases * removed unnecessary import * updated execution bits * Update modules and documentation corrections (#17) * update documentation, remove unused imports * updated requirements * updated module imports, formatting, and spelling * doc update and formatting update * z/OS Ping Module Feature Delivered (#18) * added zos_ping, test, and test fixture update * Description completeness * Grammar completeness * Pulled repetitive code into functions * updated module docs * Minor spelling corrections * Repo updates to satisfy ansible-test sanity testing requirements (#19) * update formatting * formatting changes to appease ansible-test * update tests * fix string format use and quotes * update README * remove trailing whitespace * update group_vars trailing spaces * remove shebang in attempt to fix encoding issue * add back shebang lines * Added security updates and test cases (#20) * add new types for parser, update arg handling zos_job_submit * updated module argument parsing and security tests * uncommented security tests * updated README to reflect use of MIT license * update hash function * test and formatting updates * updated documentation to pass ansible-test * switch name back to normal.py, add appropriate ignores * update ignore files * added ignore for GPL * updated README, .gitignore and removed MIT code * remove broken tests * increase robustness of zos_job_query * make job query even more robust * Webdocs content for IBM z/OS Core Collection (#21) * Initial commit for reStructuredText content * Updates and clean up to the template * Updated grammar * Grammar edit * Grammar fixes * Text updates * Updates to restructured text, fixed link, renamed collection * Fix ibm_zos_core bold * Fixes ansible-test lint: docs/source/conf.py:51:161: E501: line too long (178 > 160 characters) * ansible doc changes to modules for better formatting * remove original_message return parameter (#23) * remove original_message return parameter * Fix reported linting errors from PR * Fix reported linting errors from PR * Fix reported linting errors from PR for zos_job_query.py * Module document updates, Makefile changes and jinja support for defau… (#24) * Module document updates, Makefile changes and jinja support for default:int * Doc updates per linting rules * Update mmodule doc and remove md files (#25) * Removed all MD files from docs directory * Remove README.md , its replaced by README.rst * Fix bolding for ibm_zos_core * Fix highlight for ibm_zos_core * Updated index.rst to correspond to the README.md updates * Updated README.rst to point to webdocs * Updated README.rst webdocs URL * Update the modules to their ansible-doc * Documentation updates (#26) * Updated galaxy.yml to reflect latest version 1.0 * Documentation copyright updates and formatting * Documentation udpates * Update z/OS data set module and add catalog functionality (#27) * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * add VTOC shared utility * zos_data_set catalog/uncatalog working for non vsam * catalog for VSAM working, bugs fixed, tests added * satisfy ansible-test requirements, minor arg validation updates * removed unit tests * raise min . ansible requirement due to security vulnerability * added documentation to VTOC * Update product name in webdocs (#28) * Update product name in webdocs * Fix trailing newline * Document updates include some template bugs * Integraded document review comments (#29) * Integrated document review comments * Incorrect break for bulleted line * Bug in Jinja template where property would tab to much (#31) * Updates to documentation and galaxy configuration (#32) * Fix to README.rst, duplicate wording (#34) * Remove duplicate action plugins link (#35) * testing readme rendering (#36) Ported the README.rst file to README.md to meet collection requirements. * Corrected README to point to MD, and long tag removed (#39) * Add two additional arg types to BetterArgHandler (#44) * Add two additional arg types * Remove os from os.path.isabs Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Data Set Utility for Common Data Set Related Tasks (#48) * DataSetUtils initial commit * LISTDS and LISTCAT output processing * Raise exception when data set is open for write * Fix LISTDS regex * Fix indentation * Add USS path support * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix/832/fix non vsam non sms existence checking zos data set (#49) * updated catalog handling for non-vsam, non-sms * updated some method comment structures * bugfix/822/determine-data-set-type (#50) * Use mvscmd CLI instead of Python API * Fix linting errors * Remove unused variable Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Extend dynamic function support to the better arg parser (#51) * Dynamic argument support for dynamic functions. * improve default handling for reusability of BetterArgParser object. * add test cases for dynamic argument support * removed print statements from test cases * Updates to data-set-utils exceptions (#52) * Avoid raising exception for uncatalogued data sets * verify listcat output for uncatalogued datasets Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Removed return block from documentation (#53) Removed message and changed, empty return block now * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * Fixed PATH (#55) * Update/973/add new supported data set characters betterargparser (#56) * updated valid data set types builtin type name fix * added test cases for data set types * Bugfix when checking if data set exists (#57) * Return False instead of None when data set doesn't exist * Test for membership should ne 'not in' Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Committing module zos_operator_action_query (#43) * Reviewd changes for feature zos_operator_action_query.py * Fix linting errors reported in PR * Linting corrections * fix the syntax error * fix lint errors * fix lint error * fix lint error * fix lint error * fix function test * Added zos_operator_action_query to the sanity exemption list for lack of GPLV3 Lic * Update author tag * Update the collection reference * add function test(need revise later) & doc update * updated regex and value capturing * Initial commit for module zos_operator (#45) * Initial commit for moduel zos_operator * Linting updates * Linting corrections v2 * Linting corrections v3 * Fixed the collection dependency name * adding various command test, invalid test * fix lint errors * fix lint err * Corrected License, updated messages and renamed functional tests * Update doc and return type * Documentation update Co-authored-by: xiaoping <xiaoping@cn.ibm.com> * updated docs, made environment agnostic test cases * added support for "*" and fixed splat handling * updated test cases * updated regex matching method, updates tests * Update short description Co-authored-by: xiaoping <xiaoping@cn.ibm.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * Check for data set member existence (#58) * Check for data set member existence * Resolve linting issue Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Bugfix for job name capture group (#60) * Better ZOAU import missing messages (#61) * Better ZOAU import missing messages * added missing license and imports * Changed behavior on failure for zosoperator_action_query (#63) * changed behavior on failure * update tests for new expected output * Bugfix 1661 & 1704 fix query of job for different user zos_job_submit and zos_job_action_query (#64) * updates to fix job submission as other user * updated and optimized job utility * removed usage of ZOAU and updated jobs utility usage * update job_output return structure * improved error messages * add job log cleanup for test cases * Initial module zos_tso_command (#46) * Linting corrections * Update doc so html will generate * Lintig updates * Lintig updates v2 * Lintig updates v3 * Lintig updates v4 * Lintig updates v5 * Lintig updates v6 * Lintig updates v7 * Lintig updates v8 * Delete test_zos_tso_command_unit.py Remove the unit test as it only mocks things. * Documenation updates * Update errro message * zos-tso-command update * linting update * testcase linting updates * linting update * linting update * linting update * linting update * linting updates * linting update * new authorized long command handling method * linting update * test cases update * linting update * linting update * linting update * tso-command new version * tso-command linting update * linting update * linting update * linting update' * linting removal * lint removal * linting testcases Co-authored-by: bjmaxy <wudiha@126.com> Co-authored-by: “bjmaxy” <bjmaxy@cn.ibm.com> * Update module_utils with an encode utility (#59) * initial commmit for encode_utils * support for string encoding conversion * Support MVS data set conversion and string conversion modification * Recursively copy to PDS fix * Linting correction * Linting updates * Linting updates 01 * fix ENOENT problem after shutil.move * updates with zos_encode * Merged branch and commit for zos_encode (#65) * Merged branch and commit for zos_encode * Lint error , trailing white space * z/OS Fetch Module (#7) * initial commit * begin updating examples * zos_fetch module * return rc, stdout, stderr as separate return parameters on failure * untrack non-fetch related files * add rest of the files * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * update type in galaxy.yml tags. * Update and merge master versions to dev branch (#12) * Initial collection commit for v0.0.1 (#1) (#2) * hotfix readme url and license syntax (#4) * Corrected relative MD url(s), zos dependency; updated galaxy version * Updated playbook README version minor value to v.0.0.2 * Hotfix/122/update readme (#5) * update README.md to reflect collection * hot fix for zos_data_set existence checking and validation * update README.md version tag links to v.0.0.3 * Updates to README.md that include fixing a malformed link * Remove dependency list from modules Dependencies will be stated on the collections README.md * Update content to reflect latest version (#10) * Update content to reflect version 0.0.4 * Updated docs to use the master branch over the tagged version * Update colleciton with functional test cases (#9) * Added functional test cases and refactored existing test cases * Updated copyright and license to functional test cases * Test updates for validity with public repo * refactor and patch job output checking (#6) * refactor and patch job output checking * updated documentation, output formatting, and added RC validation * updated response information * updated parsing of abend codes * updated return example formatting * Removing unused test directories * Update README-ZOAU.md * Bugfix/remove merge message artifacts (#13) * removed artifacts * removed artifacts * added back .gitignore * removed stragglers, updated gitignore * added unit and functional tests * check file existence when LISTDS returns invalid dataset name * added 'wait_s' parameter * clarified description of encoding parameter * make use_qaulifer false by default * remove is_uss and os_vsam parameters * remove data_set directory * remove is_uss parameter from examples * message return block * Fix YAML formatting * return code formatting * remove fail_on_missing from action plugin * fail if USS file does not exist * data sets are assumed to be cataloged * instead of entire dict, pass individual values * redundant code when fetching data set member * rename transfer_from_uss to fetch_non_partitioned_data_set * remove test cases for uncataloged datasets * import naming conflict * add __future__ imports * add __future__ imports * assuming all data sets are cataloged * assuming all data sets are cataloged * fix formatting issues * fix formatting issues * remove blank lines with whitespaces * change example member name data to DATA for accuracy. * continuation line unindented for visual indent * conform to PEP8 standard * avoid automatic field numbering for format * specify python interpreter * extract only data set name if member present * Remove newline character from error message * Update RETURN block with sample message * Remove blank line from return block * Update version_added to correspond to Ansible version * Check if temp datasets are being created successfully * Datasets.read() returns None for empty data sets * Use BetterArgParser to parse module args * Added parameter types * Remove trailing whitespaces * Avoid passing extra args from action plugin * Remove unused imports * Ignore lack of GPLv3 license complaint * catch ValueError thrown ny BetterArgParser * Blank line contains whitespace * Reduce line lengths to 80 * Add white space around string addition * Checking volume type no longer required * Check if src or dest is provided first * Fix indentation * Indent multiline string for better readability * Use DataSetUtils to get data set type * Use DataSetUtils to determine data set existence * Remove unit tests * Fix linting * Add test case for empty data sets * default params for _fail_json * Test case for missing source * Test case for insufficient write permission * Test case for insufficient desgt write permission * Catch exceptions while executing module * Use updated DataSetutils constructor * Updated functional tests * Fix linting errors * Use ansible fetch module to fetch USS files * Checking for USS type is unnecessary * Use sha256 instead of sha1 for checksum validation * Remove message return block * Missing uss file test case * Minor cleanup * Add stdout_lines and stderr_lines to module response * Update action plugin with stderr_lines and stdout_lines * YAML syntax issue * Corrected spelling errors * Inaccurate description of flat * No need to wrap error messages returned by DataSetUtils * Update doc regarding HFS and ZFS support * Handle encoding for USS files * Add encoding to doc * Calculate temp file size for VSAM storage * Preserve encoding using SFTP * Resolve PEP8 issues * Check for data set member existence * raw binary not supported in PY2 * Fix linting issues * Change doc to reflect USS absolute path * Prepend dataset hlq before parameter verification * src not declared before being referenced * Add test case using use_qualifier * Updated doc link for fetch L(..) and suboptions * Use tempfile.mkstemp() for temp data storage * Remove default for encoding * default encoding values are None * Create destination dirs when flat is false * Add test case for flat hostname * Fix spelling * exist_ok is not valid keyword for PY2 * Detect read-only file system * Handle sftp errors * Capture module_stderr in case of module failure * Ensure dest is a directory for PDS transfer * Ensure dest is a directory except when fetching a member * Fix data set member destination path * Default values for encoding is not needed * Remove unused imports * Update 'msg' description * Clean up temp files in case of module failure * Fix linting errors Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> Co-authored-by: ddimatos <dimatos@gmail.com> Co-authored-by: Bryant Panyarachun <bpanyarachun@gmail.com> Co-authored-by: Blake Becker <blakejordanbecker@gmail.com> * enhanced zoau import error message (#66) * enhanced zoau import error message * satisfy pep8 witespace requirement * Added new playbook for latest module collection and a filter * fixed linting issues, added license * updated spacing to conform to formatting standards * updated shell to command where possible * Use encode util to convert encoding (#68) Co-authored-by: Asif Mahmud <asif.mahmud@ibm.com> * Update RST and MD doc files to reflect next release * Update group vars to reflect zoau 1.0.2 new env vars * Update encode short description to fit within the limits of doc length * Update makefile for more generic use by others * Update to fix RST lint error , short line * Rename sub-dir from plugin to filter, doc error * doc tree correction * Update release notes to correspond to beta release tag * Update sample playbooks with doc, configuration and new variables * Upate docs to highlight users to review sample playbooks and JCL before running * Update sample JCL to run on my systems with default values and notes * Add supporting doc for behavioral inventory parameters and ansible.cfg * Fix the md style URL in the index RST file * Update short descriptions for modules * Fix README.md link to IMS collection * Fix README.md link to collection artifacts * Linting updates * Add an example that requires a command to be escaped (#71) * refine backup_file & put it into enc…
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SUMMARY
This change introduces 4 functional test cases for the modules:
It also flattens the existing
tests/units/modulestest cases and refactors for consistency.ISSUE TYPE