New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scap facts module #47980

Open
wants to merge 10 commits into
base: devel
from

Conversation

Projects
None yet
5 participants
@defionscode
Contributor

defionscode commented Nov 2, 2018

SUMMARY

This a new module that allows for ansible to ingest the results.xml file generated by openscap-scanner's oscap xccdf eval ... command. The data is then stored as a fact.

Given that openscap, etc are generally always used in the context of audits etc, this facts module exposes the ability to cause the task to fail in the even certain score thresholds are not met either at the macro level or at the severity level. This is totally optional behavior and all thresholds are set to 0 by default.

Additionally, in order to make things easier to grok, a user can have the facts module only 'store' data for benchmarks that are passing, failing, not selected, not applicable, or not checked (or any combination).

ISSUE TYPE

New Module Pull Request

COMPONENT NAME

scap_facts

ANSIBLE VERSION
ansible 2.8.0.dev0 (oscap-facts eaaaeb2062) last updated 2018/11/02 09:50:17 (GMT -400)
  config file = None
  configured module search path = ['/Users/jonathand/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/jonathand/Repos/ansible/lib/ansible
  executable location = /Users/jonathand/Repos/ansible/bin/ansible
  python version = 3.7.0 (default, Jul 23 2018, 20:22:55) [Clang 9.1.0 (clang-902.0.39.2)]
ADDITIONAL INFORMATION
@ansibot

This comment has been minimized.

Contributor

ansibot commented Nov 2, 2018

Hi @defionscode, thank you for submitting this pull-request!

click here for bot help

@ansibot

This comment has been minimized.

Contributor

ansibot commented Nov 2, 2018

The test ansible-test sanity --test pylint [explain] failed with 20 errors:

lib/ansible/modules/system/scap_facts.py:216:39: ansible-format-automatic-specification Format string contains automatic field numbering specification
lib/ansible/modules/system/scap_facts.py:219:38: ansible-format-automatic-specification Format string contains automatic field numbering specification
lib/ansible/modules/system/scap_facts.py:223:32: ansible-format-automatic-specification Format string contains automatic field numbering specification
lib/ansible/modules/system/scap_facts.py:226:36: ansible-format-automatic-specification Format string contains automatic field numbering specification
lib/ansible/modules/system/scap_facts.py:228:163: trailing-whitespace Trailing whitespace
lib/ansible/modules/system/scap_facts.py:232:41: ansible-format-automatic-specification Format string contains automatic field numbering specification
lib/ansible/modules/system/scap_facts.py:234:30: ansible-format-automatic-specification Format string contains automatic field numbering specification
lib/ansible/modules/system/scap_facts.py:240:26: bad-whitespace Exactly one space required after comma         raw = {k : v for k,v in data.items() if v['severity'] == severity}                           ^
lib/ansible/modules/system/scap_facts.py:241:29: bad-whitespace Exactly one space required after comma         wanted = {k : v for k,v in raw.items() if v['result'] in self.include_results}                              ^
lib/ansible/modules/system/scap_facts.py:242:27: bad-whitespace Exactly one space required after comma         fail = {k : v for k,v in raw.items() if v['result'] == 'fail'}                            ^
lib/ansible/modules/system/scap_facts.py:243:30: bad-whitespace Exactly one space required after comma         passing = {k : v for k,v in raw.items() if v['result'] == 'pass'}                               ^
lib/ansible/modules/system/scap_facts.py:244:25: bad-whitespace Exactly one space required after comma         ns = {k : v for k,v in raw.items() if v['result'] == 'notselected'}                          ^
lib/ansible/modules/system/scap_facts.py:245:25: bad-whitespace Exactly one space required after comma         na = {k : v for k,v in raw.items() if v['result'] == 'notapplicable'}                          ^
lib/ansible/modules/system/scap_facts.py:246:25: bad-whitespace Exactly one space required after comma         nc = {k : v for k,v in raw.items() if v['result'] == 'notchecked'}                          ^
lib/ansible/modules/system/scap_facts.py:290:34: ansible-format-automatic-specification Format string contains automatic field numbering specification
lib/ansible/modules/system/scap_facts.py:294:40: ansible-format-automatic-specification Format string contains automatic field numbering specification
lib/ansible/modules/system/scap_facts.py:320:29: ansible-format-automatic-specification Format string contains automatic field numbering specification
lib/ansible/modules/system/scap_facts.py:325:29: ansible-format-automatic-specification Format string contains automatic field numbering specification
lib/ansible/modules/system/scap_facts.py:328:29: ansible-format-automatic-specification Format string contains automatic field numbering specification
lib/ansible/modules/system/scap_facts.py:331:29: ansible-format-automatic-specification Format string contains automatic field numbering specification

The test ansible-test sanity --test ansible-doc --python 2.6 [explain] failed with 1 error:

lib/ansible/modules/system/scap_facts.py:0:0: has a documentation error formatting or is missing documentation.

The test ansible-test sanity --test docs-build [explain] failed with the error:

Command "/usr/bin/python test/sanity/code-smell/docs-build.py" returned exit status 1.
>>> Standard Error
Command 'make singlehtmldocs' failed with status code: 2
--> Standard Output
cat _themes/srtd/static/css/theme.css | sed -e 's/^[ 	]*//g; s/[ 	]*$//g; s/\([:{;,]\) /\1/g; s/ {/{/g; s/\/\*.*\*\///g; /^$/d' | sed -e :a -e '$!N; s/\n\(.\)/\1/; ta' > _themes/srtd/static/css/theme.min.css
PYTHONPATH=../../lib ../bin/dump_config.py --template-file=../templates/config.rst.j2 --output-dir=rst/reference_appendices/ -d ../../lib/ansible/config/base.yml
mkdir -p rst/cli
PYTHONPATH=../../lib ../bin/generate_man.py --template-file=../templates/cli_rst.j2 --output-dir=rst/cli/ --output-format rst ../../lib/ansible/cli/*.py
PYTHONPATH=../../lib ../bin/dump_keywords.py --template-dir=../templates --output-dir=rst/reference_appendices/ -d ./keyword_desc.yml
PYTHONPATH=../../lib ../bin/plugin_formatter.py -t rst --template-dir=../templates --module-dir=../../lib/ansible/modules -o rst/modules/ 
rendering: scap_facts (116 previous rendering line(s) omitted)
Makefile:93: recipe for target 'modules' failed
--> Standard Error
Traceback (most recent call last):
  File "../bin/plugin_formatter.py", line 720, in <module>
    main()
  File "../bin/plugin_formatter.py", line 707, in main
    process_plugins(plugin_info, templates, outputname, output_dir, options.ansible_version, plugin_type)
  File "../bin/plugin_formatter.py", line 458, in process_plugins
    if too_old(added):
  File "../bin/plugin_formatter.py", line 392, in too_old
    readded = added_tokens[0] + "." + added_tokens[1]
IndexError: list index out of range
make: *** [modules] Error 1

The test ansible-test sanity --test compile --python 2.6 [explain] failed with 1 error:

lib/ansible/modules/system/scap_facts.py:240:24: SyntaxError: raw = {k : v for k,v in data.items() if v['severity'] == severity}

The test ansible-test sanity --test import --python 2.6 [explain] failed with 1 error:

lib/ansible/modules/system/scap_facts.py:240:24: SyntaxError: invalid syntax

The test ansible-test sanity --test no-smart-quotes [explain] failed with 1 error:

test/integration/targets/scap_facts/files/oscap-results.xml:38553:55: use ASCII quotes `'` and `"` instead of Unicode quotes

The test ansible-test sanity --test pep8 [explain] failed with 32 errors:

lib/ansible/modules/system/scap_facts.py:207:1: E302 expected 2 blank lines, found 1
lib/ansible/modules/system/scap_facts.py:220:37: E261 at least two spaces before inline comment
lib/ansible/modules/system/scap_facts.py:223:65: E261 at least two spaces before inline comment
lib/ansible/modules/system/scap_facts.py:225:17: E123 closing bracket does not match indentation of opening bracket's line
lib/ansible/modules/system/scap_facts.py:228:44: E261 at least two spaces before inline comment
lib/ansible/modules/system/scap_facts.py:228:161: E501 line too long (163 > 160 characters)
lib/ansible/modules/system/scap_facts.py:228:164: W291 trailing whitespace
lib/ansible/modules/system/scap_facts.py:229:32: E261 at least two spaces before inline comment
lib/ansible/modules/system/scap_facts.py:233:34: E261 at least two spaces before inline comment
lib/ansible/modules/system/scap_facts.py:239:37: E226 missing whitespace around arithmetic operator
lib/ansible/modules/system/scap_facts.py:240:17: E203 whitespace before ':'
lib/ansible/modules/system/scap_facts.py:240:27: E231 missing whitespace after ','
lib/ansible/modules/system/scap_facts.py:241:20: E203 whitespace before ':'
lib/ansible/modules/system/scap_facts.py:241:30: E231 missing whitespace after ','
lib/ansible/modules/system/scap_facts.py:242:18: E203 whitespace before ':'
lib/ansible/modules/system/scap_facts.py:242:28: E231 missing whitespace after ','
lib/ansible/modules/system/scap_facts.py:243:21: E203 whitespace before ':'
lib/ansible/modules/system/scap_facts.py:243:31: E231 missing whitespace after ','
lib/ansible/modules/system/scap_facts.py:244:16: E203 whitespace before ':'
lib/ansible/modules/system/scap_facts.py:244:26: E231 missing whitespace after ','
lib/ansible/modules/system/scap_facts.py:245:16: E203 whitespace before ':'
lib/ansible/modules/system/scap_facts.py:245:26: E231 missing whitespace after ','
lib/ansible/modules/system/scap_facts.py:246:16: E203 whitespace before ':'
lib/ansible/modules/system/scap_facts.py:246:26: E231 missing whitespace after ','
lib/ansible/modules/system/scap_facts.py:256:17: E126 continuation line over-indented for hanging indent
lib/ansible/modules/system/scap_facts.py:264:13: E121 continuation line under-indented for hanging indent
lib/ansible/modules/system/scap_facts.py:296:1: E302 expected 2 blank lines, found 1
lib/ansible/modules/system/scap_facts.py:307:33: E261 at least two spaces before inline comment
lib/ansible/modules/system/scap_facts.py:336:21: E126 continuation line over-indented for hanging indent
lib/ansible/modules/system/scap_facts.py:341:17: E121 continuation line under-indented for hanging indent
lib/ansible/modules/system/scap_facts.py:342:13: E123 closing bracket does not match indentation of opening bracket's line
lib/ansible/modules/system/scap_facts.py:345:1: E305 expected 2 blank lines after class or function definition, found 1

The test ansible-test sanity --test validate-modules [explain] failed with 9 errors:

lib/ansible/modules/system/scap_facts.py:0:0: E306 version_added is not a valid version number: 'local'
lib/ansible/modules/system/scap_facts.py:16:0: E107 Imports should be directly below DOCUMENTATION/EXAMPLES/RETURN/ANSIBLE_METADATA.
lib/ansible/modules/system/scap_facts.py:18:0: E107 Imports should be directly below DOCUMENTATION/EXAMPLES/RETURN/ANSIBLE_METADATA.
lib/ansible/modules/system/scap_facts.py:18:0: E107 Imports should be directly below DOCUMENTATION/EXAMPLES/RETURN/ANSIBLE_METADATA.
lib/ansible/modules/system/scap_facts.py:18:0: E107 Imports should be directly below DOCUMENTATION/EXAMPLES/RETURN/ANSIBLE_METADATA.
lib/ansible/modules/system/scap_facts.py:20:0: E107 Imports should be directly below DOCUMENTATION/EXAMPLES/RETURN/ANSIBLE_METADATA.
lib/ansible/modules/system/scap_facts.py:22:0: E107 Imports should be directly below DOCUMENTATION/EXAMPLES/RETURN/ANSIBLE_METADATA.
lib/ansible/modules/system/scap_facts.py:23:0: E107 Imports should be directly below DOCUMENTATION/EXAMPLES/RETURN/ANSIBLE_METADATA.
lib/ansible/modules/system/scap_facts.py:203:43: E313 RETURN is not valid YAML

The test ansible-test sanity --test yamllint [explain] failed with 1 error:

lib/ansible/modules/system/scap_facts.py:203:43: error RETURN: syntax error: mapping values are not allowed here

click here for bot help

defionscode added some commits Nov 2, 2018

@ansibot

This comment has been minimized.

Contributor

ansibot commented Nov 2, 2018

The test ansible-test sanity --test pylint [explain] failed with 1 error:

lib/ansible/modules/system/scap_facts.py:245:35: bad-whitespace Exactly one space required after comma         passing = dict((k, v) for k,v in raw.items() if v['result'] == 'pass')                                    ^

The test ansible-test sanity --test no-smart-quotes [explain] failed with 1 error:

test/integration/targets/scap_facts/files/oscap-results.xml:38553:55: use ASCII quotes `'` and `"` instead of Unicode quotes

The test ansible-test sanity --test pep8 [explain] failed with 9 errors:

lib/ansible/modules/system/scap_facts.py:218:49: E225 missing whitespace around operator
lib/ansible/modules/system/scap_facts.py:225:57: E261 at least two spaces before inline comment
lib/ansible/modules/system/scap_facts.py:230:161: E501 line too long (164 > 160 characters)
lib/ansible/modules/system/scap_facts.py:235:34: E261 at least two spaces before inline comment
lib/ansible/modules/system/scap_facts.py:245:36: E231 missing whitespace after ','
lib/ansible/modules/system/scap_facts.py:258:17: E126 continuation line over-indented for hanging indent
lib/ansible/modules/system/scap_facts.py:266:13: E121 continuation line under-indented for hanging indent
lib/ansible/modules/system/scap_facts.py:310:33: E261 at least two spaces before inline comment
lib/ansible/modules/system/scap_facts.py:339:21: E126 continuation line over-indented for hanging indent

click here for bot help

defionscode added some commits Nov 2, 2018

Update lib/ansible/modules/system/scap_facts.py
Co-Authored-By: defionscode <jonathan@davila.io>

@s-hertel s-hertel removed the needs_triage label Nov 2, 2018

@ansibot

This comment has been minimized.

Contributor

ansibot commented Nov 2, 2018

The test ansible-test sanity --test pep8 [explain] failed with 2 errors:

lib/ansible/modules/system/scap_facts.py:314:33: E261 at least two spaces before inline comment
lib/ansible/modules/system/scap_facts.py:343:21: E126 continuation line over-indented for hanging indent

click here for bot help

@defionscode

This comment has been minimized.

Contributor

defionscode commented Nov 2, 2018

related proposal ansible/proposals#148

@mattclay

This comment has been minimized.

Member

mattclay commented Nov 5, 2018

CI failure in integration tests (traceback):

01:05 TASK [scap_facts : High Sev Criteria check] ************************************
01:06 An exception occurred during task execution. To see the full traceback, use -vvv. The error was: TypeError: cannot concatenate 'str' and 'int' objects
01:06 fatal: [testhost]: FAILED! => {"changed": false, "module_stderr": "Traceback (most recent call last):\n  File \"/var/root/.ansible/tmp/ansible-tmp-1541171663.29-172248541979603/AnsiballZ_scap_facts.py\", line 113, in <module>\n    _ansiballz_main()\n  File \"/var/root/.ansible/tmp/ansible-tmp-1541171663.29-172248541979603/AnsiballZ_scap_facts.py\", line 105, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/var/root/.ansible/tmp/ansible-tmp-1541171663.29-172248541979603/AnsiballZ_scap_facts.py\", line 48, in invoke_module\n    imp.load_module('__main__', mod, module, MOD_DESC)\n  File \"/tmp/ansible_scap_facts_payload_UDiWH_/__main__.py\", line 354, in <module>\n  File \"/tmp/ansible_scap_facts_payload_UDiWH_/__main__.py\", line 332, in main\nTypeError: cannot concatenate 'str' and 'int' objects\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

@mattclay mattclay added the ci_verified label Nov 5, 2018

@ansibot ansibot added the stale_ci label Nov 13, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment