-
Notifications
You must be signed in to change notification settings - Fork 635
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add support for json schema validation
- Loading branch information
Showing
40 changed files
with
5,360 additions
and
62 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -76,6 +76,7 @@ isdir | |
isdisjoint | ||
iskeyword | ||
isort | ||
jsonschema | ||
junitxml | ||
kubernetes | ||
libera | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -10,3 +10,4 @@ | |
ansible.builtin.shell: echo blah | ||
args: | ||
chdir: X | ||
become_method: xx |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
--- | ||
- name: This should raise json-schema error, due to hosts missing the last letter | ||
host: localhost | ||
tasks: [] |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
--- | ||
foo | ||
# This file is valid YAML but from our point of view is an error, as is | ||
# neither a Sequence or a Mapping. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,7 @@ | ||
foo | ||
# This file is valid YAML but from our point of view is an error, as is | ||
# neither a Sequence or a Mapping. | ||
# This file is valid YAML and passed JSON Schema validation but not ansible | ||
# own syntax check. | ||
|
||
- hosts: localhost | ||
tasks: | ||
- name: invalid syntax | ||
x.y.z.w: {} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,9 +1,9 @@ | ||
--- | ||
galaxy_info: | ||
role_name: invalid-due-to-meta | ||
role_name: invalid-due-to-meta # <-- invalid role name | ||
author: foo | ||
description: foo | ||
license: MIT | ||
platforms: | ||
- name: foo | ||
min_ansible_version: 2.7 | ||
- name: AIX | ||
min_ansible_version: "2.7" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,118 @@ | ||
"""Rule definition for JSON Schema Validations.""" | ||
import json | ||
import logging | ||
import os | ||
import sys | ||
from functools import lru_cache | ||
from typing import Any, List | ||
|
||
import yaml | ||
from jsonschema import validate | ||
from jsonschema.exceptions import ValidationError | ||
|
||
from ansiblelint.config import JSON_SCHEMAS | ||
from ansiblelint.errors import MatchError | ||
from ansiblelint.file_utils import Lintable | ||
from ansiblelint.rules import AnsibleLintRule | ||
from ansiblelint.schemas import __file__ as schemas_module | ||
|
||
_logger = logging.getLogger(__name__) | ||
|
||
|
||
class ValidateSchemaRule(AnsibleLintRule): | ||
"""Perform JSON Schema Validation for known lintable kinds. | ||
Returned errors will not include exact line numbers, but they will mention | ||
the schema name being used as a tag, like ``playbook-schema``, | ||
``tasks-schema``. | ||
This rule is not skippable and stops further processing of the file. | ||
Schema bugs should be reported towards https://github.com/ansible/schemas | ||
project instead of ansible-lint. | ||
If incorrect schema was picked, you might want to either: | ||
* move the file to standard location, so its file is detected correctly. | ||
* use ``kinds:`` option in linter config to help it pick correct file type. | ||
""" | ||
|
||
id = "schema" | ||
description = __doc__ | ||
severity = "VERY_HIGH" | ||
tags = ["core", "unskippable", "experimental"] | ||
version_added = "v6.1.0" | ||
|
||
@staticmethod | ||
@lru_cache(maxsize=None) | ||
def _get_schema(kind: str) -> Any: | ||
"""Return the schema for the given kind.""" | ||
schema_file = os.path.dirname(schemas_module) + "/" + kind + ".json" | ||
with open(schema_file, encoding="utf-8") as f: | ||
return json.load(f) | ||
|
||
@staticmethod | ||
def process_lintable(lintable: Lintable) -> List[MatchError]: | ||
"""Return JSON validation errors found as a list of MatchError(s).""" | ||
result = [] | ||
if lintable.kind not in JSON_SCHEMAS: | ||
return [] | ||
|
||
try: | ||
validate( | ||
instance=yaml.safe_load(lintable.content), | ||
schema=ValidateSchemaRule._get_schema(lintable.kind), | ||
) | ||
except yaml.constructor.ConstructorError: | ||
_logger.debug( | ||
"Ignored failure to load %s for schema validation, as !vault may cause it." | ||
) | ||
return [] | ||
except ValidationError as exc: | ||
result.append( | ||
MatchError( | ||
message=exc.message, | ||
filename=lintable, | ||
rule=ValidateSchemaRule(), | ||
details=ValidateSchemaRule.description, | ||
tag=f"schema[{lintable.kind}]", | ||
) | ||
) | ||
return result | ||
|
||
|
||
# testing code to be loaded only with pytest or when executed the rule file | ||
if "pytest" in sys.modules: | ||
import pytest | ||
|
||
@pytest.mark.parametrize( | ||
("file", "expected_kind", "expected"), | ||
( | ||
# ( | ||
# "examples/playbooks/json-schema-fail.yml", | ||
# "playbook", | ||
# [ | ||
# "examples/playbooks/json-schema-fail.yml:2:3: syntax-check 'host' is not a valid attribute for a Play" | ||
# ], | ||
# ), | ||
( | ||
"examples/galaxy.yml", | ||
"galaxy", | ||
["'GPL' is not one of"], | ||
), | ||
), | ||
ids=( | ||
# "playbook-fail", | ||
"galaxy-fail", | ||
), | ||
) | ||
def test_rule(file: str, expected_kind: str, expected: List[str]) -> None: | ||
"""Validate parsing of ansible output.""" | ||
lintable = Lintable(file) | ||
assert lintable.kind == expected_kind | ||
results = ValidateSchemaRule.process_lintable(lintable) | ||
assert len(results) == len(expected), results | ||
for idx, result in enumerate(results): | ||
assert result.filename.endswith(file) | ||
assert expected[idx] in result.message | ||
assert result.tag == f"schema[{expected_kind}]" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
"""Module containing cached JSON schemas.""" | ||
import os | ||
import urllib.request | ||
|
||
from ansiblelint.config import JSON_SCHEMAS | ||
|
||
|
||
def refresh_schemas() -> None: | ||
"""Refresh JSON schemas by downloading latest versions.""" | ||
for kind, url in sorted(JSON_SCHEMAS.items()): | ||
path = f"{os.path.relpath(os.path.dirname(__file__))}/{kind}.json" | ||
print(f"Refreshing {path} ...") | ||
with urllib.request.urlopen(url) as response: | ||
content = response.read().decode("utf-8") | ||
with open(f"{path}", "r+", encoding="utf-8") as f_out: | ||
if f_out.read() != content: | ||
f_out.seek(0) | ||
f_out.write(content) |
Oops, something went wrong.