Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add JSON patching module #52931

Open
wants to merge 1 commit into
base: devel
from

Conversation

Projects
None yet
4 participants
@ParticleDecay
Copy link

ParticleDecay commented Feb 25, 2019

SUMMARY

Currently, there is no ability to manipulate JSON files. This module adds JSON patching based on RFC 6902, with notation conforming to RFC 6901 with one notable exception, that I have proposed to the RFC maintainers as well for adoption (RFC 6901 is still in "PROPOSED STANDARD").

ISSUE TYPE
  • New Module Pull Request
COMPONENT NAME

jsonpatch

ADDITIONAL INFORMATION

As a simple example, consider this JSON object in a file named 'test.json':

{
  "foo": "bar"
}

The following would add a new member to the JSON:

- name: Add or replace 'baz' member
  jsonpatch:
    src: "test.json"
    operations:
      - op: add
        path: "/baz"
        value: "quz"

This would result in:

{
  "foo": "bar",
  "baz": "quz"
}

In addition to add, I have included support for all of the RFC standard operations. For additional information and examples, see github.com/ParticleDecay/ansible-jsonpatch. See also RFC 6901, RFC 6902, and jsonpatch.com.

@ansibot

This comment has been minimized.

Copy link
Contributor

ansibot commented Feb 25, 2019

The test ansible-test sanity --test pylint [explain] failed with 1 error:

test/units/modules/files/test_jsonpatch.py:288:0: trailing-whitespace Trailing whitespace

The test ansible-test sanity --test pylint [explain] failed with 9 errors:

lib/ansible/modules/files/jsonpatch.py:168:0: trailing-whitespace Trailing whitespace
lib/ansible/modules/files/jsonpatch.py:199:0: trailing-whitespace Trailing whitespace
lib/ansible/modules/files/jsonpatch.py:226:0: trailing-whitespace Trailing whitespace
lib/ansible/modules/files/jsonpatch.py:230:0: trailing-whitespace Trailing whitespace
lib/ansible/modules/files/jsonpatch.py:249:0: trailing-whitespace Trailing whitespace
lib/ansible/modules/files/jsonpatch.py:323:0: trailing-whitespace Trailing whitespace
lib/ansible/modules/files/jsonpatch.py:389:0: trailing-whitespace Trailing whitespace
lib/ansible/modules/files/jsonpatch.py:453:0: missing-final-newline Final newline missing
lib/ansible/modules/files/jsonpatch.py:453:0: mixed-line-endings Mixed line endings LF and CRLF

The test ansible-test sanity --test ansible-doc --python 2.6 [explain] failed with 1 error:

lib/ansible/modules/files/jsonpatch.py:0:0: missing documentation (or could not parse documentation): sequence item 1: expected string or Unicode, AnsibleMapping found

The test ansible-test sanity --test ansible-doc --python 2.7 [explain] failed with 1 error:

lib/ansible/modules/files/jsonpatch.py:0:0: missing documentation (or could not parse documentation): sequence item 1: expected string or Unicode, AnsibleMapping found

The test ansible-test sanity --test ansible-doc --python 3.5 [explain] failed with 1 error:

lib/ansible/modules/files/jsonpatch.py:0:0: missing documentation (or could not parse documentation): sequence item 1: expected str instance, AnsibleMapping found

The test ansible-test sanity --test ansible-doc --python 3.6 [explain] failed with 1 error:

lib/ansible/modules/files/jsonpatch.py:0:0: missing documentation (or could not parse documentation): sequence item 1: expected str instance, AnsibleMapping found

The test ansible-test sanity --test ansible-doc --python 3.7 [explain] failed with 1 error:

lib/ansible/modules/files/jsonpatch.py:0:0: missing documentation (or could not parse documentation): sequence item 1: expected str instance, AnsibleMapping found

The test ansible-test sanity --test ansible-doc --python 3.8 [explain] failed with 1 error:

lib/ansible/modules/files/jsonpatch.py:0:0: missing documentation (or could not parse documentation): sequence item 1: expected str instance, AnsibleMapping found

The test ansible-test sanity --test docs-build [explain] failed with the error:

Command "/usr/bin/python test/sanity/code-smell/docs-build.py" returned exit status 1.
>>> Standard Error
Command 'make singlehtmldocs' failed with status code: 2
--> Standard Output
cat _themes/srtd/static/css/theme.css | sed -e 's/^[ 	]*//g; s/[ 	]*$//g; s/\([:{;,]\) /\1/g; s/ {/{/g; s/\/\*.*\*\///g; /^$/d' | sed -e :a -e '$!N; s/\n\(.\)/\1/; ta' > _themes/srtd/static/css/theme.min.css
PYTHONPATH=../../lib ../bin/dump_config.py --template-file=../templates/config.rst.j2 --output-dir=rst/reference_appendices/ -d ../../lib/ansible/config/base.yml
mkdir -p rst/cli
PYTHONPATH=../../lib ../bin/generate_man.py --template-file=../templates/cli_rst.j2 --output-dir=rst/cli/ --output-format rst ../../lib/ansible/cli/*.py
PYTHONPATH=../../lib ../bin/dump_keywords.py --template-dir=../templates --output-dir=rst/reference_appendices/ -d ./keyword_desc.yml
PYTHONPATH=../../lib ../bin/plugin_formatter.py -t rst --template-dir=../templates --module-dir=../../lib/ansible/modules -o rst/modules/ 
Evaluating module files...
rendering: blockinfile (143 previous rendering line(s) omitted)
Makefile:93: recipe for target 'modules' failed
--> Standard Error
Traceback (most recent call last):
  File "../bin/plugin_formatter.py", line 107, in rst_ify
    t = _ITALIC.sub(r"*\1*", text)
TypeError: expected string or bytes-like object

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "../bin/plugin_formatter.py", line 774, in <module>
    main()
  File "../bin/plugin_formatter.py", line 761, in main
    process_plugins(plugin_info, templates, outputname, output_dir, options.ansible_version, plugin_type)
  File "../bin/plugin_formatter.py", line 557, in process_plugins
    text = templates['plugin'].render(doc)
  File "/usr/local/lib/python3.6/dist-packages/jinja2/asyncsupport.py", line 76, in render
    return original_render(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/usr/local/lib/python3.6/dist-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/dist-packages/jinja2/_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "../templates/plugin.rst.j2", line 57, in top-level template code
    - @{ desc | rst_ify }@
  File "../bin/plugin_formatter.py", line 115, in rst_ify
    raise AnsibleError("Could not process (%s) : %s" % (text, e))
ansible.errors.AnsibleError: Could not process ({'RFC 6902': 'https://tools.ietf.org/html/rfc6902'}) : expected string or bytes-like object
make: *** [modules] Error 1

The test ansible-test sanity --test boilerplate [explain] failed with 2 errors:

lib/ansible/modules/files/jsonpatch.py:0:0: missing: __metaclass__ = type
lib/ansible/modules/files/jsonpatch.py:0:0: missing: from __future__ import (absolute_import, division, print_function)

The test ansible-test sanity --test no-underscore-variable [explain] failed with 2 errors:

lib/ansible/modules/files/jsonpatch.py:367:18: use `dummy` instead of `_` for a variable name
lib/ansible/modules/files/jsonpatch.py:415:21: use `dummy` instead of `_` for a variable name

The test ansible-test sanity --test pep8 [explain] failed with 28 errors:

lib/ansible/modules/files/jsonpatch.py:87:4: W291 trailing whitespace
lib/ansible/modules/files/jsonpatch.py:168:1: W293 blank line contains whitespace
lib/ansible/modules/files/jsonpatch.py:199:1: W293 blank line contains whitespace
lib/ansible/modules/files/jsonpatch.py:226:1: W293 blank line contains whitespace
lib/ansible/modules/files/jsonpatch.py:230:1: W293 blank line contains whitespace
lib/ansible/modules/files/jsonpatch.py:249:1: W293 blank line contains whitespace
lib/ansible/modules/files/jsonpatch.py:323:1: W293 blank line contains whitespace
lib/ansible/modules/files/jsonpatch.py:389:1: W293 blank line contains whitespace
lib/ansible/modules/files/jsonpatch.py:452:1: E305 expected 2 blank lines after class or function definition, found 1
lib/ansible/modules/files/jsonpatch.py:453:11: W292 no newline at end of file
test/units/modules/files/test_jsonpatch.py:13:1: E122 continuation line missing indentation or outdented
test/units/modules/files/test_jsonpatch.py:14:3: E121 continuation line under-indented for hanging indent
test/units/modules/files/test_jsonpatch.py:15:5: E121 continuation line under-indented for hanging indent
test/units/modules/files/test_jsonpatch.py:16:7: E121 continuation line under-indented for hanging indent
test/units/modules/files/test_jsonpatch.py:23:5: E121 continuation line under-indented for hanging indent
test/units/modules/files/test_jsonpatch.py:24:7: E121 continuation line under-indented for hanging indent
test/units/modules/files/test_jsonpatch.py:31:5: E121 continuation line under-indented for hanging indent
test/units/modules/files/test_jsonpatch.py:32:7: E121 continuation line under-indented for hanging indent
test/units/modules/files/test_jsonpatch.py:33:9: E121 continuation line under-indented for hanging indent
test/units/modules/files/test_jsonpatch.py:37:9: E121 continuation line under-indented for hanging indent
test/units/modules/files/test_jsonpatch.py:41:9: E121 continuation line under-indented for hanging indent
test/units/modules/files/test_jsonpatch.py:49:5: E266 too many leading '#' for block comment
test/units/modules/files/test_jsonpatch.py:100:5: E266 too many leading '#' for block comment
test/units/modules/files/test_jsonpatch.py:131:5: E266 too many leading '#' for block comment
test/units/modules/files/test_jsonpatch.py:150:5: E266 too many leading '#' for block comment
test/units/modules/files/test_jsonpatch.py:192:5: E266 too many leading '#' for block comment
test/units/modules/files/test_jsonpatch.py:223:5: E266 too many leading '#' for block comment
test/units/modules/files/test_jsonpatch.py:288:1: W293 blank line contains whitespace

The test ansible-test sanity --test validate-modules [explain] failed with 5 errors:

lib/ansible/modules/files/jsonpatch.py:0:0: E305 DOCUMENTATION.description.1: expected str @ data['description'][1]. Got {'RFC 6902': 'https://tools.ietf.org/html/rfc6902'}
lib/ansible/modules/files/jsonpatch.py:0:0: E307 version_added should be '2.8'. Currently '2.7'
lib/ansible/modules/files/jsonpatch.py:0:0: E312 No RETURN provided
lib/ansible/modules/files/jsonpatch.py:7:0: E106 Import found before documentation variables. All imports must appear below DOCUMENTATION/EXAMPLES/RETURN/ANSIBLE_METADATA.
lib/ansible/modules/files/jsonpatch.py:7:0: E107 Imports should be directly below DOCUMENTATION/EXAMPLES/RETURN/ANSIBLE_METADATA.

The test ansible-test sanity --test yamllint [explain] failed with 4 errors:

lib/ansible/modules/files/jsonpatch.py:98:1: key-duplicates EXAMPLES: duplication of key "jsonpatch" in mapping
lib/ansible/modules/files/jsonpatch.py:106:1: key-duplicates EXAMPLES: duplication of key "jsonpatch" in mapping
lib/ansible/modules/files/jsonpatch.py:113:1: key-duplicates EXAMPLES: duplication of key "jsonpatch" in mapping
lib/ansible/modules/files/jsonpatch.py:121:1: key-duplicates EXAMPLES: duplication of key "jsonpatch" in mapping

click here for bot help

@ansibot

This comment has been minimized.

Copy link
Contributor

ansibot commented Feb 25, 2019

The test ansible-test sanity --test validate-modules [explain] failed with 1 error:

lib/ansible/modules/files/jsonpatch.py:0:0: E312 No RETURN provided

The test ansible-test sanity --test yamllint [explain] failed with 4 errors:

lib/ansible/modules/files/jsonpatch.py:95:1: key-duplicates EXAMPLES: duplication of key "jsonpatch" in mapping
lib/ansible/modules/files/jsonpatch.py:103:1: key-duplicates EXAMPLES: duplication of key "jsonpatch" in mapping
lib/ansible/modules/files/jsonpatch.py:110:1: key-duplicates EXAMPLES: duplication of key "jsonpatch" in mapping
lib/ansible/modules/files/jsonpatch.py:118:1: key-duplicates EXAMPLES: duplication of key "jsonpatch" in mapping

click here for bot help

@ParticleDecay ParticleDecay force-pushed the ParticleDecay:jsonpatch branch from c1e38a5 to 74adbea Feb 26, 2019

@ansibot

This comment has been minimized.

Copy link
Contributor

ansibot commented Feb 26, 2019

The test ansible-test sanity --test yamllint [explain] failed with 3 errors:

lib/ansible/modules/files/jsonpatch.py:96:1: key-duplicates EXAMPLES: duplication of key "jsonpatch" in mapping
lib/ansible/modules/files/jsonpatch.py:103:1: key-duplicates EXAMPLES: duplication of key "jsonpatch" in mapping
lib/ansible/modules/files/jsonpatch.py:112:1: key-duplicates EXAMPLES: duplication of key "jsonpatch" in mapping

click here for bot help

Add JSON patching module
Fix linting issues in jsonpatch module

Add RETURN block and change some duplicate keys

Updating jsonpatch files to fix errors

@ParticleDecay ParticleDecay force-pushed the ParticleDecay:jsonpatch branch from 74adbea to a7d1431 Feb 26, 2019

@ansibot ansibot added core_review and removed needs_revision labels Feb 26, 2019

@desimaniac

This comment has been minimized.

Copy link

desimaniac commented Mar 3, 2019

+1 For this

@ansibot ansibot added the files label Mar 3, 2019

@ansibot ansibot added the stale_ci label Mar 11, 2019

@ParticleDecay ParticleDecay reopened this Mar 20, 2019

@ParticleDecay

This comment has been minimized.

Copy link
Author

ParticleDecay commented Mar 20, 2019

Reopening to get ci retested

@ansibot ansibot removed the stale_ci label Mar 20, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.