Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

combine filter: fine list handling (option b) #57894

Open
wants to merge 5 commits into
base: devel
from

Conversation

Projects
None yet
6 participants
@tchernomax
Copy link
Contributor

commented Jun 15, 2019

SUMMARY

see:

Solution (b)

args:

  • recursive: same as before
  • list_merge:
    • keep: same as before
    • append, prepend: list are appended
    • append_np, prepend_np: only elements that isn't already present in left list are append (_np stands for "non present")

README: will come if this implementation is validated.

I didn't implement 'uniq' as I think 'append_np' and 'prepend_np' are more useful, and are equivalent to unique if list are already filtered from duplication.

non backward compatible: [dict1, [dict2]] | combine doesn't work anymore, because I thought it shouldn't. If you disagree, flatten(terms, levels=1) → flatten(terms) and it's fixed.

ISSUE TYPE
  • Feature Pull Request
COMPONENT NAME

component =lib/ansible/plugins/filter/core.py

ADDITIONAL INFORMATION
max@mde-oxalide % cat toto.yml
- hosts: localhost
  gather_facts: false
  vars:
    toto_default:
      a:
        a:
          a: default_value
          b: default_value
      b:
        - 1
        - 1
        - 2
    toto:
      a:
        a:
          b: toto_value
          c: toto_value
      b:
        - 2
        - 3
        - 3
        - 4: toto_value
  tasks:
    - name: recursive=False, list_merge='keep'
      debug:
        msg: "{{toto_default|toto_combine(toto, recursive=False, list_merge='keep')}}"
    - name: recursive=False, list_merge='append'
      debug:
        msg: "{{toto_default|toto_combine(toto, recursive=False, list_merge='append')}}"
    - name: recursive=False, list_merge='prepend'
      debug:
        msg: "{{toto_default|toto_combine(toto, recursive=False, list_merge='prepend')}}"
    - name: recursive=False, list_merge='append_np'
      debug:
        msg: "{{toto_default|toto_combine(toto, recursive=False, list_merge='append_np')}}"
    - name: recursive=False, list_merge='prepend_np'
      debug:
        msg: "{{toto_default|toto_combine(toto, recursive=False, list_merge='prepend_np')}}"

    - name: recursive=True, list_merge='keep'
      debug:
        msg: "{{toto_default|toto_combine(toto, recursive=True, list_merge='keep')}}"
    - name: recursive=True, list_merge='append'
      debug:
        msg: "{{toto_default|toto_combine(toto, recursive=True, list_merge='append')}}"
    - name: recursive=True, list_merge='prepend'
      debug:
        msg: "{{toto_default|toto_combine(toto, recursive=True, list_merge='prepend')}}"
    - name: recursive=True, list_merge='append_np'
      debug:
        msg: "{{toto_default|toto_combine(toto, recursive=True, list_merge='append_np')}}"
    - name: recursive=True, list_merge='prepend_np'
      debug:
        msg: "{{toto_default|toto_combine(toto, recursive=True, list_merge='prepend_np')}}"
PLAY [localhost] ***********************************************************************

TASK [recursive=False, list_merge='keep'] **********************************************
ok: [localhost] => {
    "msg": {
        "a": {
            "a": {
                "b": "toto_value",
                "c": "toto_value"
            }
        },
        "b": [
            2,
            3,
            3,
            {
                "4": "toto_value"
            }
        ]
    }
}

TASK [recursive=False, list_merge='append'] ********************************************
ok: [localhost] => {
    "msg": {
        "a": {
            "a": {
                "b": "toto_value",
                "c": "toto_value"
            }
        },
        "b": [
            1,
            1,
            2,
            2,
            3,
            3,
            {
                "4": "toto_value"
            }
        ]
    }
}

TASK [recursive=False, list_merge='prepend'] *******************************************
ok: [localhost] => {
    "msg": {
        "a": {
            "a": {
                "b": "toto_value",
                "c": "toto_value"
            }
        },
        "b": [
            1,
            1,
            2,
            2,
            3,
            3,
            {
                "4": "toto_value"
            }
        ]
    }
}

TASK [recursive=False, list_merge='append_np'] *****************************************
ok: [localhost] => {
    "msg": {
        "a": {
            "a": {
                "b": "toto_value",
                "c": "toto_value"
            }
        },
        "b": [
            2,
            3,
            3,
            {
                "4": "toto_value"
            },
            1,
            1
        ]
    }
}

TASK [recursive=False, list_merge='prepend_np'] ****************************************
ok: [localhost] => {
    "msg": {
        "a": {
            "a": {
                "b": "toto_value",
                "c": "toto_value"
            }
        },
        "b": [
            1,
            1,
            2,
            3,
            3,
            {
                "4": "toto_value"
            }
        ]
    }
}

TASK [recursive=True, list_merge='keep'] ***********************************************
ok: [localhost] => {
    "msg": {
        "a": {
            "a": {
                "a": "default_value",
                "b": "toto_value",
                "c": "toto_value"
            }
        },
        "b": [
            2,
            3,
            3,
            {
                "4": "toto_value"
            }
        ]
    }
}

TASK [recursive=True, list_merge='append'] *********************************************
ok: [localhost] => {
    "msg": {
        "a": {
            "a": {
                "a": "default_value",
                "b": "toto_value",
                "c": "toto_value"
            }
        },
        "b": [
            1,
            1,
            2,
            2,
            3,
            3,
            {
                "4": "toto_value"
            }
        ]
    }
}

TASK [recursive=True, list_merge='prepend'] ********************************************
ok: [localhost] => {
    "msg": {
        "a": {
            "a": {
                "a": "default_value",
                "b": "toto_value",
                "c": "toto_value"
            }
        },
        "b": [
            1,
            1,
            2,
            2,
            3,
            3,
            {
                "4": "toto_value"
            }
        ]
    }
}

TASK [recursive=True, list_merge='append_np'] ******************************************
ok: [localhost] => {
    "msg": {
        "a": {
            "a": {
                "a": "default_value",
                "b": "toto_value",
                "c": "toto_value"
            }
        },
        "b": [
            2,
            3,
            3,
            {
                "4": "toto_value"
            },
            1,
            1
        ]
    }
}

TASK [recursive=True, list_merge='prepend_np'] *****************************************
ok: [localhost] => {
    "msg": {
        "a": {
            "a": {
                "a": "default_value",
                "b": "toto_value",
                "c": "toto_value"
            }
        },
        "b": [
            1,
            1,
            2,
            3,
            3,
            {
                "4": "toto_value"
            }
        ]
    }
}

PLAY RECAP *****************************************************************************
localhost                  : ok=10   changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
@ansibot

This comment has been minimized.

Copy link
Contributor

commented Jun 15, 2019

The test ansible-test sanity --test pylint [explain] failed with 1 error:

lib/ansible/plugins/filter/core.py:314:8: undefined-variable Undefined variable '_validate_mutable_mappings'

The test ansible-test sanity --test pep8 [explain] failed with 3 errors:

lib/ansible/plugins/filter/core.py:320:17: E117 over-indented
lib/ansible/plugins/filter/core.py:362:22: E261 at least two spaces before inline comment
lib/ansible/plugins/filter/core.py:374:1: E302 expected 2 blank lines, found 1

click here for bot help

@ansibot ansibot added needs_revision and removed core_review labels Jun 15, 2019

@tchernomax tchernomax force-pushed the tchernomax:combine_merge_array_b branch 4 times, most recently from efd032e to af365bb Jun 15, 2019

Show resolved Hide resolved lib/ansible/plugins/filter/core.py Outdated

@tchernomax tchernomax force-pushed the tchernomax:combine_merge_array_b branch 4 times, most recently from da849c9 to c3f0bbe Jun 18, 2019

@ansibot ansibot added the stale_ci label Jun 27, 2019

@ansibot ansibot added the stale_review label Jul 5, 2019

@tchernomax tchernomax changed the title combine filter: fine list handling WIP combine filter: fine list handling (option b) Jul 7, 2019

@ansibot ansibot added the WIP label Jul 7, 2019

@tchernomax

This comment has been minimized.

Copy link
Contributor Author

commented Jul 7, 2019

ping @bcoca (I made the change you asked for)

Show resolved Hide resolved lib/ansible/utils/vars.py Outdated
@ansibot

This comment has been minimized.

Copy link
Contributor

commented Jul 11, 2019

The test ansible-test sanity --test pylint [explain] failed with 1 error:

test/units/utils/test_vars.py:105:0: syntax-error invalid syntax (<unknown>, line 105)

The test ansible-test sanity --test compile --python 2.6 [explain] failed with 1 error:

test/units/utils/test_vars.py:105:19: SyntaxError: "b": [1, 1, 2, 3]

The test ansible-test sanity --test compile --python 2.7 [explain] failed with 1 error:

test/units/utils/test_vars.py:105:19: SyntaxError: "b": [1, 1, 2, 3]

The test ansible-test sanity --test compile --python 3.5 [explain] failed with 1 error:

test/units/utils/test_vars.py:105:19: SyntaxError: "b": [1, 1, 2, 3]

The test ansible-test sanity --test compile --python 3.6 [explain] failed with 1 error:

test/units/utils/test_vars.py:105:19: SyntaxError: "b": [1, 1, 2, 3]

The test ansible-test sanity --test compile --python 3.7 [explain] failed with 1 error:

test/units/utils/test_vars.py:105:19: SyntaxError: "b": [1, 1, 2, 3]

The test ansible-test sanity --test compile --python 3.8 [explain] failed with 1 error:

test/units/utils/test_vars.py:105:17: SyntaxError: "b": [1, 1, 2, 3]

The test ansible-test sanity --test pep8 [explain] failed with 28 errors:

lib/ansible/utils/vars.py:173:13: E265 block comment should start with '# '
test/units/utils/test_vars.py:38:5: E303 too many blank lines (2)
test/units/utils/test_vars.py:97:13: E126 continuation line over-indented for hanging indent
test/units/utils/test_vars.py:117:9: E121 continuation line under-indented for hanging indent
test/units/utils/test_vars.py:126:17: E126 continuation line over-indented for hanging indent
test/units/utils/test_vars.py:135:17: E123 closing bracket does not match indentation of opening bracket's line
test/units/utils/test_vars.py:148:17: E126 continuation line over-indented for hanging indent
test/units/utils/test_vars.py:150:17: E123 closing bracket does not match indentation of opening bracket's line
test/units/utils/test_vars.py:157:17: E126 continuation line over-indented for hanging indent
test/units/utils/test_vars.py:159:17: E123 closing bracket does not match indentation of opening bracket's line
test/units/utils/test_vars.py:166:17: E126 continuation line over-indented for hanging indent
test/units/utils/test_vars.py:168:17: E123 closing bracket does not match indentation of opening bracket's line
test/units/utils/test_vars.py:175:17: E126 continuation line over-indented for hanging indent
test/units/utils/test_vars.py:177:17: E123 closing bracket does not match indentation of opening bracket's line
test/units/utils/test_vars.py:184:17: E126 continuation line over-indented for hanging indent
test/units/utils/test_vars.py:186:17: E123 closing bracket does not match indentation of opening bracket's line
test/units/utils/test_vars.py:193:17: E126 continuation line over-indented for hanging indent
test/units/utils/test_vars.py:202:17: E123 closing bracket does not match indentation of opening bracket's line
test/units/utils/test_vars.py:209:17: E126 continuation line over-indented for hanging indent
test/units/utils/test_vars.py:218:17: E123 closing bracket does not match indentation of opening bracket's line
test/units/utils/test_vars.py:225:17: E126 continuation line over-indented for hanging indent
test/units/utils/test_vars.py:234:17: E123 closing bracket does not match indentation of opening bracket's line
test/units/utils/test_vars.py:241:17: E126 continuation line over-indented for hanging indent
test/units/utils/test_vars.py:250:17: E123 closing bracket does not match indentation of opening bracket's line
test/units/utils/test_vars.py:257:17: E126 continuation line over-indented for hanging indent
test/units/utils/test_vars.py:266:17: E123 closing bracket does not match indentation of opening bracket's line
test/units/utils/test_vars.py:273:17: E126 continuation line over-indented for hanging indent
test/units/utils/test_vars.py:282:17: E123 closing bracket does not match indentation of opening bracket's line

click here for bot help

@tchernomax tchernomax force-pushed the tchernomax:combine_merge_array_b branch from c88673f to b8e4583 Jul 12, 2019

@ansibot

This comment has been minimized.

Copy link
Contributor

commented Jul 12, 2019

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

lib/ansible/utils/vars.py:173:13: E265 block comment should start with '# '

click here for bot help

@tchernomax tchernomax force-pushed the tchernomax:combine_merge_array_b branch 2 times, most recently from 8182718 to 3c5faf8 Jul 12, 2019

@ansibot

This comment has been minimized.

Copy link
Contributor

commented Jul 13, 2019

@tchernomax tchernomax force-pushed the tchernomax:combine_merge_array_b branch from 3c5faf8 to bd2b05b Jul 13, 2019

@tchernomax tchernomax changed the title WIP combine filter: fine list handling (option b) combine filter: fine list handling (option b) Jul 13, 2019

@tchernomax

This comment has been minimized.

Copy link
Contributor Author

commented Jul 13, 2019

ready_for_review

@tchernomax tchernomax force-pushed the tchernomax:combine_merge_array_b branch from 59a84de to b251f8f Jul 13, 2019

@ansibot ansibot added core_review and removed needs_revision labels Jul 13, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.