From d00ca77f7dc2eaed4971e94a12b640c6cbf20e28 Mon Sep 17 00:00:00 2001 From: Antoine Le Morvan Date: Mon, 22 Nov 2021 22:35:48 +0100 Subject: [PATCH 1/2] feat: writing ansible chapter: working with filters --- .../07-working-with-filters.md | 481 ++++++++++++++++++ 1 file changed, 481 insertions(+) create mode 100644 docs/books/learning_ansible/07-working-with-filters.md diff --git a/docs/books/learning_ansible/07-working-with-filters.md b/docs/books/learning_ansible/07-working-with-filters.md new file mode 100644 index 0000000000..fdf7dddc48 --- /dev/null +++ b/docs/books/learning_ansible/07-working-with-filters.md @@ -0,0 +1,481 @@ +--- +title: Working with filters +--- + +# Ansible - Working with filters + +In this chapter you will learn how to transform data with jinja filters. + +**** + +**Objectives**: In this chapter you will learn how to: + +:heavy_check_mark: Transform data structures as dictionnaries or lists; +:heavy_check_mark: Transform variables; + +:checkered_flag: **ansible**, **jinja**, **filters** + +**Knowledge**: :star: :star: :star: +**Complexity**: :star: :star: :star: :star: + +**Reading time**: 20 minutes + +**** + +We have already had the opportunity, during the previous chapters, to use the jinja filters. + +These filters, written in python, allow us to manipulate and transform our ansible variables. + +!!! Note + More information can be [found here](https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html). + +Throughout this chapter, we will use the following playbook to test the different filters presented: + +``` +- name: Manipulating the data + hosts: localhost + gather_facts: false + vars: + zero: 0 + zero_string: "0" + non_zero: 4 + true_booleen: True + true_non_booleen: "True" + false_boolean: False + false_non_boolean: "False" + whatever: "It's false!" + user_name: antoine + my_dictionnary: + key1: value1 + key2: value2 + my_simple_list: + - value_list_1 + - value_list_2 + - value_list_3 + my_simple_list_2: + - value_list_3 + - value_list_4 + - value_list_5 + my_list: + - element: element1 + value: value1 + - element: element2 + value: value2 + + tasks: + - name: Print an integer + debug: + var: zero +``` + +!!! Note + The following is a non-exhaustive list of filters that you are most likely to encounter or need. + Fortunately, there are many others. You could even write your own! + +The playbook will be played as follows: + +``` +ansible-playbook play-filter.yml +``` + +## Converting data + +Data can be converted from one type to another. + +To know the type of a data (the type in python language), you have to use the `type_debug` filter. + +Example: + +``` +- name: Display the type of a variable + debug: + var: true_boolean|type_debug +``` + +which gives us: + +``` +TASK [Display the type of a variable] ****************************************************************** +ok: [localhost] => { + "true_boolean|type_debug": "bool" +} +``` + +It is possible to transform an integer into a string: + +``` +- name: Transforming a variable type + debug: + var: zero|string +``` + +``` +TASK [Transforming a variable type] *************************************************************** +ok: [localhost] => { + "zero|string": "0" +} +``` + +Transform a string into an integer: + +``` +- name: Transforming a variable type + debug: + var: zero_string|int +``` + +or a variable into a boolean: + +``` +- name: Display an integer as a boolean + debug: + var: non_zero | bool + +- name: Display a string as a boolean + debug: + var: true_non_boolean | bool + +- name: Display a string as a boolean + debug: + var: false_non_boolean | bool + +- name: Display a string as a boolean + debug: + var: whatever | bool + +``` + +A character string can be transformed into upper or lower case : + +``` +- name: Lowercase a string of characters + debug: + var: whatever | lower + +- name: Upercase a string of characters + debug: + var: whatever | upper +``` + +which gives us: + +``` +TASK [Lowercase a string of characters] ***************************************************** +ok: [localhost] => { + "whatever | lower": "it's false!" +} + +TASK [Upercase a string of characters] ***************************************************** +ok: [localhost] => { + "whatever | upper": "IT'S FALSE!" +} +``` + +The `replace` filter allows you to replace characters by others. + +Here we remove spaces or even replace a word: + +``` +- name: Replace a character in a string + debug: + var: whatever | replace(" ", "") + +- name: Replace a word in a string + debug: + var: whatever | replace("false", "true") +``` + +which gives us: + +``` +TASK [Replace a character in a string] ***************************************************** +ok: [localhost] => { + "whatever | replace(\" \", \"\")": "It'sfalse!" +} + +TASK [Replace a word in a string] ***************************************************** +ok: [localhost] => { + "whatever | replace(\"false\", \"true\")": "It's true !" +} +``` + +The `split` filter splits a string into a list based on a character: + +``` +- name: Cutting a string of characters + debug: + var: whatever | split(" ", "") +``` + + +``` +TASK [Cutting a string of characters] ***************************************************** +ok: [localhost] => { + "whatever | split(\" \")": [ + "It's", + "false!" + ] +} +``` + +## Join the elements of a list + +It is frequent to have to join the different elements in a single string. +We can then specify a character or a string to insert between each element. + +``` +- name: Joining elements of a list + debug: + var: my_simple_list|join(",") + +- name: Joining elements of a list + debug: + var: my_simple_list|join(" | ") +``` + +which gives us: + +``` +TASK [Joining elements of a list] ***************************************************************** +ok: [localhost] => { + "my_simple_list|join(\",\")": "value_list_1,value_list_2,value_list_3" +} + +TASK [Joining elements of a list] ***************************************************************** +ok: [localhost] => { + "my_simple_list|join(\" | \")": "value_list_1 | value_list_2 | value_list_3" +} + +``` + +## Transforming dictionaries into lists (and vice versa) + +The filters `dict2items` and `itemstodict`, a bit more complex to implement, +are frequently used, especially in loops. + +Note that it is possible to specify the name of the key and of the value to use in the transformation. + +``` +- name: Display a dictionnary + debug: + var: my_dictionnary + +- name: Transforming a dictionary into a list + debug: + var: my_dictionnary | dict2items + +- name: Transforming a dictionary into a list + debug: + var: my_dictionnary | dict2items(key_name='key', value_name='value') + +- name: Transforming a list into a dictionnary + debug: + var: my_list | items2dict(key_name='element', value_name='value') +``` + +``` +TASK [Display a dictionnary] ************************************************************************* +ok: [localhost] => { + "my_dictionnary": { + "key1": "value1", + "key2": "value2" + } +} + +TASK [Transforming a dictionary into a list] ************************************************************* +ok: [localhost] => { + "my_dictionnary | dict2items": [ + { + "key": "key1", + "value": "value1" + }, + { + "key": "key2", + "value": "value2" + } + ] +} + +TASK [Transforming a dictionary into a list] ************************************************************* +ok: [localhost] => { + "mon_dictionnaire | dict2items(key_name='clef', value_name='valeur')": [ + { + "key": "key1", + "value": "value1" + }, + { + "key": "key2", + "value": "value2" + } + ] +} + +TASK [Transforming a list into a dictionnary] ************************************************************ +ok: [localhost] => { + "my_list | items2dict(key_name='element', value_name='value')": { + "element1": "value1", + "element2": "value2" + } +} +``` + +## Working with lists + +It is possible to merge or filter data from one or more lists: + +``` +- name: Merger of two lists + debug: + var: my_simple_list | union(my_simple_list_2) +``` + +``` +ok: [localhost] => { + "my_simple_list | union(my_simple_list_2)": [ + "value_list_1", + "value_list_2", + "value_list_3", + "value_list_4", + "value_list_5" + ] +} +``` + +To keep only the intersection of the 2 lists (the values present in the 2 lists): + +``` +- name: Merger of two lists + debug: + var: my_simple_list | intersect(my_simple_list_2) +``` + +``` +TASK [Merger of two lists] ******************************************************************************* +ok: [localhost] => { + "my_simple_list | intersect(my_simple_list_2)": [ + "value_list_3" + ] +} +``` + +Or on the contrary keep only the difference (the values that do not exist in the second list): + +``` +- name: Merger of two lists + debug: + var: my_simple_list | difference(my_simple_list_2) +``` + +``` +TASK [Merger of two lists] ******************************************************************************* +ok: [localhost] => { + "my_simple_list | difference(my_simple_list_2)": [ + "value_list_1", + "value_list_2", + ] +} +``` + +If your list contains non-unique values, it is also possible to filter them with the `unique` filter. + +``` +- name: Unique value in a list + debug: + var: my_simple_list | unique +``` + +## Transformation json/yaml + +You may have to import json data (from an API for example) or export data in yaml or json. + +``` +- name: Display a variable in yaml + debug: + var: my_list | to_nice_yaml(indent=4) + +- name: Display a variable in json + debug: + var: my_list | to_nice_json(indent=4) +``` + +``` +TASK [Display a variable in yaml] ******************************************************************** +ok: [localhost] => { + "my_list | to_nice_yaml(indent=4)": "- element: element1\n value: value1\n- element: element2\n value: value2\n" +} + +TASK [Display a variable in json] ******************************************************************** +ok: [localhost] => { + "my_list | to_nice_json(indent=4)": "[\n {\n \"element\": \"element1\",\n \"value\": \"value1\"\n },\n {\n \"element\": \"element2\",\n \"value\": \"value2\"\n }\n]" +} +``` + +## Default values, optional variables, protect variables + +You will quickly be confronted with errors in the execution of your playbooks if you do not provide default values for your variables, or if you do not protect them. + +The value of a variable can be substituted for another one if it does not exist with the `default` filter: + +``` +- name: Default value + debug: + var: variablethatdoesnotexists | default(whatever) +``` + +``` +TASK [Default value] ******************************************************************************** +ok: [localhost] => { + "variablethatdoesnotexists | default(whatever)": "It's false!" +} +``` + +Note the presence of the apostrophe `'` which should be protected, for example, if you were using the `shell` module: + +``` +- name: Default value + debug: + var: variablethatdoesnotexists | default(whatever| quote) +``` + +``` +TASK [Default value] ******************************************************************************** +ok: [localhost] => { + "variablethatdoesnotexists | default(whatever|quote)": "'It'\"'\"'s false!'" +} +``` + +Finally, an optional variable in a module can be ignored if it does not exist with the keyword `omit` in the `default` filter, which will save you an error at runtime. + +``` +- name: Add a new user + ansible.builtin.user: + name: "{{ user_name }}" + comment: "{{ user_comment | default(omit) }}" +``` + +## Associate a value according to another one (`ternary`) + +Sometimes you need to use a condition to assign a value to a variable, in which case it is common to go through a `set_fact` step. + +This can be avoided by using the `ternary` filter: + +``` +- name: Default value + debug: + var: (user_name == 'antoine') | ternary('admin', 'normal_user') +``` + +``` +TASK [Default value] ******************************************************************************** +ok: [localhost] => { + "(user_name == 'antoine') | ternary('admin', 'normal_user')": "admin" +} +``` + +## Some other filters + + * `{{ 10000 | random }}`: as its name indicates, gives a random value. + * `{{ my_simple_list | first }}`: extracts the first element of the list. + * `{{ my_simple_list | length }}`: gives the length (of a list or a string). + * `{{ ip_list | ansible.netcommon.ipv4 }}`: only displays v4 IPs. Without dwelling on this, if you need, there are many filters dedicated to the network. + * `{{ user_password | password_hash('sha512') }}`: generates a hashed password in sha512. From 74a5c986ca9b1c57a0b29569027e4ec826bcce04 Mon Sep 17 00:00:00 2001 From: Steven Spencer Date: Tue, 23 Nov 2021 11:04:00 -0600 Subject: [PATCH 2/2] Edit 391 - chapter 7 of Ansible book * updated the meta to include author, contributors, update, as well as capitalizing the section so that it shows up nicely in the side menu * fixed spelling of "dictionary" * fixed issue with space before colon in descriptions (not in code). There was only one of these. * Editing pass - formatting: all looked good when displayed using mkdocs * Editing pass - reading: all content looked good, no changes needed * One change to French code not fully translated to English. fixed --- .../07-working-with-filters.md | 29 ++++++++++--------- 1 file changed, 16 insertions(+), 13 deletions(-) diff --git a/docs/books/learning_ansible/07-working-with-filters.md b/docs/books/learning_ansible/07-working-with-filters.md index fdf7dddc48..ebc0ae3280 100644 --- a/docs/books/learning_ansible/07-working-with-filters.md +++ b/docs/books/learning_ansible/07-working-with-filters.md @@ -1,5 +1,8 @@ --- -title: Working with filters +title: Working With Filters +author: Antoine Le Morvan +contributors: Steven Spencer +update: 11-23-2021 --- # Ansible - Working with filters @@ -45,7 +48,7 @@ Throughout this chapter, we will use the following playbook to test the differen false_non_boolean: "False" whatever: "It's false!" user_name: antoine - my_dictionnary: + my_dictionary: key1: value1 key2: value2 my_simple_list: @@ -145,7 +148,7 @@ or a variable into a boolean: ``` -A character string can be transformed into upper or lower case : +A character string can be transformed into upper or lower case: ``` - name: Lowercase a string of characters @@ -256,27 +259,27 @@ are frequently used, especially in loops. Note that it is possible to specify the name of the key and of the value to use in the transformation. ``` -- name: Display a dictionnary +- name: Display a dictionary debug: - var: my_dictionnary + var: my_dictionary - name: Transforming a dictionary into a list debug: - var: my_dictionnary | dict2items + var: my_dictionary | dict2items - name: Transforming a dictionary into a list debug: - var: my_dictionnary | dict2items(key_name='key', value_name='value') + var: my_dictionary | dict2items(key_name='key', value_name='value') -- name: Transforming a list into a dictionnary +- name: Transforming a list into a dictionary debug: var: my_list | items2dict(key_name='element', value_name='value') ``` ``` -TASK [Display a dictionnary] ************************************************************************* +TASK [Display a dictionary] ************************************************************************* ok: [localhost] => { - "my_dictionnary": { + "my_dictionary": { "key1": "value1", "key2": "value2" } @@ -284,7 +287,7 @@ ok: [localhost] => { TASK [Transforming a dictionary into a list] ************************************************************* ok: [localhost] => { - "my_dictionnary | dict2items": [ + "my_dictionary | dict2items": [ { "key": "key1", "value": "value1" @@ -298,7 +301,7 @@ ok: [localhost] => { TASK [Transforming a dictionary into a list] ************************************************************* ok: [localhost] => { - "mon_dictionnaire | dict2items(key_name='clef', value_name='valeur')": [ + "my_dictionary | dict2items (key_name = 'key', value_name = 'value')": [ { "key": "key1", "value": "value1" @@ -310,7 +313,7 @@ ok: [localhost] => { ] } -TASK [Transforming a list into a dictionnary] ************************************************************ +TASK [Transforming a list into a dictionary] ************************************************************ ok: [localhost] => { "my_list | items2dict(key_name='element', value_name='value')": { "element1": "value1",