lineinfile not working with `insertbefore: BOF` in Ansible 2.5.0 #38219

craigfrancis · 2018-04-03T14:31:52Z

ISSUE TYPE

Bug Report

COMPONENT NAME

lineinfile

ANSIBLE VERSION

ansible 2.5.0
config file = /Volumes/WebServer/Projects/ansible.local/ansible.cfg
configured module search path = [u'/Users/craig/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /Library/Python/2.7/site-packages/ansible
executable location = /usr/local/bin/ansible
python version = 2.7.10 (default, Oct 6 2017, 22:29:07) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]

CONFIGURATION

ANSIBLE_PIPELINING(/Users/craig/Downloads/ansible/ansible.cfg) = True
DEFAULT_HOST_LIST(/Users/craig/Downloads/ansible/ansible.cfg) = [u'/Users/craig/Downloads/ansible/inventory']
RETRY_FILES_ENABLED(/Users/craig/Downloads/ansible/ansible.cfg) = False

OS / ENVIRONMENT

MacOS 10.13.4 (17E199)

SUMMARY

When looking for a line that is at the beginning of the file (BOF), it doesn't always detect it exists, or insert it in the correct location.

STEPS TO REPRODUCE

    - name: "Example BOF"
      lineinfile:
        dest: "/path/to/file.txt"
        insertbefore: BOF
        line: "Hello"

Create the dest file so it's empty: First run, the line is added (good); Second time, "An exception occurred during task execution" ... "The error was: IndexError: list index out of range".

Replace the contents of this file with a newline, followed by some text (i.e. 2 lines, but the first line is blank): First run, the line is added (good); Second time the line is added again.

Replace the contents of this file with 3 lines of random text: First run, the line is added (good); Second time the line is added again (on the 4th line).

EXPECTED RESULTS

When the line is missing, it should be added to the file... but once it exists, it shouldn't be added again.

ACTUAL RESULTS

Depends on the contents of the file before hand.

Video of results

The text was updated successfully, but these errors were encountered:

ansibot · 2018-04-03T14:44:14Z

Files identified in the description:

lib/ansible/modules/files/lineinfile.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

ansibot · 2018-04-03T14:44:14Z

cc @ahtik @dhozac
click here for bot help

apollo13 · 2018-04-09T11:55:03Z

Full playbook to reproduce:

---

- name: test
  hosts: all
  gather_facts: no

  tasks:
    - shell: echo 123 > test_file creates=test_file

    - lineinfile:
        dest: test_file
        line: 123
        insertbefore: BOF

Error:

ansible-playbook 2.5.0
  config file = None
  configured module search path = ['/home/florian/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/florian/.local/venvs/ansible/lib/python3.6/site-packages/ansible
  executable location = /home/florian/.local/bin/ansible-playbook
  python version = 3.6.4 (default, Jan  5 2018, 02:13:53) [GCC 7.2.0]
No config file found; using defaults
Parsed /home/florian/sources/test/inventory inventory source with ini plugin

PLAYBOOK: site.yml *********************************************************************************************************************************************************************************************
1 plays in site.yml

PLAY [test] ****************************************************************************************************************************************************************************************************
META: ran handlers

TASK [shell] ***************************************************************************************************************************************************************************************************
task path: /home/florian/sources/test/site.yml:8
Using module file /home/florian/.local/venvs/ansible/lib/python3.6/site-packages/ansible/modules/commands/command.py
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: florian
<localhost> EXEC /bin/sh -c 'echo ~ && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/florian/.ansible/tmp/ansible-tmp-1523274750.470957-227748945958289 `" && echo ansible-tmp-1523274750.470957-227748945958289="` echo /home/florian/.ansible/tmp/ansible-tmp-1523274750.470957-227748945958289 `" ) && sleep 0'
<localhost> PUT /home/florian/.ansible/tmp/ansible-local-15912i0yzblad/tmpnyoyhojz TO /home/florian/.ansible/tmp/ansible-tmp-1523274750.470957-227748945958289/command.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/florian/.ansible/tmp/ansible-tmp-1523274750.470957-227748945958289/ /home/florian/.ansible/tmp/ansible-tmp-1523274750.470957-227748945958289/command.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /home/florian/.ansible/tmp/ansible-tmp-1523274750.470957-227748945958289/command.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/florian/.ansible/tmp/ansible-tmp-1523274750.470957-227748945958289/ > /dev/null 2>&1 && sleep 0'
ok: [localhost] => {
    "changed": false,
    "cmd": "echo 123 > test_file",
    "invocation": {
        "module_args": {
            "_raw_params": "echo 123 > test_file",
            "_uses_shell": true,
            "chdir": null,
            "creates": "test_file",
            "executable": null,
            "removes": null,
            "stdin": null,
            "warn": true
        }
    },
    "rc": 0,
    "stdout": "skipped, since test_file exists",
    "stdout_lines": [
        "skipped, since test_file exists"
    ]
}

TASK [lineinfile] **********************************************************************************************************************************************************************************************
task path: /home/florian/sources/test/site.yml:10
Using module file /home/florian/.local/venvs/ansible/lib/python3.6/site-packages/ansible/modules/files/lineinfile.py
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: florian
<localhost> EXEC /bin/sh -c 'echo ~ && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/florian/.ansible/tmp/ansible-tmp-1523274750.7862477-66171061825838 `" && echo ansible-tmp-1523274750.7862477-66171061825838="` echo /home/florian/.ansible/tmp/ansible-tmp-1523274750.7862477-66171061825838 `" ) && sleep 0'
<localhost> PUT /home/florian/.ansible/tmp/ansible-local-15912i0yzblad/tmp9acry5n4 TO /home/florian/.ansible/tmp/ansible-tmp-1523274750.7862477-66171061825838/lineinfile.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/florian/.ansible/tmp/ansible-tmp-1523274750.7862477-66171061825838/ /home/florian/.ansible/tmp/ansible-tmp-1523274750.7862477-66171061825838/lineinfile.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /home/florian/.ansible/tmp/ansible-tmp-1523274750.7862477-66171061825838/lineinfile.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/florian/.ansible/tmp/ansible-tmp-1523274750.7862477-66171061825838/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_Ly5cPo/ansible_module_lineinfile.py", line 512, in <module>
    main()
  File "/tmp/ansible_Ly5cPo/ansible_module_lineinfile.py", line 503, in main
    ins_aft, ins_bef, create, backup, backrefs, firstmatch)
  File "/tmp/ansible_Ly5cPo/ansible_module_lineinfile.py", line 337, in present
    elif b_lines[index[1] - 1].rstrip(b('\r\n')) != b_line:
IndexError: list index out of range

fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_Ly5cPo/ansible_module_lineinfile.py\", line 512, in <module>\n    main()\n  File \"/tmp/ansible_Ly5cPo/ansible_module_lineinfile.py\", line 503, in main\n    ins_aft, ins_bef, create, backup, backrefs, firstmatch)\n  File \"/tmp/ansible_Ly5cPo/ansible_module_lineinfile.py\", line 337, in present\n    elif b_lines[index[1] - 1].rstrip(b('\\r\\n')) != b_line:\nIndexError: list index out of range\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 1
}
	to retry, use: --limit @/home/florian/sources/test/site.retry

PLAY RECAP *****************************************************************************************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=1   

➜  test ansible-playbook -vvvDi inventory site.yml 
ansible-playbook 2.5.0
  config file = None
  configured module search path = ['/home/florian/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/florian/.local/venvs/ansible/lib/python3.6/site-packages/ansible
  executable location = /home/florian/.local/bin/ansible-playbook
  python version = 3.6.4 (default, Jan  5 2018, 02:13:53) [GCC 7.2.0]
No config file found; using defaults
Parsed /home/florian/sources/test/inventory inventory source with ini plugin

PLAYBOOK: site.yml *********************************************************************************************************************************************************************************************
1 plays in site.yml

PLAY [test] ****************************************************************************************************************************************************************************************************
META: ran handlers

TASK [shell] ***************************************************************************************************************************************************************************************************
task path: /home/florian/sources/test/site.yml:8
Using module file /home/florian/.local/venvs/ansible/lib/python3.6/site-packages/ansible/modules/commands/command.py
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: florian
<localhost> EXEC /bin/sh -c 'echo ~ && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/florian/.ansible/tmp/ansible-tmp-1523274866.5070899-41694213921926 `" && echo ansible-tmp-1523274866.5070899-41694213921926="` echo /home/florian/.ansible/tmp/ansible-tmp-1523274866.5070899-41694213921926 `" ) && sleep 0'
<localhost> PUT /home/florian/.ansible/tmp/ansible-local-160322t4x8g9x/tmpsgmu62r6 TO /home/florian/.ansible/tmp/ansible-tmp-1523274866.5070899-41694213921926/command.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/florian/.ansible/tmp/ansible-tmp-1523274866.5070899-41694213921926/ /home/florian/.ansible/tmp/ansible-tmp-1523274866.5070899-41694213921926/command.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /home/florian/.ansible/tmp/ansible-tmp-1523274866.5070899-41694213921926/command.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/florian/.ansible/tmp/ansible-tmp-1523274866.5070899-41694213921926/ > /dev/null 2>&1 && sleep 0'
ok: [localhost] => {
    "changed": false,
    "cmd": "echo 123 > test_file",
    "invocation": {
        "module_args": {
            "_raw_params": "echo 123 > test_file",
            "_uses_shell": true,
            "chdir": null,
            "creates": "test_file",
            "executable": null,
            "removes": null,
            "stdin": null,
            "warn": true
        }
    },
    "rc": 0,
    "stdout": "skipped, since test_file exists",
    "stdout_lines": [
        "skipped, since test_file exists"
    ]
}

TASK [lineinfile] **********************************************************************************************************************************************************************************************
task path: /home/florian/sources/test/site.yml:10
Using module file /home/florian/.local/venvs/ansible/lib/python3.6/site-packages/ansible/modules/files/lineinfile.py
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: florian
<localhost> EXEC /bin/sh -c 'echo ~ && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/florian/.ansible/tmp/ansible-tmp-1523274866.8191464-173365190910096 `" && echo ansible-tmp-1523274866.8191464-173365190910096="` echo /home/florian/.ansible/tmp/ansible-tmp-1523274866.8191464-173365190910096 `" ) && sleep 0'
<localhost> PUT /home/florian/.ansible/tmp/ansible-local-160322t4x8g9x/tmp9s3cjrn2 TO /home/florian/.ansible/tmp/ansible-tmp-1523274866.8191464-173365190910096/lineinfile.py
<localhost> EXEC /bin/sh -c 'chmod u+x /home/florian/.ansible/tmp/ansible-tmp-1523274866.8191464-173365190910096/ /home/florian/.ansible/tmp/ansible-tmp-1523274866.8191464-173365190910096/lineinfile.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /home/florian/.ansible/tmp/ansible-tmp-1523274866.8191464-173365190910096/lineinfile.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /home/florian/.ansible/tmp/ansible-tmp-1523274866.8191464-173365190910096/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/tmp/ansible_aNUD7p/ansible_module_lineinfile.py", line 512, in <module>
    main()
  File "/tmp/ansible_aNUD7p/ansible_module_lineinfile.py", line 503, in main
    ins_aft, ins_bef, create, backup, backrefs, firstmatch)
  File "/tmp/ansible_aNUD7p/ansible_module_lineinfile.py", line 337, in present
    elif b_lines[index[1] - 1].rstrip(b('\r\n')) != b_line:
IndexError: list index out of range

fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/tmp/ansible_aNUD7p/ansible_module_lineinfile.py\", line 512, in <module>\n    main()\n  File \"/tmp/ansible_aNUD7p/ansible_module_lineinfile.py\", line 503, in main\n    ins_aft, ins_bef, create, backup, backrefs, firstmatch)\n  File \"/tmp/ansible_aNUD7p/ansible_module_lineinfile.py\", line 337, in present\n    elif b_lines[index[1] - 1].rstrip(b('\\r\\n')) != b_line:\nIndexError: list index out of range\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE",
    "rc": 1
}
	to retry, use: --limit @/home/florian/sources/test/site.retry

PLAY RECAP *****************************************************************************************************************************************************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=1

dkewley · 2018-05-13T01:15:18Z

@samdoran I think this is related to #36474

davidsev7 · 2018-05-13T01:46:23Z

The interesting behavior I see is that running this again on a file that has previously been updated to insert a line at BOF using insertbefore causes a second line to be inserted on the line just above the last line in the file.

BarbzYHOOL · 2018-05-26T19:40:29Z

Yes also it added the line at the end of a file instead of at the beginning... very weird

wouterh-dev · 2018-06-11T15:13:19Z

lineinfile now inserts at the end of file instead of start of file, breaking configuration files.

earchibald-lv · 2018-06-13T21:01:33Z

I can also reproduce this behavior:

lineinfile:
  path: /tmp/test
  create: yes
  insertbefore: BOF
  line: 'InsertThisLine'

nonexistent file with create: yes creates one-line file, as expected.
re-running against this one-line file with the correct line crashes with IndexError.
empty but existent file becomes one-line file, as expected.
5-line file without target line correctly inserts line at BOF.
6- line file with target line at BOF inserts the line, incorrectly, before the last line, making it the second to last line in the now 7-line file.
7-line file created above is treated as OK by lineinfile.

6-line file produced above

InsertThisLine
one
two
three
four
five

7-line file produced above and read by lineinfile as ok

InsertThisLine
one
two
three
four
InsertThisLine
five

earchibald-lv · 2018-06-20T20:19:38Z

Hey, this has been open for more than a month. It's breaking playbooks. What's the story?

samdoran · 2018-06-20T21:41:39Z

I am working on a fix. Apologies for the delay. I see why this is happening but it's at odds with the changes I made in #36474. I'm trying to balance the two. I've also added tests for this to prevent a regression.

samdoran · 2018-06-21T01:04:39Z

This may actually be more related to what was done in #33393. Anyway, still working through it.

samdoran · 2018-06-21T01:55:15Z

I believe I have it fixed. PR coming shortly.

samdoran · 2018-06-21T02:02:50Z

Please test with #41767 and see if that resolves your issue. Thank you for reporting this.

craigfrancis · 2018-07-02T09:38:54Z

Thanks @samdoran, looks like this change has been included in Ansible 2.6.0, and fixes my issue.

samdoran · 2018-07-02T14:11:50Z

@craigfrancis Thanks for the follow up confirmation. Glad this fixed your issue.

maxamillion removed the needs_triage Needs a first human triage before being processed. label Apr 3, 2018

philfry mentioned this issue Apr 4, 2018

ansible 2.5: lineinfile broken #38274

Closed

samdoran self-assigned this May 17, 2018

samdoran added this to To Do in 2.5.x blocker list via automation Jun 21, 2018

samdoran added the affects_2.6 This issue/PR affects Ansible v2.6 label Jun 21, 2018

samdoran mentioned this issue Jun 21, 2018

Skip if insertbefore is using BOF until later in the module #41767

Merged

samdoran closed this as completed in #41767 Jun 25, 2018

ansible locked and limited conversation to collaborators Jun 26, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

lineinfile not working with `insertbefore: BOF` in Ansible 2.5.0 #38219

lineinfile not working with `insertbefore: BOF` in Ansible 2.5.0 #38219

craigfrancis commented Apr 3, 2018

ansibot commented Apr 3, 2018

ansibot commented Apr 3, 2018

apollo13 commented Apr 9, 2018

dkewley commented May 13, 2018

davidsev7 commented May 13, 2018 •

edited

Loading

BarbzYHOOL commented May 26, 2018

wouterh-dev commented Jun 11, 2018 •

edited

Loading

earchibald-lv commented Jun 13, 2018 •

edited

Loading

earchibald-lv commented Jun 20, 2018

samdoran commented Jun 20, 2018 •

edited

Loading

samdoran commented Jun 21, 2018

samdoran commented Jun 21, 2018

samdoran commented Jun 21, 2018

craigfrancis commented Jul 2, 2018

samdoran commented Jul 2, 2018

lineinfile not working with insertbefore: BOF in Ansible 2.5.0 #38219

lineinfile not working with insertbefore: BOF in Ansible 2.5.0 #38219

Comments

craigfrancis commented Apr 3, 2018

ISSUE TYPE

COMPONENT NAME

ANSIBLE VERSION

CONFIGURATION

OS / ENVIRONMENT

SUMMARY

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

ansibot commented Apr 3, 2018

ansibot commented Apr 3, 2018

apollo13 commented Apr 9, 2018

dkewley commented May 13, 2018

davidsev7 commented May 13, 2018 • edited Loading

BarbzYHOOL commented May 26, 2018

wouterh-dev commented Jun 11, 2018 • edited Loading

earchibald-lv commented Jun 13, 2018 • edited Loading

earchibald-lv commented Jun 20, 2018

samdoran commented Jun 20, 2018 • edited Loading

samdoran commented Jun 21, 2018

samdoran commented Jun 21, 2018

samdoran commented Jun 21, 2018

craigfrancis commented Jul 2, 2018

samdoran commented Jul 2, 2018

lineinfile not working with `insertbefore: BOF` in Ansible 2.5.0 #38219

lineinfile not working with `insertbefore: BOF` in Ansible 2.5.0 #38219

davidsev7 commented May 13, 2018 •

edited

Loading

wouterh-dev commented Jun 11, 2018 •

edited

Loading

earchibald-lv commented Jun 13, 2018 •

edited

Loading

samdoran commented Jun 20, 2018 •

edited

Loading