NFS mounts are remounted at every execution

[nohaj]

Hello,

I'm trying to mount some NFS shares on my servers and it seems that I'm facing the same problem as the unresolved issues #15289 and #53688 .

[root@myserver ~]# salt-call --versions-report
Salt Version:
           Salt: 3000.2

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
         Jinja2: 2.7.2
        libgit2: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.6.2
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.5 (default, Apr  2 2020, 13:16:51)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.3.0
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.1.4

System Versions:
           dist: centos 7.8.2003 Core
         locale: UTF-8
        machine: x86_64
        release: 3.10.0-1127.8.2.el7.x86_64
         system: Linux
        version: CentOS Linux 7.8.2003 Core

/mnt/backup:
  mount.mounted:
    - device: myserver:/nfs/backup
    - fstype: nfs
    - options: "vers=4.1,hard,proto=tcp,sec=sys"
    - mkmnt: true
    - persist: true

At the first execution, the folder is created and the mount is done.

Then, for all the others executions I get :

     ID: /mnt/backup
    Function: mount.mounted
      Result: True
     Comment: Target was successfully mounted. Entry already exists in the fstab.
     Started: 14:18:59.361976
    Duration: 322.251 ms
     Changes:   
              ----------
              mount:
                  True
              umount:
                  Forced unmount because devices don't match. Wanted: myserver:/nfs/backup, current: myserver:/nfs/backup, /root/myserver:/nfs/backup

Same in python :

>>> os.path.realpath('myserver:/nfs/backup')
'/root/myserver:/nfs/backup'

Am I doing something wrong ?

Regards,

Johan

[dmurphy18]

@nohaj Can you upgrade to Salt 3000.3 as there were a number of issues fixed in runner's etc. in that release, and this might have affected nfs mounts too. And then rerun your tests.
Thanks

[nohaj]

Hello,

Just updated in 3000.3, same issue =(

[dmurphy18]

This might be related to #39292 causing issues since the fstab is not quite the same.

[nohaj]

OK so I added another NFS share today on some servers but in NFS 4.0 version (different NFS server) and I don't have this issue with this mount.

So I switched the others mounts in 4.0 version and the issue is gone.

- options: "vers=4.1,hard,proto=tcp,sec=sys"
+ options: "vers=4.0,hard,proto=tcp,sec=sys"

Does it help you ?

[piterpunk]

The mount.mounted state doesn't expect the parameter "options". It should be "opts" as stated by documentation. I did a few tests with Salt 3001 with the following state:

/srv/mirrors:
  mount.mounted:
    - device: marvin:/srv/www/htdocs/mirrors
    - fstype: nfs
    - opts: "vers=4.1,hard,proto=tcp"
    - mkmnt: true
    - persist: true

In the first execution it works as expected:

local:
----------
          ID: /srv/mirrors
    Function: mount.mounted
      Result: True
     Comment: Target was successfully mounted. Added new entry to the fstab.
     Started: 04:39:04.325111
    Duration: 310.876 ms
     Changes:   
              ----------                                                                                                            
              mount:
                  True
              persist:
                  new

Summary for local                                                                                                                   
------------                                                                                                                        
Succeeded: 1 (changed=1)

The filesystem is mounted and the fstab entry created. But if I apply the state a second time, the fstab is updated:

local:
----------
          ID: /srv/mirrors
    Function: mount.mounted
      Result: True
     Comment: Target was already mounted. Updated the entry in the fstab.
     Started: 04:40:14.435902
    Duration: 77.426 ms
     Changes:   
              ----------                                                                                                            
              persist:
                  update

Summary for local                                                                                                                   
------------                                                                                                                        
Succeeded: 1 (changed=1)

Now, the third time it works as expected, with no changes:

local:
----------
          ID: /srv/mirrors
    Function: mount.mounted
      Result: True
     Comment: Target was already mounted. Entry already exists in the fstab.
     Started: 04:40:49.145368
    Duration: 74.772 ms
     Changes:   

Summary for local
------------
Succeeded: 1

But if I umount the /srv/mirrors and apply the state again, the fstab will be updated again:

local:
----------
          ID: /srv/mirrors
    Function: mount.mounted
      Result: True
     Comment: Target was successfully mounted. Updated the entry in the fstab.
     Started: 04:42:31.015303
    Duration: 304.953 ms
     Changes:   
              ----------
              mount:
                  True
              persist:
                  update

Summary for local
------------
Succeeded: 1 (changed=1)

And the fstab updating cycle restarts.

This one happens because when the filesystem is umounted, the generated fstab entry uses the options in the same order that is in sls file. But when the filesystem is mounted, the generated entry uses the options alphabetically ordered. So, they doesn't matches and it keeps updating the fstab depending if the state is applied with the filesystem mounted or not.

While I haven't got the same message of the original post, this seems to be a related problem and keeps the fstab being updating even with the correct entry present.

I have the fix for this issue, but I don't know if I should include it on PR #57669 which already modifies states/mount.py or create a new one. What would be the better approach?

[nohaj]

The mount.mounted state doesn't expect the parameter "options". It should be "opts" as stated by documentation. I did a few tests with Salt 3001 with the following state:

You're right. The "options" parameter come from a formula, but inside the formula it's indeed "opts" =)

[piterpunk]

You're right. The "options" parameter come from a formula, but inside the formula it's indeed "opts" =)

Can you share this formula so I can test all the workflow?

[nohaj]

I use this one : https://github.com/saltstack-formulas/nfs-formula

[piterpunk]

@nohaj , probably you have something custom in there.

The linked nfs-formula also don't use "options", it expects "opts" on calling state or "mount_opts" on pillar.

Can you apply the state using salt-call with "-l debug" or "-l trace" and put the output here?

[nohaj]

Sorry about that @piterpunk, I am actually using "opts" and not "options". I don't know why I wrote that in my initial post.

And you are right, the third time nothing happen anymore, so it's not as bad as i thought.

[piterpunk]

@nohaj nice! So the bug is exactly the one I fixed. Thanks!