New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve filesystem detection for NFS #8734

Closed
nuxwin opened this Issue Sep 8, 2015 · 11 comments

Comments

Projects
None yet
2 participants
@nuxwin
Contributor

nuxwin commented Sep 8, 2015

Hello ;

I installed Piwik on a test server which is managed by i-MSCP (A Linux control panel for shared hosting environments). When I check the system using Piwik, I get the following warning:

Your server is using an NFS filesystem.
This means Piwik will be extremely slow when using file based sessions. 

and therefore, the session handler is automatically set to dbtable.

While this behavior is expected for NFS, this should not be the case for bind mounts. Indeed, Piwik detects that I use NFS but this is totally wrong. i-MSCP automatically re-mounts the Web folder directories (bind mount) as shared subtree. More explaination can be found here: https://github.com/i-MSCP/imscp/blob/1.3.x/docs/1.3.x_errata.md#shared-subtree-homedir

In my case, the Web folder is re-mounted as follow:

/dev/sda1 on /var/www/virtual/test.tld type ext4 (rw,relatime,errors=remount-ro,data=ordered)

As you can see here, that is not an NFS filesystem. I don't know how the check is made by Piwik but it seem that the filesystem type is not really checked.

That bind mount is managed by i-MSCP:

# fstab-like configuration file - auto-generated by i-MSCP
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
/var/www/virtual/test.tld /var/www/virtual/test.tld none shared,bind
/var/log/apache2/test.tld /var/www/virtual/test.tld/logs/test.tld none bind
/var/log/apache2/piwik.test.tld /var/www/virtual/test.tld/logs/piwik.test.tld none bind

See https://github.com/i-MSCP/imscp/blob/1.3.x/docs/1.3.x_errata.md#imscp_mountall-service

Note: I'm the i-MSCP project director.

@nuxwin

This comment has been minimized.

Show comment
Hide comment
@nuxwin

nuxwin Sep 8, 2015

Contributor

Re ;

I could provide a patch if you want (PR). It seem that the check is done in the core/Filesystem.php file. I must investigate but right now, when running the df command manually on my system, I get

root@jessie:/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions# LANG=C df -T -t nfs  /var/www/virtual/test.tld/piwik/htdocs/tmp/sessions 2>&1
df: no file systems processed
root@jessie:/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions# echo $?
1

which is expected. Thus, I presume that the check is wrong somewhere in the checkIfFileSystemIsNFS() method of the Piwik\Filesystem class.

My environment:

root@jessie:/var/www/virtual/test.tld/piwik/htdocs# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 8.2 (jessie)
Release:    8.2
Codename:   jessie

root@jessie:/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions# apache2ctl -v
Server version: Apache/2.4.10 (Debian)
Server built:   Aug 28 2015 16:28:08

root@jessie:/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions# php5-fpm -v
PHP 5.6.12-0+deb8u1 (fpm-fcgi) (built: Aug 16 2015 12:16:05)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2015 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2015, by Zend Technologies

For the record, when I run the following script (CLI):

<?php

function checkIfFileSystemIsNFS()
{
        $sessionsPath = '/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions';

        // this command will display details for the filesystem that holds the $sessionsPath
        // path, but only if its type is NFS. if not NFS, df will return one or less lines
        // and the return code 1. if NFS, it will return 0 and at least 2 lines of text.
        $command = "df -T -t nfs \"$sessionsPath\" 2>&1";

        if (function_exists('exec')) {
            // use exec

            $output = $returnCode = null;
            @exec($command, $output, $returnCode);

            // check if filesystem is NFS
            if ($returnCode == 0
                && count($output) > 1
            ) {
                return true;
            }
        } elseif (function_exists('shell_exec')) {
            // use shell_exec

            $output = @shell_exec($command);
            if ($output) {
                $output = explode("\n", $output);
                if (count($output) > 1) {
                    // check if filesystem is NFS

                    return true;
                }
            }
        }

        return false; // not NFS, or we can't run a program to find out
}


if(checkIfFileSystemIsNFS()) {
        print "NFS filesystem has been detected\n";
} else {
        print "Not an NFS filesystem\n";
}

I get the expected result:

root@jessie:/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions# php test.php 
Not an NFS filesystem

Thus, I must investigate more...

Contributor

nuxwin commented Sep 8, 2015

Re ;

I could provide a patch if you want (PR). It seem that the check is done in the core/Filesystem.php file. I must investigate but right now, when running the df command manually on my system, I get

root@jessie:/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions# LANG=C df -T -t nfs  /var/www/virtual/test.tld/piwik/htdocs/tmp/sessions 2>&1
df: no file systems processed
root@jessie:/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions# echo $?
1

which is expected. Thus, I presume that the check is wrong somewhere in the checkIfFileSystemIsNFS() method of the Piwik\Filesystem class.

My environment:

root@jessie:/var/www/virtual/test.tld/piwik/htdocs# lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description:    Debian GNU/Linux 8.2 (jessie)
Release:    8.2
Codename:   jessie

root@jessie:/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions# apache2ctl -v
Server version: Apache/2.4.10 (Debian)
Server built:   Aug 28 2015 16:28:08

root@jessie:/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions# php5-fpm -v
PHP 5.6.12-0+deb8u1 (fpm-fcgi) (built: Aug 16 2015 12:16:05)
Copyright (c) 1997-2015 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2015 Zend Technologies
    with Zend OPcache v7.0.6-dev, Copyright (c) 1999-2015, by Zend Technologies

For the record, when I run the following script (CLI):

<?php

function checkIfFileSystemIsNFS()
{
        $sessionsPath = '/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions';

        // this command will display details for the filesystem that holds the $sessionsPath
        // path, but only if its type is NFS. if not NFS, df will return one or less lines
        // and the return code 1. if NFS, it will return 0 and at least 2 lines of text.
        $command = "df -T -t nfs \"$sessionsPath\" 2>&1";

        if (function_exists('exec')) {
            // use exec

            $output = $returnCode = null;
            @exec($command, $output, $returnCode);

            // check if filesystem is NFS
            if ($returnCode == 0
                && count($output) > 1
            ) {
                return true;
            }
        } elseif (function_exists('shell_exec')) {
            // use shell_exec

            $output = @shell_exec($command);
            if ($output) {
                $output = explode("\n", $output);
                if (count($output) > 1) {
                    // check if filesystem is NFS

                    return true;
                }
            }
        }

        return false; // not NFS, or we can't run a program to find out
}


if(checkIfFileSystemIsNFS()) {
        print "NFS filesystem has been detected\n";
} else {
        print "Not an NFS filesystem\n";
}

I get the expected result:

root@jessie:/var/www/virtual/test.tld/piwik/htdocs/tmp/sessions# php test.php 
Not an NFS filesystem

Thus, I must investigate more...

@nuxwin

This comment has been minimized.

Show comment
Hide comment
@nuxwin

nuxwin Sep 8, 2015

Contributor

I find the cause:

        } elseif (function_exists('shell_exec')) {
            // use shell_exec

            $output = @shell_exec($command);
            if ($output) {
                $output = explode("\n", $output);
                if (count($output) > 1) {
                    // check if filesystem is NFS

                    return true;
                }
            }
        }

is wrong... You cannot rely on the output of that command AS THIS to detect filesystem.

In my case, output array (after explode) hold:

Array
(
    [0] => df: no file systems processed
    [1] => 
)

Then, the result is erroned.

You can fix that issue by applying trim() on the output before the explode:

$output = trim($output);

@mattab Do you want a PR?

Contributor

nuxwin commented Sep 8, 2015

I find the cause:

        } elseif (function_exists('shell_exec')) {
            // use shell_exec

            $output = @shell_exec($command);
            if ($output) {
                $output = explode("\n", $output);
                if (count($output) > 1) {
                    // check if filesystem is NFS

                    return true;
                }
            }
        }

is wrong... You cannot rely on the output of that command AS THIS to detect filesystem.

In my case, output array (after explode) hold:

Array
(
    [0] => df: no file systems processed
    [1] => 
)

Then, the result is erroned.

You can fix that issue by applying trim() on the output before the explode:

$output = trim($output);

@mattab Do you want a PR?

@tsteur

This comment has been minimized.

Show comment
Hide comment
@tsteur

tsteur Sep 8, 2015

Member

PR would be awesome 👍

Member

tsteur commented Sep 8, 2015

PR would be awesome 👍

@nuxwin

This comment has been minimized.

Show comment
Hide comment
@nuxwin

nuxwin Sep 8, 2015

Contributor

@tsteur

The PR must be made against master or do you have a specific branch for development purpose?

Contributor

nuxwin commented Sep 8, 2015

@tsteur

The PR must be made against master or do you have a specific branch for development purpose?

@tsteur

This comment has been minimized.

Show comment
Hide comment
@tsteur

tsteur Sep 8, 2015

Member

PR should be made against master 👍

Member

tsteur commented Sep 8, 2015

PR should be made against master 👍

@nuxwin

This comment has been minimized.

Show comment
Hide comment
@nuxwin

nuxwin Sep 8, 2015

Contributor

@tsteur

Ok, then. I'll do that PR in few minutes ;)

Contributor

nuxwin commented Sep 8, 2015

@tsteur

Ok, then. I'll do that PR in few minutes ;)

nuxwin added a commit to nuxwin/piwik that referenced this issue Sep 8, 2015

@nuxwin

This comment has been minimized.

Show comment
Hide comment
@nuxwin

nuxwin Sep 8, 2015

Contributor

@tsteur

Done. See #8737

BTW:

@mattab

The following check which was previously added in master:

$commandFailed = (false !== strpos($output, "no file systems processed"));

was not sufficient. Think about LANG environment variable which can be different.

Contributor

nuxwin commented Sep 8, 2015

@tsteur

Done. See #8737

BTW:

@mattab

The following check which was previously added in master:

$commandFailed = (false !== strpos($output, "no file systems processed"));

was not sufficient. Think about LANG environment variable which can be different.

tsteur added a commit that referenced this issue Sep 8, 2015

Merge pull request #8737 from nuxwin/master
#8734 Bad filesystem detection (NFS)
@tsteur

This comment has been minimized.

Show comment
Hide comment
@tsteur

tsteur Sep 8, 2015

Member

I presume we can close this one then?

Member

tsteur commented Sep 8, 2015

I presume we can close this one then?

@nuxwin

This comment has been minimized.

Show comment
Hide comment
@nuxwin

nuxwin Sep 8, 2015

Contributor

@tsteur
Sure since you merged the referenced PR ;)

Contributor

nuxwin commented Sep 8, 2015

@tsteur
Sure since you merged the referenced PR ;)

@nuxwin

This comment has been minimized.

Show comment
Hide comment
@nuxwin

nuxwin Sep 8, 2015

Contributor

@tsteur

I close it right now but I cannot add the BUG label on it ;)

Contributor

nuxwin commented Sep 8, 2015

@tsteur

I close it right now but I cannot add the BUG label on it ;)

@nuxwin nuxwin closed this Sep 8, 2015

@tsteur tsteur added the Bug label Sep 8, 2015

@tsteur tsteur added this to the 2.15.0 milestone Sep 8, 2015

@tsteur tsteur self-assigned this Sep 8, 2015

@tsteur

This comment has been minimized.

Show comment
Hide comment
@tsteur

tsteur Sep 8, 2015

Member

Done, thx

Member

tsteur commented Sep 8, 2015

Done, thx

@mattab mattab changed the title from Bad filesystem detection (NFS) to Improve filesystem detection for NFS Oct 13, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment