Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Contacts no longer being displayed #6744

Closed
darose opened this issue May 9, 2019 · 21 comments

Comments

@darose
Copy link

commented May 9, 2019

I'm not sure when or how it broke, but the Contacts in my roundcube installation are no longer working:

  • When I compose a new message, and start to type a contact's name in order to invoke auto-complete of their email address, it no longer auto-completes. Screenshot_2019-05-09_18-49-42

  • When I look in my Contacts, my contacts are listed. But the entries contain no data.
    Screenshot_2019-05-09_18-50-33

Both of these pieces of functionality were definitely working until fairly recently.

The only clue I'm seeing as to what might be wrong is messages like the following in the roundcubemail/errors log file:

[09-May-2019 22:34:32 UTC] PHP Warning: preg_replace(): Compilation failed: escape sequence is invalid in character class at offset 8 in /usr/share/webapps/roundcubemail/program/lib/Roundcube/rcube_vcard.php on line 547

I'm on Arch Linux, running the following versions of roundcube and plugins:
roundcubemail 1.3.9-1
roundcubemail-plugin-chbox 0.2.4-2
roundcubemail-plugin-contextmenu 2.3-1
roundcubemail-plugin-jquery-mobile 1.1-1
roundcubemail-plugin-keyboard-shortcuts-ng 0.9.4-1
roundcubemail-plugin-markasjunk2 1.11.1-1
roundcubemail-plugin-mobile 0.4.8-1

Any idea what the issue might be, and how to fix?

@alecpl

This comment has been minimized.

Copy link
Member

commented May 10, 2019

Try with disabled all plugins. What PHP version? Could you show lines 530-550 in rcube_vcard?

@darose

This comment has been minimized.

Copy link
Author

commented May 10, 2019

php 7.3.5

Disabling all plugins didn't remove problem.

Relevant code is as follows:

/**
 * Normalize vcard data for better parsing
 *
 * @param string vCard block
 *
 * @return string Cleaned vcard block
 */
public static function cleanup($vcard)
{
    // convert Apple X-ABRELATEDNAMES into X-* fields for better compatibility
    $vcard = preg_replace_callback(
        '/item(\d+)\.(X-ABRELATEDNAMES)([^:]*?):(.*?)item\1.X-ABLabel:(?:_\$!<)?([\w() -]*)(?:>!\$_)?./s',
        array('self', 'x_abrelatednames_callback'),
        $vcard);

    // Cleanup
    $vcard = preg_replace(array(
            // convert special types (like Skype) to normal type='skype' classes with this simple regex ;)
            '/item(\d+)\.(TEL|EMAIL|URL)([^:]*?):(.*?)item\1.X-ABLabel:(?:_\$!<)?([\w() -]*)(?:>!\$_)?./si',
            '/^item\d*\.X-AB.*$/mi',  // remove cruft like item1.X-AB*
            '/^item\d*\./mi',         // remove item1.ADR instead of ADR
            '/\n+/',                 // remove empty lines
            '/^(N:[^;\R]*)$/m',      // if N doesn't have any semicolons, add some
        ),
        array(
            '\2;type=\5\3:\4',
            '',
            '',
            "\n",
            '\1;;;;',
        ), $vcard);

    // convert X-WAB-GENDER to X-GENDER
    if (preg_match('/X-WAB-GENDER:(\d)/', $vcard, $matches)) {
        $value = $matches[1] == '2' ? 'male' : 'female';
        $vcard = preg_replace('/X-WAB-GENDER:\d/', 'X-GENDER:' . $value, $vcard);
    }

    return $vcard;
}
@alecpl

This comment has been minimized.

Copy link
Member

commented May 10, 2019

I cannot reproduce with PHP 7.3.5.

I thought it's #6398, but it actually was a different warning. Could you check if it works when you replace /^(N:[^;\R]*)$/m with /^(N:[^;\r\n]*)$/m?

@darose

This comment has been minimized.

Copy link
Author

commented May 10, 2019

Looks like that fixed it! Tnx!

@darose darose closed this May 10, 2019

@alecpl alecpl reopened this May 10, 2019

@alecpl

This comment has been minimized.

Copy link
Member

commented May 10, 2019

Need to be fix upstream ;) Reopened.

@alecpl alecpl added this to the 1.4-rc2 milestone May 10, 2019

@alecpl

This comment has been minimized.

Copy link
Member

commented May 10, 2019

I'm trying to reproduce, but with no luck. It may be a PHP bug. What platform are you using? What php package version precisely? Could you provide output of php -i | grep -i pcre?

alecpl added a commit that referenced this issue May 10, 2019

Fix bug in parsing vCard data using PHP 7.3 due to an invalid regexp (#…
…6744)

Looks like \R is not allowed in character class, but \r\n is fine.
On PHP 7.3.5 it throws warnings and empty result from preg_replace(),
though I couldn't reproduce.

alecpl added a commit that referenced this issue May 10, 2019

Fix bug in parsing vCard data using PHP 7.3 due to an invalid regexp (#…
…6744)

Looks like \R is not allowed in character class, but \r\n is fine.
On PHP 7.3.5 it throws warnings and empty result from preg_replace(),
though I couldn't reproduce.
@alecpl

This comment has been minimized.

Copy link
Member

commented May 10, 2019

Looks like indeed \R is not allowed in character class. Fixed.

@alecpl alecpl closed this May 10, 2019

@darose

This comment has been minimized.

Copy link
Author

commented May 10, 2019

I'm trying to reproduce, but with no luck. It may be a PHP bug. What platform are you using? What php package version precisely? Could you provide output of php -i | grep -i pcre?

As I wrote above:

  • Arch Linux

  • php version 7.3.5

  • roundcubemail version 1.3.9

$ php -i | grep -i pcre
Configure Command =>  './configure'  '--srcdir=../php-7.3.5' '--config-cache' '--prefix=/usr' '--sbindir=/usr/bin' '--sysconfdir=/etc/php' '--localstatedir=/var' '--with-layout=GNU' '--with-config-file-path=/etc/php' '--with-config-file-scan-dir=/etc/php/conf.d' '--disable-rpath' '--mandir=/usr/share/man' '--without-pear' '--enable-cgi' '--enable-fpm' '--with-fpm-systemd' '--with-fpm-acl' '--with-fpm-user=http' '--with-fpm-group=http' '--enable-embed=shared' '--enable-bcmath=shared' '--enable-calendar=shared' '--enable-dba=shared' '--enable-exif=shared' '--enable-ftp=shared' '--enable-intl=shared' '--enable-mbstring' '--enable-shmop=shared' '--enable-soap=shared' '--enable-sockets=shared' '--enable-sysvmsg=shared' '--enable-sysvsem=shared' '--enable-sysvshm=shared' '--enable-zip=shared' '--with-bz2=shared' '--with-curl=shared' '--with-db4=/usr' '--with-enchant=shared,/usr' '--with-freetype-dir=/usr' '--with-gd=shared,/usr' '--with-gdbm' '--with-gettext=shared' '--with-gmp=shared' '--with-iconv=shared' '--with-imap-ssl' '--with-imap=shared' '--with-kerberos=/usr' '--with-ldap=shared' '--with-ldap-sasl' '--with-libzip' '--with-mhash' '--with-mysql-sock=/run/mysqld/mysqld.sock' '--with-mysqli=shared,mysqlnd' '--with-openssl' '--with-password-argon2' '--with-pcre-regex=/usr' '--with-pdo-dblib=shared,/usr' '--with-pdo-mysql=shared,mysqlnd' '--with-pdo-odbc=shared,unixODBC,/usr' '--with-pdo-pgsql=shared' '--with-pdo-sqlite=shared,/usr' '--with-pgsql=shared' '--with-pspell=shared' '--with-readline' '--with-snmp=shared' '--with-sodium=shared' '--with-sqlite3=shared,/usr' '--with-tidy=shared' '--with-unixODBC=shared,/usr' '--with-xmlrpc=shared' '--with-xsl=shared' '--with-zlib' '--enable-pcntl'
pcre
PCRE (Perl Compatible Regular Expressions) Support => enabled
PCRE Library Version => 10.33 2019-04-16
PCRE Unicode Version => 11.0.0
PCRE JIT Support => enabled
PCRE JIT Target => x86 64bit (little endian + unaligned)
pcre.backtrack_limit => 1000000 => 1000000
pcre.jit => 1 => 1
pcre.recursion_limit => 100000 => 100000
@alecpl

This comment has been minimized.

Copy link
Member

commented May 11, 2019

For posterity, I tested with PHP 7.3.5 on Ubuntu.

PCRE (Perl Compatible Regular Expressions) Support => enabled
PCRE Library Version => 10.31 2018-02-12
PCRE Unicode Version => 10.0.0
PCRE JIT Support => enabled
PCRE JIT Target => x86 64bit (little endian + unaligned)
pcre.backtrack_limit => 1000000 => 1000000
pcre.jit => 1 => 1
pcre.recursion_limit => 100000 => 100000
pcre => enabled
@pc-rabo

This comment has been minimized.

Copy link

commented Jun 5, 2019

I thought it's #6398, but it actually was a different warning. Could you check if it works when you replace /^(N:[^;\R]*)$/m with /^(N:[^;\r\n]*)$/m?

I have the same issue and above regex change fixed it too. Archlinux php 7.3.5, roundcube 1.3.9

@ianbarton

This comment has been minimized.

Copy link

commented Jun 7, 2019

Just tried altering the regexp on Arch with PHP 7.3.6 roundcube 1.3.9 and it's still broken for me. Can anyone else confirm this?

@darose

This comment has been minimized.

Copy link
Author

commented Jun 7, 2019

It's you. Works for me with those exact package numbers.

@ianbarton

This comment has been minimized.

Copy link

commented Jun 7, 2019

OK, it's almost certainly me being stupid:) I edited the function, as shown below, then for good measure rebooted the server. What am I doing wrong?

Roundcube is in:
/srv/http/webmail/program/lib/Roundcube/rcube_vcard.php
public function load($vcard, $charset = RCUBE_CHARSET, $detect = false)

`$vcard = preg_replace(array(
// convert special types (like Skype) to normal type='skype' classes with this simple regex ;)
'/item(\d+).(TEL|EMAIL|URL)([^:]?):(.?)item\1.X-ABLabel:(?:$!<)?([\w() -]*)(?:>!$)?./si',
'/^item\d*.X-AB.$/mi', // remove cruft like item1.X-AB
'/^item\d*./mi', // remove item1.ADR instead of ADR
'/\n+/', // remove empty lines
'/^(N:[^;\r\n]*)$/m', // if N doesn't have any semicolons, add some

        ),   `
@darose

This comment has been minimized.

Copy link
Author

commented Jun 7, 2019

I didn't have to touch the load function. I had to touch the cleanup function. This is how it reads:

    public static function cleanup($vcard)
    {
        // convert Apple X-ABRELATEDNAMES into X-* fields for better compatibility
        $vcard = preg_replace_callback(
            '/item(\d+)\.(X-ABRELATEDNAMES)([^:]*?):(.*?)item\1.X-ABLabel:(?:_\$!<)?([\w() -]*)(?:>!\$_)?./s',
            array('self', 'x_abrelatednames_callback'),
            $vcard);

        // Cleanup
        $vcard = preg_replace(array(
                // convert special types (like Skype) to normal type='skype' classes with this simple regex ;)
                '/item(\d+)\.(TEL|EMAIL|URL)([^:]*?):(.*?)item\1.X-ABLabel:(?:_\$!<)?([\w() -]*)(?:>!\$_)?./si',
                '/^item\d*\.X-AB.*$/mi',  // remove cruft like item1.X-AB*
                '/^item\d*\./mi',         // remove item1.ADR instead of ADR
                '/\n+/',                 // remove empty lines
                '/^(N:[^;\r\n]*)$/m',      // if N doesn't have any semicolons, add some
            ),
            array(
                '\2;type=\5\3:\4',
                '',
                '',
                "\n",
                '\1;;;;',
            ), $vcard);

        // convert X-WAB-GENDER to X-GENDER
        if (preg_match('/X-WAB-GENDER:(\d)/', $vcard, $matches)) {
            $value = $matches[1] == '2' ? 'male' : 'female';
            $vcard = preg_replace('/X-WAB-GENDER:\d/', 'X-GENDER:' . $value, $vcard);
        }

        return $vcard;
    }
@ianbarton

This comment has been minimized.

Copy link

commented Jun 7, 2019

Thanks. Still doesn't work for me. I'll try archiving my current installation and re-install from scratch.

@amishmm

This comment has been minimized.

Copy link

commented Jun 8, 2019

@ianbarton the fix worked for me too.. so it must be something wrong somewhere in your settings.

Arch linux - PHP 7.3.6 - Roundcube 1.3.9

@ianbarton

This comment has been minimized.

Copy link

commented Jun 8, 2019

Thanks all. I have just installed the fix on my backup server and works. So there must be something on my main server that is causing the problem.

@amishmm

This comment has been minimized.

Copy link

commented Jun 8, 2019

Make sure you have not disabled autocomplete in config files. Normally when something doesnt work we try to alter settings and when fix is found - we completely forget to revert the settings to normal. (human tendency)

@ianbarton

This comment has been minimized.

Copy link

commented Jun 19, 2019

Finally solved this, but not sure what caused the problem. Tried exporting my contacts to as vcard and they were all blank. Ran mysqlcheck which didn't report any problems. Browsed the contacts table with phpmyadmin and I could see all my contact data. If I tried adding back an existing contact using Roundcube, I got a warning about it already existing as I expected.

Finally deleted all my contacts using Roundcube and then added them back. All now working correctly.. Probably an attack by the gremlins.

@linuxuser424

This comment has been minimized.

Copy link

commented Jul 24, 2019

Hi @ all,

I had the same problem on Debian with php 7.3 from sury.org (#6864).
The fix works here, too.

php -i | grep -i pcre pcre PCRE (Perl Compatible Regular Expressions) Support => enabled PCRE Library Version => 10.33 2019-04-16 PCRE Unicode Version => 11.0.0 PCRE JIT Support => enabled PCRE JIT Target => x86 64bit (little endian + unaligned) pcre.backtrack_limit => 1000000 => 1000000 pcre.jit => 1 => 1 pcre.recursion_limit => 100000 => 100000

Thank you.

@GeorgSchlisio

This comment has been minimized.

Copy link

commented Jul 25, 2019

Can confirm issue on Archlinux with RC 1.3.9 and PHP 7.3.7. Tweaking program/lib/Roundcube/rcube_vcard.php:539 to show \r\n instead of \R fixed the problem for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants
You can’t perform that action at this time.