Contacts no longer being displayed #6744

darose · 2019-05-09T22:53:22Z

I'm not sure when or how it broke, but the Contacts in my roundcube installation are no longer working:

When I compose a new message, and start to type a contact's name in order to invoke auto-complete of their email address, it no longer auto-completes.
When I look in my Contacts, my contacts are listed. But the entries contain no data.

Both of these pieces of functionality were definitely working until fairly recently.

The only clue I'm seeing as to what might be wrong is messages like the following in the roundcubemail/errors log file:

[09-May-2019 22:34:32 UTC] PHP Warning: preg_replace(): Compilation failed: escape sequence is invalid in character class at offset 8 in /usr/share/webapps/roundcubemail/program/lib/Roundcube/rcube_vcard.php on line 547

I'm on Arch Linux, running the following versions of roundcube and plugins:
roundcubemail 1.3.9-1
roundcubemail-plugin-chbox 0.2.4-2
roundcubemail-plugin-contextmenu 2.3-1
roundcubemail-plugin-jquery-mobile 1.1-1
roundcubemail-plugin-keyboard-shortcuts-ng 0.9.4-1
roundcubemail-plugin-markasjunk2 1.11.1-1
roundcubemail-plugin-mobile 0.4.8-1

Any idea what the issue might be, and how to fix?

The text was updated successfully, but these errors were encountered:

alecpl · 2019-05-10T04:45:48Z

Try with disabled all plugins. What PHP version? Could you show lines 530-550 in rcube_vcard?

darose · 2019-05-10T13:55:56Z

php 7.3.5

Disabling all plugins didn't remove problem.

Relevant code is as follows:

/**
 * Normalize vcard data for better parsing
 *
 * @param string vCard block
 *
 * @return string Cleaned vcard block
 */
public static function cleanup($vcard)
{
    // convert Apple X-ABRELATEDNAMES into X-* fields for better compatibility
    $vcard = preg_replace_callback(
        '/item(\d+)\.(X-ABRELATEDNAMES)([^:]*?):(.*?)item\1.X-ABLabel:(?:_\$!<)?([\w() -]*)(?:>!\$_)?./s',
        array('self', 'x_abrelatednames_callback'),
        $vcard);

    // Cleanup
    $vcard = preg_replace(array(
            // convert special types (like Skype) to normal type='skype' classes with this simple regex ;)
            '/item(\d+)\.(TEL|EMAIL|URL)([^:]*?):(.*?)item\1.X-ABLabel:(?:_\$!<)?([\w() -]*)(?:>!\$_)?./si',
            '/^item\d*\.X-AB.*$/mi',  // remove cruft like item1.X-AB*
            '/^item\d*\./mi',         // remove item1.ADR instead of ADR
            '/\n+/',                 // remove empty lines
            '/^(N:[^;\R]*)$/m',      // if N doesn't have any semicolons, add some
        ),
        array(
            '\2;type=\5\3:\4',
            '',
            '',
            "\n",
            '\1;;;;',
        ), $vcard);

    // convert X-WAB-GENDER to X-GENDER
    if (preg_match('/X-WAB-GENDER:(\d)/', $vcard, $matches)) {
        $value = $matches[1] == '2' ? 'male' : 'female';
        $vcard = preg_replace('/X-WAB-GENDER:\d/', 'X-GENDER:' . $value, $vcard);
    }

    return $vcard;
}

alecpl · 2019-05-10T14:35:11Z

I cannot reproduce with PHP 7.3.5.

I thought it's #6398, but it actually was a different warning. Could you check if it works when you replace /^(N:[^;\R]*)$/m with /^(N:[^;\r\n]*)$/m?

darose · 2019-05-10T16:48:16Z

Looks like that fixed it! Tnx!

alecpl · 2019-05-10T16:51:57Z

Need to be fix upstream ;) Reopened.

alecpl · 2019-05-10T18:51:19Z

I'm trying to reproduce, but with no luck. It may be a PHP bug. What platform are you using? What php package version precisely? Could you provide output of php -i | grep -i pcre?

…6744) Looks like \R is not allowed in character class, but \r\n is fine. On PHP 7.3.5 it throws warnings and empty result from preg_replace(), though I couldn't reproduce.

alecpl · 2019-05-10T19:11:48Z

Looks like indeed \R is not allowed in character class. Fixed.

darose · 2019-05-10T21:28:06Z

I'm trying to reproduce, but with no luck. It may be a PHP bug. What platform are you using? What php package version precisely? Could you provide output of php -i | grep -i pcre?

As I wrote above:

Arch Linux
php version 7.3.5
roundcubemail version 1.3.9

$ php -i | grep -i pcre
Configure Command =>  './configure'  '--srcdir=../php-7.3.5' '--config-cache' '--prefix=/usr' '--sbindir=/usr/bin' '--sysconfdir=/etc/php' '--localstatedir=/var' '--with-layout=GNU' '--with-config-file-path=/etc/php' '--with-config-file-scan-dir=/etc/php/conf.d' '--disable-rpath' '--mandir=/usr/share/man' '--without-pear' '--enable-cgi' '--enable-fpm' '--with-fpm-systemd' '--with-fpm-acl' '--with-fpm-user=http' '--with-fpm-group=http' '--enable-embed=shared' '--enable-bcmath=shared' '--enable-calendar=shared' '--enable-dba=shared' '--enable-exif=shared' '--enable-ftp=shared' '--enable-intl=shared' '--enable-mbstring' '--enable-shmop=shared' '--enable-soap=shared' '--enable-sockets=shared' '--enable-sysvmsg=shared' '--enable-sysvsem=shared' '--enable-sysvshm=shared' '--enable-zip=shared' '--with-bz2=shared' '--with-curl=shared' '--with-db4=/usr' '--with-enchant=shared,/usr' '--with-freetype-dir=/usr' '--with-gd=shared,/usr' '--with-gdbm' '--with-gettext=shared' '--with-gmp=shared' '--with-iconv=shared' '--with-imap-ssl' '--with-imap=shared' '--with-kerberos=/usr' '--with-ldap=shared' '--with-ldap-sasl' '--with-libzip' '--with-mhash' '--with-mysql-sock=/run/mysqld/mysqld.sock' '--with-mysqli=shared,mysqlnd' '--with-openssl' '--with-password-argon2' '--with-pcre-regex=/usr' '--with-pdo-dblib=shared,/usr' '--with-pdo-mysql=shared,mysqlnd' '--with-pdo-odbc=shared,unixODBC,/usr' '--with-pdo-pgsql=shared' '--with-pdo-sqlite=shared,/usr' '--with-pgsql=shared' '--with-pspell=shared' '--with-readline' '--with-snmp=shared' '--with-sodium=shared' '--with-sqlite3=shared,/usr' '--with-tidy=shared' '--with-unixODBC=shared,/usr' '--with-xmlrpc=shared' '--with-xsl=shared' '--with-zlib' '--enable-pcntl'
pcre
PCRE (Perl Compatible Regular Expressions) Support => enabled
PCRE Library Version => 10.33 2019-04-16
PCRE Unicode Version => 11.0.0
PCRE JIT Support => enabled
PCRE JIT Target => x86 64bit (little endian + unaligned)
pcre.backtrack_limit => 1000000 => 1000000
pcre.jit => 1 => 1
pcre.recursion_limit => 100000 => 100000

alecpl · 2019-05-11T05:09:19Z

For posterity, I tested with PHP 7.3.5 on Ubuntu.

PCRE (Perl Compatible Regular Expressions) Support => enabled
PCRE Library Version => 10.31 2018-02-12
PCRE Unicode Version => 10.0.0
PCRE JIT Support => enabled
PCRE JIT Target => x86 64bit (little endian + unaligned)
pcre.backtrack_limit => 1000000 => 1000000
pcre.jit => 1 => 1
pcre.recursion_limit => 100000 => 100000
pcre => enabled

pc-rabo · 2019-06-05T13:57:01Z

I thought it's #6398, but it actually was a different warning. Could you check if it works when you replace /^(N:[^;\R]*)$/m with /^(N:[^;\r\n]*)$/m?

I have the same issue and above regex change fixed it too. Archlinux php 7.3.5, roundcube 1.3.9

ianbarton · 2019-06-07T15:50:44Z

Just tried altering the regexp on Arch with PHP 7.3.6 roundcube 1.3.9 and it's still broken for me. Can anyone else confirm this?

darose · 2019-06-07T16:04:43Z

It's you. Works for me with those exact package numbers.

ianbarton · 2019-06-07T16:21:55Z

OK, it's almost certainly me being stupid:) I edited the function, as shown below, then for good measure rebooted the server. What am I doing wrong?

Roundcube is in:
/srv/http/webmail/program/lib/Roundcube/rcube_vcard.php
public function load($vcard, $charset = RCUBE_CHARSET, $detect = false)

`$vcard = preg_replace(array(
// convert special types (like Skype) to normal type='skype' classes with this simple regex ;)
'/item(\d+).(TEL|EMAIL|URL)([^:]?):(.?)item\1.X-ABLabel:(?:$!<)?([\w() -]*)(?:>!$)?./si',
'/^item\d*.X-AB.$/mi', // remove cruft like item1.X-AB
'/^item\d*./mi', // remove item1.ADR instead of ADR
'/\n+/', // remove empty lines
'/^(N:[^;\r\n]*)$/m', // if N doesn't have any semicolons, add some

        ),   `

darose · 2019-06-07T16:24:39Z

I didn't have to touch the load function. I had to touch the cleanup function. This is how it reads:

    public static function cleanup($vcard)
    {
        // convert Apple X-ABRELATEDNAMES into X-* fields for better compatibility
        $vcard = preg_replace_callback(
            '/item(\d+)\.(X-ABRELATEDNAMES)([^:]*?):(.*?)item\1.X-ABLabel:(?:_\$!<)?([\w() -]*)(?:>!\$_)?./s',
            array('self', 'x_abrelatednames_callback'),
            $vcard);

        // Cleanup
        $vcard = preg_replace(array(
                // convert special types (like Skype) to normal type='skype' classes with this simple regex ;)
                '/item(\d+)\.(TEL|EMAIL|URL)([^:]*?):(.*?)item\1.X-ABLabel:(?:_\$!<)?([\w() -]*)(?:>!\$_)?./si',
                '/^item\d*\.X-AB.*$/mi',  // remove cruft like item1.X-AB*
                '/^item\d*\./mi',         // remove item1.ADR instead of ADR
                '/\n+/',                 // remove empty lines
                '/^(N:[^;\r\n]*)$/m',      // if N doesn't have any semicolons, add some
            ),
            array(
                '\2;type=\5\3:\4',
                '',
                '',
                "\n",
                '\1;;;;',
            ), $vcard);

        // convert X-WAB-GENDER to X-GENDER
        if (preg_match('/X-WAB-GENDER:(\d)/', $vcard, $matches)) {
            $value = $matches[1] == '2' ? 'male' : 'female';
            $vcard = preg_replace('/X-WAB-GENDER:\d/', 'X-GENDER:' . $value, $vcard);
        }

        return $vcard;
    }

ianbarton · 2019-06-07T17:13:17Z

Thanks. Still doesn't work for me. I'll try archiving my current installation and re-install from scratch.

amishmm · 2019-06-08T05:11:32Z

@ianbarton the fix worked for me too.. so it must be something wrong somewhere in your settings.

Arch linux - PHP 7.3.6 - Roundcube 1.3.9

ianbarton · 2019-06-08T10:31:09Z

Thanks all. I have just installed the fix on my backup server and works. So there must be something on my main server that is causing the problem.

amishmm · 2019-06-08T13:42:26Z

Make sure you have not disabled autocomplete in config files. Normally when something doesnt work we try to alter settings and when fix is found - we completely forget to revert the settings to normal. (human tendency)

ianbarton · 2019-06-19T10:55:37Z

Finally solved this, but not sure what caused the problem. Tried exporting my contacts to as vcard and they were all blank. Ran mysqlcheck which didn't report any problems. Browsed the contacts table with phpmyadmin and I could see all my contact data. If I tried adding back an existing contact using Roundcube, I got a warning about it already existing as I expected.

Finally deleted all my contacts using Roundcube and then added them back. All now working correctly.. Probably an attack by the gremlins.

linuxuser424 · 2019-07-24T07:51:39Z

Hi @ all,

I had the same problem on Debian with php 7.3 from sury.org (#6864).
The fix works here, too.

php -i | grep -i pcre pcre PCRE (Perl Compatible Regular Expressions) Support => enabled PCRE Library Version => 10.33 2019-04-16 PCRE Unicode Version => 11.0.0 PCRE JIT Support => enabled PCRE JIT Target => x86 64bit (little endian + unaligned) pcre.backtrack_limit => 1000000 => 1000000 pcre.jit => 1 => 1 pcre.recursion_limit => 100000 => 100000

Thank you.

GeorgSchlisio · 2019-07-25T14:23:29Z

Can confirm issue on Archlinux with RC 1.3.9 and PHP 7.3.7. Tweaking program/lib/Roundcube/rcube_vcard.php:539 to show \r\n instead of \R fixed the problem for me.

aommundsen · 2019-08-28T18:44:33Z

I'm trying to reproduce, but with no luck. It may be a PHP bug. What platform are you using? What php package version precisely? Could you provide output of php -i | grep -i pcre?

It seems the issue is only visible when running PCRE2 10.33, but when downgrading to 10.32 it works: https://forum.directadmin.com/showthread.php?t=58745

alecpl added the need feedback label May 10, 2019

darose closed this as completed May 10, 2019

alecpl reopened this May 10, 2019

alecpl added bug C: PHP backend and removed need feedback labels May 10, 2019

alecpl added this to the 1.4-rc2 milestone May 10, 2019

alecpl closed this as completed May 10, 2019

alecpl mentioned this issue Jul 18, 2019

Address helper in mail composition doesn't work with more than 1 character #6855

Closed

sven-ruess mentioned this issue Jul 24, 2019

Adressbook and php 7.3 #6864

Closed

This was referenced Aug 28, 2019

PCRE2 10.33 with PHP 7.3 causes preg_replace errors #6903

Closed

Autofill the names of recipients #6910

Closed

alecpl mentioned this issue Nov 25, 2019

Wrong pcre-metachar in rcube_vcard.php ? #7068

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Contacts no longer being displayed #6744

Contacts no longer being displayed #6744

darose commented May 9, 2019

alecpl commented May 10, 2019

darose commented May 10, 2019

alecpl commented May 10, 2019

darose commented May 10, 2019

alecpl commented May 10, 2019

alecpl commented May 10, 2019

alecpl commented May 10, 2019

darose commented May 10, 2019

alecpl commented May 11, 2019

pc-rabo commented Jun 5, 2019

ianbarton commented Jun 7, 2019

darose commented Jun 7, 2019

ianbarton commented Jun 7, 2019

darose commented Jun 7, 2019

ianbarton commented Jun 7, 2019

amishmm commented Jun 8, 2019 •

edited

Loading

ianbarton commented Jun 8, 2019

amishmm commented Jun 8, 2019 •

edited

Loading

ianbarton commented Jun 19, 2019

linuxuser424 commented Jul 24, 2019

GeorgSchlisio commented Jul 25, 2019 •

edited

Loading

aommundsen commented Aug 28, 2019

Contacts no longer being displayed #6744

Contacts no longer being displayed #6744

Comments

darose commented May 9, 2019

alecpl commented May 10, 2019

darose commented May 10, 2019

alecpl commented May 10, 2019

darose commented May 10, 2019

alecpl commented May 10, 2019

alecpl commented May 10, 2019

alecpl commented May 10, 2019

darose commented May 10, 2019

alecpl commented May 11, 2019

pc-rabo commented Jun 5, 2019

ianbarton commented Jun 7, 2019

darose commented Jun 7, 2019

ianbarton commented Jun 7, 2019

darose commented Jun 7, 2019

ianbarton commented Jun 7, 2019

amishmm commented Jun 8, 2019 • edited Loading

ianbarton commented Jun 8, 2019

amishmm commented Jun 8, 2019 • edited Loading

ianbarton commented Jun 19, 2019

linuxuser424 commented Jul 24, 2019

GeorgSchlisio commented Jul 25, 2019 • edited Loading

aommundsen commented Aug 28, 2019

amishmm commented Jun 8, 2019 •

edited

Loading

amishmm commented Jun 8, 2019 •

edited

Loading

GeorgSchlisio commented Jul 25, 2019 •

edited

Loading