Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Mail.ru search engine results encoding has changed #2761

Closed
anonymous-piwik-user opened this Issue · 8 comments

3 participants

@anonymous-piwik-user

File: SearchEngines.php

Original (shows incorrect encoding):
// Mail.ru
'go.mail.ru' => array('Mailru', 'q', 'search?q={k}', 'windows-1251'),

I changed to:
// Mail.ru
'go.mail.ru' => array('Mailru', 'q', 'search?rch=e&q={k}'),

And now it seems to work correctly.

@robocoder

(In [5413]) fixes #2761 - confirmed that go.mail.ru search results are now utf-8

@kiav

As for now, Mail.ru uses UTF-8 in most cases. But rarely it still uses windows-1251 too.

I had to change extractSearchEngineInformationFromUrl function in /core/Common.php

if(function_exists('iconv')
    && isset($searchEngines[$refererHost][3]))
{
    // accepts string, array or comma separated list string in preferred order
    if (!is_array($searchEngines[$refererHost][3]))
        $charsets = explode(',', $searchEngines[$refererHost][3]);
    else
        $charsets = $searchEngines[$refererHost][3];

    if(!empty($charsets))
    {
        $charset = mb_detect_encoding($key, $charsets);
        if ($charset === false)
            $charset = $charsets[0];

        $newkey = @iconv($charset, 'UTF-8//IGNORE', $key);
        if(!empty($newkey))
        {
            $key = $newkey;
        }
    }
}

It works with

'go.mail.ru' => array('Mailru', 'q', 'search?q={k}', array('UTF-8', 'windows-1251')),

in /core/DataFiles/SearchEngines.php

@robocoder

Thanks for the patch.

I don't think we need to support comma separated list. We do have to check for mbstring and have a unit test.

@kiav

Comma separated list is already supported by mb_detect_encoding.

By the way, mb_strtolower is already used in Common.php (in original Piwik code in the extractSearchEngineInformationFromUrl function) without any checks tests.

@robocoder

Can you provide a sample referrer url with windows-1251 encoding?

I've done some refactoring and added some more tests, but can never have enough.

@robocoder

Awesome! Thanks!

@robocoder

(In [5682]) fixes #2761

@anonymous-piwik-user anonymous-piwik-user added this to the 1.7 Piwik 1.7 milestone
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.