Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implements wrapper method for a more secure unserialize with PHP 7 #13285

Merged
merged 4 commits into from
Oct 10, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,10 +63,10 @@ matrix:
sudo: false
addons: false
# All tests after another
- php: 5.6
- php: 7
env: TEST_SUITE=AllTests MYSQL_ADAPTER=MYSQLI ALLTEST_EXTRA_OPTIONS="--run-first-half-only"
sudo: required
- php: 5.6
- php: 7
env: TEST_SUITE=AllTests MYSQL_ADAPTER=MYSQLI ALLTEST_EXTRA_OPTIONS="--run-second-half-only"
sudo: required
# UITests use a specific version because the default 5.5 (== 5.5.38) is missing FreeType support
Expand Down
27 changes: 27 additions & 0 deletions core/Common.php
Original file line number Diff line number Diff line change
Expand Up @@ -256,6 +256,33 @@ public static function mb_strtoupper($string)
return $string;
}

/**
* Secure wrapper for unserialize, which by default disallows unserializing classes
*
* @param string $string String to unserialize
* @param array $allowedClasses Class names that should be allowed to unserialize
*
* @return mixed
*/
public static function safe_unserialize($string, $allowedClasses = [])
{
if (PHP_MAJOR_VERSION >= 7) {
try {
return unserialize($string, ['allowed_classes' => empty($allowedClasses) ? false : $allowedClasses]);
} catch (\Throwable $e) {
$logger = StaticContainer::get('Psr\Log\LoggerInterface');
$logger->debug('Unable to unserialize a string: {message} (string = {string})', [
'message' => $e->getMessage(),
'backtrace' => $e->getTraceAsString(),
'string' => $string,
]);
return false;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we log the string as well? For a debug log I presume it's ok.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added this myself, let me know if that's a problem (can remove in new PR if needed).

}
}

return @unserialize($string);
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should unserialize errors be reported in any way? Is it a problem if an exception is thrown here?

Copy link
Member

@diosmosis diosmosis Sep 28, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see a lot of uses of @unserialize(), what about allowing callers to specify whether Common::safe_unserialize() should have errors silenced? Think that might be more flexible.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@diosmosis you mean something like a third parameter that defines whether to ignore error or not?
Btw. unserialize doesn't throw an exception if the values can't be deserialized. It triggers a E_NOTICE. For PHP 7 it might be fine as it's "catchable" with the Error class. For PHP 5.x that simply results in errors depending on the php configuration.
Do you think it might be suitable to always ignore errors ion PHP 5 and always catch errors on PHP 7, but trigger a Log::debug in that case?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought ErrorHandler might turn the notice into an exception, but I can't really remember and haven't tested it... But for this I figured we could let the callers silence Common::safe_unserialize() if they wanted, eg:

$value = @Common::safe_unserialize($value);

I thought about exceptions but noticed some of the uses you changed assume it will only return false on failure, so I guess we can't throw everywhere.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 for logging, though

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a debug log for that case and also added some tests...


/*
* Escaping input
*/
Expand Down
3 changes: 2 additions & 1 deletion core/Concurrency/DistributedList.php
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
*/
namespace Piwik\Concurrency;

use Piwik\Common;
use Piwik\Container\StaticContainer;
use Piwik\Option;
use Psr\Log\LoggerInterface;
Expand Down Expand Up @@ -160,7 +161,7 @@ protected function getListOptionValue()

$result = array();
if ($array
&& ($array = unserialize($array))
&& ($array = Common::safe_unserialize($array))
&& count($array)
) {
$result = $array;
Expand Down
4 changes: 2 additions & 2 deletions core/CronArchive.php
Original file line number Diff line number Diff line change
Expand Up @@ -864,7 +864,7 @@ protected function processArchiveDays($idSite, $lastTimestampWebsiteProcessedDay
}

$content = $this->request($url);
$daysResponse = @unserialize($content);
$daysResponse = Common::safe_unserialize($content);

if (empty($content)
|| !is_array($daysResponse)
Expand Down Expand Up @@ -1015,7 +1015,7 @@ private function archiveReportsFor($idSite, $period, $date, $archiveSegments, Ti
$success = $success && $this->checkResponse($content, $url);

if ($noSegmentUrl == $url && $success) {
$stats = @unserialize($content);
$stats = Common::safe_unserialize($content);

if (!is_array($stats)) {
$this->logError("Error unserializing the following response from $url: " . $content);
Expand Down
2 changes: 1 addition & 1 deletion core/DataAccess/ArchiveSelector.php
Original file line number Diff line number Diff line change
Expand Up @@ -316,7 +316,7 @@ public static function getArchiveData($archiveIds, $recordNames, $archiveDataTyp
private static function moveChunkRowToRows(&$rows, $row, Chunk $chunk, $loadAllSubtables, $idSubtable)
{
// $blobs = array([subtableID] = [blob of subtableId])
$blobs = unserialize($row['value']);
$blobs = Common::safe_unserialize($row['value']);

if (!is_array($blobs)) {
return;
Expand Down
6 changes: 5 additions & 1 deletion core/DataTable.php
Original file line number Diff line number Diff line change
Expand Up @@ -1336,7 +1336,11 @@ public function getSerialized($maximumRowsInDataTable = null,
private function unserializeRows($serialized)
{
$serialized = str_replace(self::$previousRowClasses, self::$rowClassToUseForUnserialize, $serialized);
$rows = unserialize($serialized);
$rows = Common::safe_unserialize($serialized, [
Row::class,
DataTableSummaryRow::class,
\Piwik_DataTable_SerializedRow::class
]);

if ($rows === false) {
throw new Exception("The unserialization has failed!");
Expand Down
3 changes: 2 additions & 1 deletion core/DataTable/Renderer/Php.php
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
namespace Piwik\DataTable\Renderer;

use Exception;
use Piwik\Common;
use Piwik\DataTable\Renderer;
use Piwik\DataTable\Simple;
use Piwik\DataTable;
Expand Down Expand Up @@ -77,7 +78,7 @@ public function render($dataTable = null)

if ($this->prettyDisplay) {
if (!is_array($toReturn)) {
$toReturn = unserialize($toReturn);
$toReturn = Common::safe_unserialize($toReturn);
}
$toReturn = "<pre>" . var_export($toReturn, true) . "</pre>";
}
Expand Down
3 changes: 2 additions & 1 deletion core/Scheduler/Timetable.php
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@

namespace Piwik\Scheduler;

use Piwik\Common;
use Piwik\Option;
use Piwik\Date;

Expand All @@ -24,7 +25,7 @@ class Timetable
public function __construct()
{
$optionData = Option::get(self::TIMETABLE_OPTION_STRING);
$unserializedTimetable = @unserialize($optionData);
$unserializedTimetable = Common::safe_unserialize($optionData);

$this->timetable = $unserializedTimetable === false ? array() : $unserializedTimetable;
}
Expand Down
2 changes: 1 addition & 1 deletion core/Tracker/Visit/ReferrerSpamFilter.php
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ private function loadSpammerList()
$list = Option::get(self::OPTION_STORAGE_NAME);

if ($list) {
$this->spammerList = unserialize($list);
$this->spammerList = Common::safe_unserialize($list);
} else {
// Fallback to reading the bundled list
$file = PIWIK_VENDOR_PATH . '/matomo/referrer-spam-blacklist/spammers.txt';
Expand Down
2 changes: 1 addition & 1 deletion core/Updates/3.0.0-b1.php
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ private function getPluginSettingsMigrations($queries)
foreach ($options as $option) {
$name = $option['option_name'];
$pluginName = str_replace(array('Plugin_', '_Settings'), '', $name);
$values = @unserialize($option['option_value']);
$values = Common::safe_unserialize($option['option_value']);

if (empty($values)) {
continue;
Expand Down
3 changes: 2 additions & 1 deletion plugins/Annotations/AnnotationList.php
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
namespace Piwik\Plugins\Annotations;

use Exception;
use Piwik\Common;
use Piwik\Date;
use Piwik\Option;
use Piwik\Piwik;
Expand Down Expand Up @@ -330,7 +331,7 @@ private function getAnnotationsForSite()
$serialized = Option::get($optionName);

if ($serialized !== false) {
$result[$id] = @unserialize($serialized);
$result[$id] = Common::safe_unserialize($serialized);
if (empty($result[$id])) {
// in case unserialize failed
$result[$id] = array();
Expand Down
2 changes: 1 addition & 1 deletion plugins/Referrers/SearchEngine.php
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ private function loadDefinitions()
$list = Option::get(self::OPTION_STORAGE_NAME);

if ($list) {
$this->definitionList = unserialize(base64_decode($list));
$this->definitionList = Common::safe_unserialize(base64_decode($list));
} else {
// Fallback to reading the bundled list
$yml = file_get_contents(PIWIK_INCLUDE_PATH . self::DEFINITION_FILE);
Expand Down
3 changes: 2 additions & 1 deletion plugins/Referrers/Social.php
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
*/
namespace Piwik\Plugins\Referrers;
use Piwik\Cache;
use Piwik\Common;
use Piwik\Option;
use Piwik\Piwik;
use Piwik\Singleton;
Expand Down Expand Up @@ -51,7 +52,7 @@ private function loadDefinitions()
$list = Option::get(self::OPTION_STORAGE_NAME);

if ($list) {
$this->definitionList = unserialize(base64_decode($list));
$this->definitionList = Common::safe_unserialize(base64_decode($list));
} else {
// Fallback to reading the bundled list
$yml = file_get_contents(PIWIK_INCLUDE_PATH . self::DEFINITION_FILE);
Expand Down
4 changes: 2 additions & 2 deletions plugins/UserCountryMap/Controller.php
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ public function visitorMap($fetch = false, $segmentOverride = false)
. '&filter_limit=-1'
);
$config = array();
$config['visitsSummary'] = unserialize($request->process());
$config['visitsSummary'] = Common::safe_unserialize($request->process());
$config['countryDataUrl'] = $this->_report('UserCountry', 'getCountry',
$this->idSite, $period, $date, $token_auth, false, $segment);
$config['regionDataUrl'] = $this->_report('UserCountry', 'getRegion',
Expand Down Expand Up @@ -277,7 +277,7 @@ private function getMetrics($idSite, $period, $date, $token_auth)
. '&token_auth=' . $token_auth
. '&filter_limit=-1'
);
$metaData = unserialize($request->process());
$metaData = Common::safe_unserialize($request->process());

$metrics = array();
if (!empty($metaData[0]['metrics']) && is_array($metaData[0]['metrics'])) {
Expand Down
39 changes: 39 additions & 0 deletions tests/PHPUnit/Unit/CommonTest.php
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,13 @@

use Exception;
use PHPUnit_Framework_TestCase;
use Piwik\Application\Environment;
use Piwik\Common;
use Piwik\Container\StaticContainer;
use Piwik\Filesystem;
use Piwik\Intl\Data\Provider\RegionDataProvider;
use Piwik\Log;
use Piwik\Tests\Framework\Mock\FakeLogger;

/**
* @backupGlobals enabled
Expand Down Expand Up @@ -279,6 +282,42 @@ public function testIsValidFilenameNotValidValues()
}
}

public function testSafeUnserialize()
{
if (PHP_MAJOR_VERSION < 7) {
$this->markTestSkipped('secure unserialize tests are for PHP7 only');
return;
}

// should unserialize an allowed class
$this->assertTrue(Common::safe_unserialize('O:12:"Piwik\Common":0:{}', ['Piwik\Common']) instanceof Common);

// not allowed classed should result in an incomplete class
$this->assertTrue(Common::safe_unserialize('O:12:"Piwik\Common":0:{}') instanceof \__PHP_Incomplete_Class);

// strings not unserializable should return false and trigger a debug log
$logger = $this->createFakeLogger();
$this->assertFalse(Common::safe_unserialize('{1:somebroken}'));
$this->assertContains('Unable to unserialize a string: unserialize(): Error at offset 0 of 14 bytes', $logger->output);
}

private function createFakeLogger()
{
$logger = new FakeLogger();

$newEnv = new Environment('test', array(
'Psr\Log\LoggerInterface' => $logger,
'Tests.log.allowAllHandlers' => true,
));
$newEnv->init();

$newMonologLogger = $newEnv->getContainer()->make('Psr\Log\LoggerInterface');
$oldLogger = new Log($newMonologLogger);
Log::setSingletonInstance($oldLogger);

return $logger;
}

/**
* Dataprovider for testGetBrowserLanguage
*/
Expand Down