Browse files

Test multiple database / storage adapters using Behat (#550)

* Enable autoloading of Behat context classes

Instead of adding the path of the Behat feature context classes to the
Behat configuration file, and manually including the files in the
bootstrap for the unit tests the directory is added to autoload-dev in

* Add interface for Behat feature context classes

The interface must be implemented by all feature context classes so they
can provide an instance of a database and storage adapter to be used in
the different suites for Behat.

* Use PSR-4 instead of PSR-0 for context classes

To make the name of the Behat context feature classes more clear, we
should use PSR-4 instead of PSR-0. This allows for the usage of
underscores in the classname without having the autoloader automatically
assuming the _ in the classname matches a / in the path.

Because of this we can now match:

Database_Storage => %path%/Database_Storage.php

and not

Database_Storage => %path%/Database/Storage.php

* Enable Behat testing of all adapters

This commit introduces different suites in the Behat test suite, which
enables testing of all the different database and storage adapters, and
combinations of these.

Currently only the MongoDB database adapter and the GridFS storage
adapter is tested, which has been the case since Imbo first started
using Behat for integration tests.

Each suite, which is a combination of a database adapter and a storage
adapter should be added to behat.yml.dist, and should follow the
following naming convention:

<database adapter>_<storage adapter>, for instance "mongodb_gridfs" and
must refer to a context class which follows the same convention, only
that the names should match the specific names of the adapters:

mongodb_gridfs => MongoDB_GridFS which in turn refers to the following

- Imbo\Database\MongoDB
- Imbo\Storage\GridFS

Each new context class must implement the ImboFeatureContext interface,
which introduces the two methods that is required to return the database
and storage adapters. Each feature context can also have one or more
Behat hooks that can be used to setup / tear down the specific suites.
Each suite is expected to clear up all resources it generates in the
test run.

Resolves #279

* Add a namespace for Behat feature context classes

This commits adds ImboBehatFeatureContext as a namespace to all classes
related to the feature context classes used in Behat tests. Some classes
have also been renamed to better suit their purpose.

* Add traits for database and storage adapters

To more easily test combinations of database and storage adapters in the
Behat test suites we can use traits for the different adapters. Each
adapter should be responsible of setup / teardown functionality related
to the different adapters. Each suite could then simply use two traits
in combination to test them.

* Skip traits and use configuration for suites

Instead of having traits for the different adapters we want to test, and
a class for each combination this commit will use configuration from the
suites in the behat.yml[.dist] file instead. We also need to exchange
configuration between the context triggered by Behat (which runs ::setUp() /
::tearDown()), and the context from when Imbo gets a request from the test
(::getAdapter). This is done with a request header. The main feature
context has been renamed back to FeatureContext as we no longer need the
previously added FeatureContext interface. The interface added two
methods that is no longer needed.

* Add suite for MongoDB and Filesystem adapters

* Provide sane default values for the Images model

This gives the Imbo\Model\Images model sane deafult values for hits,
limit and page.

Resolves #553

* Add test-suite for Doctrine/SQLite and filesystem

Adds a separate suite for testing the combination of the
Imbo\Database\Doctrine database adapter (with a SQLite database) and the
Imbo\Storage\Filesystem storage adapter.

* Add database setup for Doctrine/SQLite test

The doctrine_sqlite_filesystem suite uses a SQLite database for testing,
and this class is responsible for setting up / clearing the database
used for testing.

* Adjust comparison to work for non-type databases

As the Doctrine database does not handle types in the same way as MySQL
/ MongoDB we need to adjust the tests as the Behat API Extension is
type-sensitive on comparisons such as theses.

* Order by the update flag for correct results

To get the lastest value from the updated column the adapter must add

Resolves #556.

* Add method for fetching users in the database

This is a helper method for fetching the unique user names currently
found in the database. If no users-filter is used with the global images
endpoint we need to fetch the list of all users currently in the
database to be sure that the public key has access to all of them.

* Clean up tests for database adapters

Some missing tests have been added, and a generic clean-up has been done
as well, as will be done to all test cases. Add test for the newly added

* Fix getImages and getLastModified

Implement the new behaviour for getImages, where images from all users
will be returned if the users-parameter is empty. The
getLastModified-methods are also updated and implements the behaviour as
specified in the method documentation found in the interface.

* Update docs to reflect new behaviour of /images

* Remove is_array check

The Request::getUsers() method always returns an array, so no need for
the check.

* Clean up test case annotations

* Fetch all users for auth when query param is empty

When the client does not specify the users-filter when requesting the
global images endpoint Imbo should use all users found in the current
database to use for authentication. Also added test case to verify new

* Rename test class

* Use setup file for SQL

Instead of duplicating the SQL for creating the tables in the database
the test classes now use the contents of the file found in the setup dir,
which is also included in the documentation.

* Supply suite settings for the setUp method

Some suites needs extra settings that can be used in the set up, so this
commit adds a parameter to the setUp method that holds all settings for
the  current suite.

* Add suite for Doctrine (MySQL) and Filesystem

This commit adds a Behat suite for testing the Doctrine adapter with a
MySQL database in combination with the Filesystem storage adapter. For
this to work we need to enable the mysql service in Travis-CI.

* Update README to reflect new Behat suites

* Fix issues raised in code review of #550

As we can't define constants through the Behat configuration file I
decided to simply add a config entry in each suite for the path to the
project root. The suites don't inherit configuration from some default
place, so the same key => value has to be defined in every suite. For
PHPUnit a project root constant has been set in the bootstrap script.
  • Loading branch information...
christeredvartsen committed Jul 28, 2017
1 parent 8de7ffd commit c8f210355b2bd18445ca363f4c7e9c825c89576c
@@ -22,12 +22,17 @@ branches:
- mongodb
- mysql
- pecl list
- php -i
- printf "\n" | pecl install --force mongodb
- printf "\n" | pecl install imagick
- mysql -e "CREATE DATABASE IF NOT EXISTS imbo_test;" -u root
- mysql -e "CREATE USER imbo_test@localhost IDENTIFIED BY 'imbo_test';" -u root
- mysql -e "GRANT ALL PRIVILEGES ON imbo_test.* TO imbo_test@localhost;" -u root
- phpenv config-rm xdebug.ini
@@ -1,9 +1,33 @@
'': %paths.base%/tests/behat/features/bootstrap
project_root: %paths.base%
paths: [%paths.base%/tests/behat/features]
contexts: [ImboBehatFeatureContext\FeatureContext]
database: MongoDB
storage: GridFS
project_root: %paths.base%
paths: [%paths.base%/tests/behat/features]
contexts: [ImboBehatFeatureContext\FeatureContext]
database: MongoDB
storage: Filesystem
project_root: %paths.base%
paths: [%paths.base%/tests/behat/features]
contexts: [ImboBehatFeatureContext\FeatureContext]
database: DoctrineSQLite
storage: Filesystem
project_root: %paths.base%
paths: [%paths.base%/tests/behat/features]
contexts: [ImboBehatFeatureContext\FeatureContext]
database: DoctrineMySQL
database.hostname: localhost
database.database: imbo_test
database.username: imbo_test
database.password: imbo_test
storage: Filesystem
@@ -63,7 +63,8 @@
"autoload-dev": {
"psr-4": {
"ImboUnitTest\\": "tests/phpunit/unit",
"ImboIntegrationTest\\": "tests/phpunit/integration"
"ImboIntegrationTest\\": "tests/phpunit/integration",
"ImboBehatFeatureContext\\": "tests/behat/features/bootstrap"
"bin": [
@@ -16,6 +16,11 @@ Below are the changes you need to be aware of when upgrading to Imbo-3.0.0.
:depth: 2
Global images endpoint returns images for all users
If the ``users`` filter is excluded when making requests against the :ref:`global-images-resource` resource, images from all users will be returned if the public key used has access to **all** users in the result set.
MD5 image identifier generator has been removed
@@ -636,7 +636,7 @@ Global images resource - ``/images``
The global images resource is used to search for images across users, given that the public key has access to the images of these users.
This resource is read only, and behaves in the same way as described in the `Get image collections` section of :ref:`images-resource`. In addition to the parameters specified for `Get image collections`, the following query parameter must be specified:
This resource is read only, and behaves in the same way as described in the `Get image collections` section of :ref:`images-resource`. In addition to the parameters specified for `Get image collections`, the following query parameter can be specified:
An array of users to get images for.
@@ -647,6 +647,8 @@ This resource is read only, and behaves in the same way as described in the `Get
results in a response with the exact same format as shown under `Get image collections`.
If the ``users[]`` parameter is not set, the endpoint will return images from all users, granted that the public key has access to all users present in the database.
.. _publickey-resource:
Public key resource - ``/keys/<publicKey>``
@@ -86,7 +86,8 @@ function deleteMetadata($user, $imageIdentifier);
* This method is also responsible for setting a correct "hits" number in the images model.
* @param array $users The users which the images belongs to
* @param array $users The users which the images belongs to. If an empty array is specified
* the adapter should return images for all users.
* @param Query $query A query instance
* @param Images $model The images model
* @return array
@@ -117,9 +118,11 @@ function getImageProperties($user, $imageIdentifier);
* Get the last modified timestamp for given users
* If the $imageIdentifier parameter is set, return when that image was last updated. If not
* set, return the most recent date when one of the specified users last updated any image. If
* the provided users does not have any images stored, return the current timestamp.
* Find the last modification timestamp of one or more users. If the image identifier parameter
* is set the query will only look for that image in the set of users. If none of the specified
* users have the image a 404 exception will be thrown. If the image identifier is skipped the
* method will return either the current timestamp, or the max timestamp of any of the given
* users.
* @param array $users The users
* @param string $imageIdentifier The image identifier
@@ -225,4 +228,11 @@ function getShortUrlParams($shortUrlId);
* @return boolean
function deleteShortUrls($user, $imageIdentifier, $shortUrlId = null);
* Return a list of the users present in the database
* @return string[]
function getAllUsers();
@@ -10,20 +10,19 @@
namespace Imbo\Database;
use Imbo\Model\Image,
use Imbo\Model\Image;
use Imbo\Model\Images;
use Imbo\Resource\Images\Query;
use Imbo\Exception\DatabaseException;
use Imbo\Exception\InvalidArgumentException;
use Imbo\Exception\DuplicateImageIdentifierException;
use Doctrine\DBAL\DriverManager;
use Doctrine\DBAL\Connection;
use Doctrine\DBAL\DBALException;
use Doctrine\DBAL\Exception\UniqueConstraintViolationException;
use PDO;
use DateTime;
use DateTimeZone;
* Doctrine 2 database driver
@@ -240,16 +239,18 @@ public function getImages(array $users, Query $query, Images $model) {
$qb = $this->getConnection()->createQueryBuilder();
$qb->select('*')->from($this->tableNames['imageinfo'], 'i');
// Filter on users
$expr = $qb->expr();
$composite = $expr->orX();
if ($users) {
// Filter on users
$expr = $qb->expr();
$composite = $expr->orX();
foreach ($users as $i => $user) {
$composite->add($expr->eq('i.user', ':user' . $i));
$qb->setParameter(':user' . $i, $user);
foreach ($users as $i => $user) {
$composite->add($expr->eq('i.user', ':user' . $i));
$qb->setParameter(':user' . $i, $user);
if ($sort = $query->sort()) {
// Fields valid for sorting
@@ -414,29 +415,32 @@ public function load($user, $imageIdentifier, Image $image) {
public function getLastModified(array $users, $imageIdentifier = null) {
$query = $this->getConnection()->createQueryBuilder();
->from($this->tableNames['imageinfo'], 'i');
->from($this->tableNames['imageinfo'], 'i')
->orderBy('i.updated', 'DESC')
// Filter on users
$expr = $query->expr();
$composite = $expr->orX();
if (!empty($users)) {
$expr = $query->expr();
$composite = $expr->orX();
foreach ($users as $i => $user) {
$composite->add($expr->eq('i.user', ':user' . $i));
$query->setParameter(':user' . $i, $user);
foreach ($users as $i => $user) {
$composite->add($expr->eq('i.user', ':user' . $i));
$query->setParameter(':user' . $i, $user);
if ($imageIdentifier) {
if ($imageIdentifier !== null) {
$query->andWhere('i.imageIdentifier = :imageIdentifier')
->setParameter(':imageIdentifier', $imageIdentifier);
$stmt = $query->execute();
$row = $stmt->fetch(PDO::FETCH_ASSOC);
if (!$row && $imageIdentifier) {
if (!$row && $imageIdentifier !== null) {
throw new DatabaseException('Image not found', 404);
} else if (!$row) {
$row = ['updated' => time()];
@@ -628,6 +632,17 @@ public function deleteShortUrls($user, $imageIdentifier, $shortUrlId = null) {
return (boolean) $qb->execute();
* {@inheritdoc}
public function getAllUsers() {
$query = $this->getConnection()->createQueryBuilder();
->from($this->tableNames['imageinfo'], 'i');
return array_column($query->execute()->fetchAll(), 'user');
* Get the Doctrine connection
@@ -15,6 +15,7 @@
MongoDB\Client as MongoClient,
@@ -268,9 +269,13 @@ public function deleteMetadata($user, $imageIdentifier) {
public function getImages(array $users, Query $query, Images $model) {
// Initialize return value
$images = [];
$queryData = [];
// Query data
$queryData = ['user' => ['$in' => $users]];
if ($users) {
// Only filter on users if the array contains any values
$queryData['user']['$in'] = $users;
$from = $query->from();
$to = $query->to();
@@ -398,16 +403,17 @@ public function load($user, $imageIdentifier, Image $image) {
* {@inheritdoc}
public function getLastModified(array $users, $imageIdentifier = null) {
try {
// Query on the user
$query = ['user' => ['$in' => $users]];
$query = [];
if ($imageIdentifier) {
// We want information about a single image. Add the identifier to the query
$query['imageIdentifier'] = $imageIdentifier;
if ($users) {
$query['user']['$in'] = $users;
if ($imageIdentifier !== null) {
$query['imageIdentifier'] = $imageIdentifier;
// Find a document
try {
$data = $this->getImageCollection()->findOne($query, [
'sort' => [
'updated' => -1,
@@ -420,7 +426,7 @@ public function getLastModified(array $users, $imageIdentifier = null) {
throw new DatabaseException('Unable to fetch image data', 500, $e);
if ($data === null && $imageIdentifier) {
if ($data === null && $imageIdentifier !== null) {
throw new DatabaseException('Image not found', 404);
} else if ($data === null) {
$data = ['updated' => time()];
@@ -634,6 +640,13 @@ public function deleteShortUrls($user, $imageIdentifier, $shortUrlId = null) {
return true;
* {@inheritdoc}
public function getAllUsers() {
return $this->getImageCollection()->distinct('user');
* Fetch the image collection
@@ -235,10 +235,6 @@ public function loadImages(EventInterface $event) {
$users = $event->getArgument('users');
} else {
$users = $event->getRequest()->getUsers();
if (!is_array($users)) {
$users = [];
$response = $event->getResponse();
@@ -36,21 +36,21 @@ class Images implements ModelInterface {
* @var int
private $hits;
private $hits = 0;
* Limit the number of images
* @var int
private $limit;
private $limit = 20;
* The page number
* @var int
private $page;
private $page = 1;
* Set the array of images
@@ -55,7 +55,7 @@ public function getImages(EventInterface $event) {
$acl = $event->getAccessControl();
$missingAccess = [];
$users = $event->getRequest()->getUsers();
$users = $event->getRequest()->getUsers() ?: $event->getDatabase()->getAllUsers();
foreach ($users as $user) {
$hasAccess = $acl->hasAccess(
Oops, something went wrong.

0 comments on commit c8f2103

Please sign in to comment.