Test multiple database / storage adapters using Behat #550

christeredvartsen · 2017-06-23T15:19:05Z

This PR enables testing of all the available database and storage adapters in Imbo by adding suites to behat.yml.dist. Each suite must have an accompanying feature context class that must implement the ImboFeatureContext interface.

This currently only allows testing of the different database (Imbo\Database) and storage (Imbo\Storage) adapters. It should be expanded to also enable testing of other configurable adapters:

Imbo\Auth\AccessControl\Adapter
Imbo\EventListener\ImageVariations\Database
Imbo\EventListener\ImageVariations\Storage

Instead of adding the path of the Behat feature context classes to the Behat configuration file, and manually including the files in the bootstrap for the unit tests the directory is added to autoload-dev in composer.json

The interface must be implemented by all feature context classes so they can provide an instance of a database and storage adapter to be used in the different suites for Behat.

To make the name of the Behat context feature classes more clear, we should use PSR-4 instead of PSR-0. This allows for the usage of underscores in the classname without having the autoloader automatically assuming the _ in the classname matches a / in the path. Because of this we can now match: Database_Storage => %path%/Database_Storage.php and not Database_Storage => %path%/Database/Storage.php

This commit introduces different suites in the Behat test suite, which enables testing of all the different database and storage adapters, and combinations of these. Currently only the MongoDB database adapter and the GridFS storage adapter is tested, which has been the case since Imbo first started using Behat for integration tests. Each suite, which is a combination of a database adapter and a storage adapter should be added to behat.yml.dist, and should follow the following naming convention: <database adapter>_<storage adapter>, for instance "mongodb_gridfs" and must refer to a context class which follows the same convention, only that the names should match the specific names of the adapters: mongodb_gridfs => MongoDB_GridFS which in turn refers to the following adapters: - Imbo\Database\MongoDB - Imbo\Storage\GridFS Each new context class must implement the ImboFeatureContext interface, which introduces the two methods that is required to return the database and storage adapters. Each feature context can also have one or more Behat hooks that can be used to setup / tear down the specific suites. Each suite is expected to clear up all resources it generates in the test run. Resolves #279

christeredvartsen · 2017-06-23T15:34:11Z

Before merging this PR the following adapters must be included in the tests:

Imbo\Database\Doctrine (MySQL) (e6b12bf)
Imbo\Database\Doctrine (SQLite) (134d8ec)
~~Imbo\Database\Doctrine (Postgres)~~
Imbo\Database\MongoDB (9bdd216)
Imbo\Storage\Filesystem (a3ec2c6)
~~Imbo\Storage\S3~~ (*)
~~Imbo\Storage\B2~~ (*)
Imbo\Storage\GridFS (9bdd216)

*) These might be somewhat problematic to test on Travis. If we are to test them we need to use private environment variables on Travis, and the test-runs might actually end up costing $'s.

This commits adds ImboBehatFeatureContext as a namespace to all classes related to the feature context classes used in Behat tests. Some classes have also been renamed to better suit their purpose.

To more easily test combinations of database and storage adapters in the Behat test suites we can use traits for the different adapters. Each adapter should be responsible of setup / teardown functionality related to the different adapters. Each suite could then simply use two traits in combination to test them.

Instead of having traits for the different adapters we want to test, and a class for each combination this commit will use configuration from the suites in the behat.yml[.dist] file instead. We also need to exchange configuration between the context triggered by Behat (which runs ::setUp() / ::tearDown()), and the context from when Imbo gets a request from the test (::getAdapter). This is done with a request header. The main feature context has been renamed back to FeatureContext as we no longer need the previously added FeatureContext interface. The interface added two methods that is no longer needed.

This gives the Imbo\Model\Images model sane deafult values for hits, limit and page. Resolves #553

Adds a separate suite for testing the combination of the Imbo\Database\Doctrine database adapter (with a SQLite database) and the Imbo\Storage\Filesystem storage adapter.

The doctrine_sqlite_filesystem suite uses a SQLite database for testing, and this class is responsible for setting up / clearing the database used for testing.

As the Doctrine database does not handle types in the same way as MySQL / MongoDB we need to adjust the tests as the Behat API Extension is type-sensitive on comparisons such as theses.

To get the lastest value from the updated column the adapter must add ordering. Resolves #556.

This is a helper method for fetching the unique user names currently found in the database. If no users-filter is used with the global images endpoint we need to fetch the list of all users currently in the database to be sure that the public key has access to all of them.

Some missing tests have been added, and a generic clean-up has been done as well, as will be done to all test cases. Add test for the newly added getAllUsers-method.

Implement the new behaviour for getImages, where images from all users will be returned if the users-parameter is empty. The getLastModified-methods are also updated and implements the behaviour as specified in the method documentation found in the interface.

The Request::getUsers() method always returns an array, so no need for the check.

When the client does not specify the users-filter when requesting the global images endpoint Imbo should use all users found in the current database to use for authentication. Also added test case to verify new behaviour.

Instead of duplicating the SQL for creating the tables in the database the test classes now use the contents of the file found in the setup dir, which is also included in the documentation.

Some suites needs extra settings that can be used in the set up, so this commit adds a parameter to the setUp method that holds all settings for the current suite.

This commit adds a Behat suite for testing the Doctrine adapter with a MySQL database in combination with the Filesystem storage adapter. For this to work we need to enable the mysql service in Travis-CI.

christeredvartsen · 2017-07-28T08:32:33Z

@matslindh Care do to a quick review so we can get this merged? I'm not including suites for Postgres (Doctrine), S3 or B2 for now. These can be added later.

matslindh

I'm not familiar with the Behat stuff, so I've just given it a very rudimentary read. A few stylistic changes otherwise. Also surprised that we don't have visibility qualifiers (public) in the interface definition?

matslindh · 2017-07-28T09:16:15Z

src/Database/Doctrine.php


-        if ($imageIdentifier) {
+        if ($imageIdentifier !== null) {
            $query->andWhere('i.imageIdentifier = :imageIdentifier')
                  ->setParameter(':imageIdentifier', $imageIdentifier);
        }

        $stmt = $query->execute();


We should probably have something similar to LIMIT 1; here, so that the DB doesn't have to build the complete result set, just find the one sorted at the top? Or SELECT MAX(...)..

matslindh · 2017-07-28T09:17:32Z

src/Database/Doctrine.php

+
+        return array_map(function($row) {
+            return $row['user'];
+        }, $query->execute()->fetchAll());


Replace with array_column instead?

matslindh · 2017-07-28T09:18:01Z

src/Database/MongoDB.php


        // Query data
-        $queryData = ['user' => ['$in' => $users]];
+        if (!empty($users)) {
+            // Only filter on users if the array contains any values


This could be just if ($users)

matslindh · 2017-07-28T09:18:31Z

src/Database/Doctrine.php

-        // Filter on users
-        $expr = $qb->expr();
-        $composite = $expr->orX();
+        if (!empty($users)) {


This could be just if ($users)

matslindh · 2017-07-28T09:18:55Z

src/Database/MongoDB.php

-                // We want information about a single image. Add the identifier to the query
-                $query['imageIdentifier'] = $imageIdentifier;
-            }
+        if (!empty($users)) {


This could be just if ($users)

matslindh · 2017-07-28T09:20:41Z

tests/behat/features/bootstrap/DatabaseTest/DoctrineMySQL.php

+
+        array_map(function($query) use ($pdo) {
+            $pdo->query($query);
+        }, explode("\n\n", file_get_contents(__DIR__ . '/../../../../../setup/doctrine.mysql.sql')));


This seems very convoluted - could we introduce a PROJECT_ROOT or something similar in the test setup? I have no idea where this file lives from trying to parse the string..

matslindh · 2017-07-28T09:21:04Z

tests/behat/features/bootstrap/DatabaseTest/DoctrineSQLite.php

+
+        array_map(function($query) use ($pdo) {
+            $pdo->query($query);
+        }, explode("\n\n", file_get_contents(__DIR__ . '/../../../../../setup/doctrine.sqlite.sql')));


Same here - PROJECT_ROOT or similar would be helpful.

matslindh · 2017-07-28T09:21:51Z

tests/phpunit/integration/Database/DoctrineTest.php

+
+        array_map(function($query) {
+            $this->pdo->query($query);
+        }, explode("\n\n", file_get_contents(__DIR__ . '/../../../../setup/doctrine.sqlite.sql')));


.. and path fix here as well?

As we can't define constants through the Behat configuration file I decided to simply add a config entry in each suite for the path to the project root. The suites don't inherit configuration from some default place, so the same key => value has to be defined in every suite. For PHPUnit a project root constant has been set in the bootstrap script.

christeredvartsen · 2017-07-28T13:07:04Z

All issues have been fixed in 3e024c0.

With regards to the missing public-keyword from the interfaces, this is basically just something I started doing years back, as all methods in the interfaces are public anyways. I'm not really against having them there, it's just a habit of mine to not include them.

christeredvartsen added 4 commits June 23, 2017 14:31

Enable autoloading of Behat context classes

c913a49

Instead of adding the path of the Behat feature context classes to the Behat configuration file, and manually including the files in the bootstrap for the unit tests the directory is added to autoload-dev in composer.json

Add interface for Behat feature context classes

2493291

The interface must be implemented by all feature context classes so they can provide an instance of a database and storage adapter to be used in the different suites for Behat.

christeredvartsen added Improvement labels Jun 23, 2017

christeredvartsen added this to the Imbo-3.0 milestone Jun 23, 2017

christeredvartsen self-assigned this Jun 23, 2017

christeredvartsen added 20 commits June 24, 2017 10:35

Add a namespace for Behat feature context classes

bc33d4f

This commits adds ImboBehatFeatureContext as a namespace to all classes related to the feature context classes used in Behat tests. Some classes have also been renamed to better suit their purpose.

Add suite for MongoDB and Filesystem adapters

a3ec2c6

Provide sane default values for the Images model

5798ee8

This gives the Imbo\Model\Images model sane deafult values for hits, limit and page. Resolves #553

Add test-suite for Doctrine/SQLite and filesystem

134d8ec

Adds a separate suite for testing the combination of the Imbo\Database\Doctrine database adapter (with a SQLite database) and the Imbo\Storage\Filesystem storage adapter.

Add database setup for Doctrine/SQLite test

648bfdc

The doctrine_sqlite_filesystem suite uses a SQLite database for testing, and this class is responsible for setting up / clearing the database used for testing.

Adjust comparison to work for non-type databases

0db326b

As the Doctrine database does not handle types in the same way as MySQL / MongoDB we need to adjust the tests as the Behat API Extension is type-sensitive on comparisons such as theses.

Order by the update flag for correct results

5300e79

To get the lastest value from the updated column the adapter must add ordering. Resolves #556.

Clean up tests for database adapters

3ff8383

Some missing tests have been added, and a generic clean-up has been done as well, as will be done to all test cases. Add test for the newly added getAllUsers-method.

Update docs to reflect new behaviour of /images

6714116

Remove is_array check

4d4836b

The Request::getUsers() method always returns an array, so no need for the check.

Clean up test case annotations

2c36c30

Fetch all users for auth when query param is empty

3a188dd

When the client does not specify the users-filter when requesting the global images endpoint Imbo should use all users found in the current database to use for authentication. Also added test case to verify new behaviour.

Rename test class

06575bf

Use setup file for SQL

a28cabc

Instead of duplicating the SQL for creating the tables in the database the test classes now use the contents of the file found in the setup dir, which is also included in the documentation.

Supply suite settings for the setUp method

6aeb0fa

Some suites needs extra settings that can be used in the set up, so this commit adds a parameter to the setUp method that holds all settings for the current suite.

Add suite for Doctrine (MySQL) and Filesystem

e6b12bf

This commit adds a Behat suite for testing the Doctrine adapter with a MySQL database in combination with the Filesystem storage adapter. For this to work we need to enable the mysql service in Travis-CI.

Update README to reflect new Behat suites

4562182

matslindh requested changes Jul 28, 2017

View reviewed changes

christeredvartsen merged commit c8f2103 into develop Jul 28, 2017

christeredvartsen deleted the issue/279-test-multiple-adapters branch July 28, 2017 16:35

christeredvartsen mentioned this pull request Jul 28, 2017

Imbo-3: Invalid SQL generated when not specifying users to /images.json (Doctrine) #552

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test multiple database / storage adapters using Behat #550

Test multiple database / storage adapters using Behat #550

christeredvartsen commented Jun 23, 2017

christeredvartsen commented Jun 23, 2017 •

edited

Loading

christeredvartsen commented Jul 28, 2017

matslindh left a comment

matslindh Jul 28, 2017

matslindh Jul 28, 2017

matslindh Jul 28, 2017

matslindh Jul 28, 2017

matslindh Jul 28, 2017

matslindh Jul 28, 2017

matslindh Jul 28, 2017

matslindh Jul 28, 2017

christeredvartsen commented Jul 28, 2017

Test multiple database / storage adapters using Behat #550

Test multiple database / storage adapters using Behat #550

Conversation

christeredvartsen commented Jun 23, 2017

christeredvartsen commented Jun 23, 2017 • edited Loading

christeredvartsen commented Jul 28, 2017

matslindh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

christeredvartsen commented Jul 28, 2017

christeredvartsen commented Jun 23, 2017 •

edited

Loading