A data abstraction layer in PHP for access read and write Relational Databases, No Sql, Text, Xml, Json, SparQl and others
Clone or download
Latest commit 4116174 Jul 1, 2018



SensioLabsInsight Scrutinizer Code Quality Build Status Code Coverage


A data abstraction layer in PHP to manipulate any set of data with a standardized interface from any data source.


Read an write* with a number of data sources accessed by a standardized interface (see more here):

  • Array
  • Relational Databases (based on PDO - Sqlite, MySql, Postgres, Sql Server, Oracle, and others)
  • DBLib (SQL Server php native)
  • OCI8 (Oracle php native interface)
  • Text files (fixed and delimeted like CSV)
  • Json documents
  • Xml documents
  • Sockets
  • MongoDB
  • Amazon Aws S3
  • SparQL


Querying Datasets (Text, Xml, Json, Sockets, Array, etc)

The easiest way to work is to get an repository and get an iterator for navigate throught the data.

$repository = new \ByJG\AnyDataset\Dataset\TextFileDataset(
    ['field1', 'field2', 'field3'],
$iterator = $repository->getIterator();

// and then:
foreach ($iterator as $row) {
    echo $row->get('field1');  // or $row->toArray();

// Or 

Querying Relational Databases

$dbDriver = \ByJG\AnyDataset\Factory::getDbRelationalInstance('mysql://username:password@host/database');
$iterator = $dbDriver->getIterator('select * from table where field = :param', ['param' => 'value']);

Cache results

You can easily cache your results with the DbCached class; You need to add to your project an implementation of PSR-6. We suggested you add "byjg/cache".

$dbDriver = \ByJG\AnyDataset\Factory::getDbRelationalInstance('mysql://username:password@host/database');
$dbCached = new \ByJG\AnyDataset\Store\DbCached($dbDriver, $psrCacheEngine, 30);

// Use the DbCached instance instead the DbDriver
$iterator = $dbCached->getIterator('select * from table where field = :param', ['param' => 'value']);

Connection based on URI

The connection string for databases is based on URL.

See below the current implemented drivers:

Database Connection String Factory
Sqlite sqlite:///path/to/file getDbRelationalInstance()
MySql/MariaDb mysql://username:password@hostname:port/database getDbRelationalInstance()
Postgres psql://username:password@hostname:port/database getDbRelationalInstance()
Sql Server dblib://username:password@hostname:port/database getDbRelationalInstance()
Oracle (OCI) oci://username:password@hostname:port/database getDbRelationalInstance()
Oracle (OCI8) oci8://username:password@hostname:port/database getDbRelationalInstance()
MongoDB mongodb://username:passwortd@host:port/database getNoSqlInstance()
Amazon S3 s3://key:secret@region/bucket getKeyValueInstance()

Querying Non-Relational Databases

// Get a document
$dbDriver = \ByJG\AnyDataset\Factory::getNoSqlInstance('mongodb://host');
$document = $dbDriver->getDocumentById('iddcouemnt');

// Update some fields in there
$data = $document->getDocument();
$data['some_field'] = 'some_value';

// Save the document

Querying Key/Value Databases

// Get a document
$dbDriver = \ByJG\AnyDataset\Factory::getKeyValueInstance('s3://awsid:secret@region');
$file = $dbDriver->get('key');

// Save the document
$dbDriver->put('key', file_get_contents('/path/to/file'));

// Delete the document

Load balance and connection pooling

The API have support for connection load balancing, connection pooling and persistent connection.

There is the Route class an DbDriverInterface implementation with route capabilities. Basically you have to define the routes and the system will choose the proper DbDriver based on your route definition.


$dbDriver = new \ByJG\AnyDataset\Store\Route();

// Define the available connections (even different databases)
    ->addDbDriverInterface('route1', 'sqlite:///tmp/a.db')
    ->addDbDriverInterface('route2', 'sqlite:///tmp/b.db')
    ->addDbDriverInterface('route3', 'sqlite:///tmp/c.db')

// Define the route
    ->addRouteForRead('route2', 'mytable')

// Query the database
$iterator = $dbDriver->getIterator('select * from mytable'); // Will select route2
$iterator = $dbDriver->getIterator('select * from othertable'); // Will select route3
$dbDriver->execute('insert into table (a) values (1)'); // Will select route1;

And more

And more...


Just type: composer require "byjg/anydataset=3.0.*"

Running Unit tests

Running the Unit tests


Running database tests

Run integration tests require you to have the databases up e run with the follow configuration

The easiest way to run the tests is:

Prepare the environment

npm i
node_modules/.bin/usdocker --refresh
node_modules/.bin/usdocker -v --no-link mssql up
node_modules/.bin/usdocker -v --no-link mysql up
node_modules/.bin/usdocker -v --no-link postgres up
node_modules/.bin/usdocker -v --no-link mongodb up

Run the tests

phpunit testsdb/PdoMySqlTest.php 
phpunit testsdb/PdoSqliteTest.php 
phpunit testsdb/PdoPostgresTest.php 
phpunit testsdb/PdoDblibTest.php 
phpunit testsdb/MongoDbDriverTest.php 

Optionally you can set the password for Mysql and PostgresSQL

export MYSQL_PASSWORD=newpassword    # use '.' if want have a null password
export PSQL_PASSWORD=newpassword     # use '.' if want have a null password

Open source ByJG