Attempt to implement an in-memory store using SQLite #147

k00ni · 2020-12-22T13:19:16Z

What?

This PR contains code for an in-memory store which uses PDO + SQLite (:memory:). Because it is based on existing classes it provides the same functionality as its MySQL based pendant.

I also extended PHP-CS-Fixer coverage and improved code readability overall.

Details

PDOAdapter class got another check to make sure it is only used with mysql protocol.
Adapted classes like ARC2_Store to distinguish between MySQL and SQLite approaches
ARC2_StoreLoadQueryHandler::checkSQLBuffers: removed fourth parameter, because it is unused inside the function
ARC2_StoreSelectQueryHandler::getTempTableDef: renamed it to getTempTableDefForMySQL and added a second version for SQLite (called getTempTableDefForSQLite)
reworked Github Actions to test the library (focused on simplifications and clear code)
fixed coding styles in further files to improve code readability
removed further instances of the error control operator @, e.g. @include_once (suppressing errors is bad coding style because you might not see when something went wrong)
improved README.md (shows test cases, added in-memory example)
added test coverage for MySQL 5.5

Ref: #105

…+ coveralls + further refinements

* prepered basic classes * copied ARC2_StoreTableManager to provide create table functions for SQLite instead

+ moved Adapter tests to tests/integration + removed further @'s before functions

because it fails local but not in CI

only failures left

to see why SQLite tests are failing

…ed values but not both

to avoid index access problems

ref: https://github.com/semsol/arc2/pull/147/checks?check_run_id=1618107109#step:5:24

the problem is that I can't reproduce the error locally.

problem was, it missed table prefix in some cases

k00ni · 2020-12-28T17:51:44Z

We have an in-memory store now! 🚀

Added a small section to README.md about the new store: https://github.com/semsol/arc2/tree/pr/sqlite-in-memory-store#in-memory-store-sqlite-based

@semsol (or should I use @bnowack from now on?): What do you think? I know there are many changes, but if you could take a look at SQLite related changes in store/. My approach was to keep what you've built and add some if-clauses, in case SQL differs between MySQL and SQLite. Thanks in advance.

k00ni · 2020-12-28T19:13:51Z

@zozlak: What do you think about a standalone library providing an in-memory triple store? It could be based on this code here, using SQLite with :memory:.

zozlak · 2020-12-29T16:48:00Z

What do you think about a standalone library providing an in-memory triple store? It could be based on this code here, using SQLite with :memory:.

I think there are many things to discuss here but I would be glad to talk with you about it. I propose you contact me by email not to spam here too much.

bnowack · 2020-12-29T17:47:42Z

Wow! how cool is that?

k00ni added 3 commits December 22, 2020 13:38

php-cs-fixer: added further files

cb7c684

updated README to show Github action status badges instead of travis …

0df42e2

…+ coveralls + further refinements

further coding style issue fixes in tests/

14ff5a7

k00ni added enhancement store Related to the RDF store. labels Dec 22, 2020

k00ni self-assigned this Dec 22, 2020

k00ni added 24 commits December 22, 2020 18:47

first step towards SQLite support

6a87dab

* prepered basic classes * copied ARC2_StoreTableManager to provide create table functions for SQLite instead

further preparations to allow SQLite; many refinements

b138656

+ moved Adapter tests to tests/integration + removed further @'s before functions

github actions: added dedicated SQLite case

8310992

removed StoreTest::testProcessTables

f88094d

because it fails local but not in CI

added further checks before run mysql related tests

3a43011

removed Makefile and simplified phpunit.xml

87ed97c

SQLite compatibility: further improvements

44f726f

fixed/adapted further tests

014a8b1

only failures left

added a var_dump to ARC2_ClassTest

534cfe5

to see why SQLite tests are failing

tests/bootstrap.php: fixed mysqli-if clause

4a736d3

reworked tests/bootstrap.php to either custom config or only predefin…

84be5e2

…ed values but not both

tests/bootstrap.php: check $_SERVER vor env vars too

71f85ea

github actions: reworked test approach

5345c41

github actions: fixed wrong php-reference

12d9c70

SQLite related refinements in bootstrap.php

86a4b6c

tests/bootstrap: fixed outdated variable

2c48d82

tests/bootstrap.php: fixed misspelling in $dbConfig['db_pdo_protocol']

375892e

further fixes in test environment

9e7b22d

ARC2_ClassTest: removed var_dump

48364e8

re-added remaining MariaDB test cases (10.2-10.5)

2c3ec35

renamed PDOSQLite to PDOSQLiteAdapter; enabled further tests

a37a43e

updated README.md, added MySQL 5.6 to tests

fa0496c

added var_dump to bootstrap.php again

5918b09

CachedPDOAdapterTest added check for dbConfig['db_pdo_protocol']

b8c52e1

to avoid index access problems

k00ni added 7 commits December 24, 2020 15:25

another small fix for CachedPDOAdapterTest

17b7a5d

force PDOSQLiteAdapterTest to use SQLite config

18c0e56

fixed array access problem in PDOSQLiteAdapterTest

2447a28

for mysqli: fixed PDOSQLiteAdapterTest configuration usage

fb81ea8

fixed further SQLite related test problems

3ebe429

SQLite: fixed relational expression SQL

714c20a

made SelectQueryTest::testSelectLeftJoinUsingOptional more clear

c70a415

k00ni marked this pull request as ready for review December 28, 2020 16:24

k00ni added 12 commits December 28, 2020 17:33

added var_dump to find out why testReplace is failing

79f371c

ref: https://github.com/semsol/arc2/pull/147/checks?check_run_id=1618107109#step:5:24

added further echos to debug

7f5df79

removed all workflow files except for SQLite to get faster test results

444084e

the problem is that I can't reproduce the error locally.

made ARC2_StoreTest::testSetup to do an assertion

c9ceb68

hopefully fixed ARC2_StoreTest::getGraphs

5c6fa11

problem was, it missed table prefix in some cases

re-added ALL workflow files; enabled all SQLite tests

089badd

removed debug output

cad7e29

fixed mysql 5.6 workflow (used 5.7 too)

b31bbf1

added mysql 5.5 workflow

52b5c7b

README.md: updated status for mysql 5.5

ef5f3dc

README.md: added MySQL 5.5 test status badge

5879580

README.md: added section for new in-memory store

c8df43d

k00ni requested review from bnowack and semsol December 28, 2020 19:24

k00ni merged commit 586f303 into master Jan 8, 2021

k00ni deleted the pr/sqlite-in-memory-store branch January 8, 2021 07:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Attempt to implement an in-memory store using SQLite #147

Attempt to implement an in-memory store using SQLite #147

k00ni commented Dec 22, 2020 •

edited

k00ni commented Dec 28, 2020 •

edited

k00ni commented Dec 28, 2020

zozlak commented Dec 29, 2020

bnowack commented Dec 29, 2020

Attempt to implement an in-memory store using SQLite #147

Attempt to implement an in-memory store using SQLite #147

Conversation

k00ni commented Dec 22, 2020 • edited

What?

Details

k00ni commented Dec 28, 2020 • edited

k00ni commented Dec 28, 2020

zozlak commented Dec 29, 2020

bnowack commented Dec 29, 2020

k00ni commented Dec 22, 2020 •

edited

k00ni commented Dec 28, 2020 •

edited