Skip to content

Database migration#739

Merged
chreman merged 92 commits intodevfrom
database-migration
Sep 29, 2023
Merged

Database migration#739
chreman merged 92 commits intodevfrom
database-migration

Conversation

@chreman
Copy link
Copy Markdown
Member

@chreman chreman commented Sep 29, 2023

What's new

  • Persistence.php is now an interface
  • Changes to SQLitePersistence so that it can function as an interface to on-disk SQLite database
  • Addition of PostgresPersistence.PHP which is an interface for the persistence API
  • Introduction of DispatchingPersistence which is a wrapper for SQLite and PostgresPersistence and mediates which one is used
  • Introduction of shift_read_percentage parameter in config_local.ini which controls percentage of reads that are directed to new postgres database
  • Changes to PHP endpoints to utilize DispatchingPersistence according to the shift_read_percentage parameter
  • Changes in Python API in persistence.py to allow for multiple hosts in a primary/standby configuration
  • Updates and cleanup in Python API to align with data structures and expected returns in the PHP endpoints
  • Addition of PHP tests and a dockerised end-to-end test

chreman and others added 30 commits June 29, 2023 12:49
@chreman
php dev env initial commit
f8fed34
@chreman
README added
21d714c
@chreman
basic test demo and config
6a8967a
add first test for db connection
3d9f05e
@chreman
add linting for php docker env
bf1c7d4
@chreman
expand linting
b477fda
tests in process, optimize db connection
66f3bfa
tests in process, optimize db connection
c7c99b6
tests in process, optimize db writing
16d8438
tests in process, optimize db writing
3fc3896
tests in process
dc9c98e
tests in process
751b535
tests in process, optimize db writing
2cecbc1
Attempt to make an inital test that passes
3805a0c 
This commit contains one test that covers a few of the different
methods of the SQLite persistence class.

There are quite a few comments; not all tests need to be this verbose
@vrednyydragon
Merge pull request #726 from OpenKnowledgeMaps/ta/db-test-example
7400ed3 
Attempt to make an inital test that passes
integration test updates
363c571
updates asserts
fba8f4e
@chreman
end2end test first wip
2fa6076
@chreman
legacy code cleanup
e5d4576
@chreman
tests now with test app fixture
e377338
@chreman
end2end test wip
a25b743
@chreman
test updates
4095550
@chreman
re-add autoloader
e9d17a3
@chreman
end-2-end test README update
c329dfd
@chreman
test structure cleanup
1b19b3f
@chreman
test structure cleanup
d16c09c
@chreman
test structure cleanup
c5d0ab5
@chreman
dev env cleanup and gitignore update
e5b4484
@chreman
addition of postgres API and postgres to test containers
0151f4e
@chreman
postgres test integration
11f906a
Alexandra Shubenko and others added 27 commits September 1, 2023 15:29
refactor $shiftReadPercentage
80f3be8
@chreman
update
f507800
@chreman
Merge remote-tracking branch 'upstream/database-migration' into datab…
d09ceee 
…ase-migration
@chreman
backend endpoint cleanup and stricter tests
a259b00
@chreman
Merge remote-tracking branch 'upstream/master' into database-migration
3e2a1a7
@chreman
now working again with legacy sqlite persistence
95eecbb
@chreman
now working again with legacy sqlite persistence
fb1bc51
@chreman
cleanup
9657674
@chreman
bugfix service endpoint integration
732356f
@chreman
end2endtest
6b1f773
@chreman
end2endtest
6fd543e
@chreman
end2endtest
02736a0
@chreman
postgres connection updates
bba5cfb
@chreman
hardcode host, unfortunately
3d71d8a
@chreman
not good
16c9ac8
@chreman
maybe better
8dcd3fd
@chreman
more networking issues
3c4071d
@chreman
remove viper-only database column
55be3d9
@chreman
idle transactions bugfix
1a9af06
@chreman
idle transactions bugfix
77d130d
@chreman
reduce size of connection pool per gunicorn worker
eeaf7c8
@chreman
cleanup
97eb447
@chreman
config updates
04beb0d
@chreman
config updates
f1a0e10
@chreman
config updates
75c9dc1
@chreman
config updates
71e8bf9
@chreman
config updates
5f6a52f
@chreman chreman changed the base branch from master to dev September 29, 2023 17:04
@chreman chreman merged commit a516b1b into dev Sep 29, 2023
@chreman chreman deleted the database-migration branch March 15, 2024 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@chreman @vrednyydragon