Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CI for synapse_port_db #6140

Merged
merged 21 commits into from Oct 28, 2019

Conversation

@babolivier
Copy link
Member

babolivier commented Oct 1, 2019

Paired with matrix-org/pipelines#9

This PR adds:

  • a test sqlite database
  • a configuration file for the sqlite database
  • a configuration file for a postgresql database (using the credentials in .buildkite/docker-compose.pyXX.pgXX.yaml)

as well as a new script named .buildkite/scripts/test_synapse_port_db.sh that:

  1. installs Synapse
  2. updates the test sqlite database to the latest schema and runs background updates on it
  3. creates an empty postgresql database
  4. run the synapse_port_db script to migrate the test sqlite database to the empty postgresql database (with coverage)

Step 2 is done via a new script located at scripts-dev/update_database.

The test sqlite database is extracted from a SyTest run, so that it can be considered as an actual homeserver's database with actual data in it.

@babolivier babolivier force-pushed the babolivier/port_db_ci branch 3 times, most recently from 6d062d3 to f370c32 Oct 3, 2019
@babolivier babolivier marked this pull request as ready for review Oct 4, 2019
@babolivier babolivier requested a review from matrix-org/synapse-core Oct 4, 2019
@babolivier babolivier force-pushed the babolivier/port_db_ci branch from c191072 to 0b68a54 Oct 7, 2019
@erikjohnston

This comment has been minimized.

Copy link
Member

erikjohnston commented Oct 8, 2019

Can you give us a commit range to review or something please? There are too many commits to figure out whats going on without some help :)

@erikjohnston erikjohnston removed the request for review from matrix-org/synapse-core Oct 8, 2019
@babolivier

This comment has been minimized.

Copy link
Member Author

babolivier commented Oct 8, 2019

Can you give us a commit range to review or something please? There are too many commits to figure out whats going on without some help :)

Right, sorry, I assumed this review would come after the ones on the PRs this one is built on top of. A list of all commits specific to this PR can be seen here, and for maximum clarity this PR only changes the following files:

  • .buildkite/scripts/test_synapse_port_db.sh (creation)
  • .buildkite/postgres-config.yaml (creation)
  • .buildkite/sqlite-config.yaml (creation)
  • .buildkite/test_db.db (creation)
  • scripts-dev/create_postgres_db (creation)
  • scripts-dev/update_database (creation)
  • tox.ini (addition of the synapse_port_db test environment)

None of the other PRs edit these files, so the reviewer can review just the changes to these files, without worrying about whether they've been changed by this PR or another.

@babolivier babolivier requested a review from matrix-org/synapse-core Oct 8, 2019
Copy link
Member

richvdh left a comment

a few comments; it kinda needs #6102 to land before it gets a proper review.

tox.ini Outdated Show resolved Hide resolved
scripts-dev/create_postgres_db Outdated Show resolved Hide resolved
scripts-dev/update_database Show resolved Hide resolved
@babolivier babolivier requested a review from matrix-org/synapse-core Oct 22, 2019
@richvdh

This comment has been minimized.

Copy link
Member

richvdh commented Oct 23, 2019

pending merge of #6102

@richvdh richvdh removed the request for review from matrix-org/synapse-core Oct 23, 2019
babolivier added 13 commits Sep 27, 2019
babolivier added 2 commits Oct 22, 2019
@babolivier babolivier force-pushed the babolivier/port_db_ci branch from d9efe03 to 05463e4 Oct 23, 2019
babolivier added 2 commits Oct 23, 2019
@babolivier babolivier requested a review from matrix-org/synapse-core Oct 23, 2019
@babolivier babolivier added this to In progress in Homeserver Task Board via automation Oct 23, 2019
@babolivier babolivier moved this from In progress to Review in Homeserver Task Board Oct 23, 2019
@babolivier

This comment has been minimized.

Copy link
Member Author

babolivier commented Oct 23, 2019

#6102 has been merged; I've rebased the branch on develop so it doesn't pull any more commit than it should.

changelog.d/6140.bugfix Outdated Show resolved Hide resolved
.buildkite/postgres-config.yaml Show resolved Hide resolved
.buildkite/scripts/test_synapse_port_db.sh Outdated Show resolved Hide resolved
.buildkite/sqlite-config.yaml Show resolved Hide resolved
.buildkite/scripts/test_synapse_port_db.sh Outdated Show resolved Hide resolved
.buildkite/scripts/test_synapse_port_db.sh Outdated Show resolved Hide resolved
scripts-dev/update_database Outdated Show resolved Hide resolved
scripts-dev/update_database Outdated Show resolved Hide resolved
scripts-dev/update_database Outdated Show resolved Hide resolved
scripts-dev/update_database Show resolved Hide resolved
babolivier added 2 commits Oct 24, 2019
@babolivier babolivier requested a review from matrix-org/synapse-core Oct 24, 2019
@richvdh richvdh self-assigned this Oct 24, 2019
.buildkite/postgres-config.yaml Outdated Show resolved Hide resolved
.buildkite/scripts/test_synapse_port_db.sh Outdated Show resolved Hide resolved
@babolivier babolivier requested a review from matrix-org/synapse-core Oct 28, 2019
Copy link
Member

richvdh left a comment

lgtm!

@babolivier babolivier merged commit 14504ad into develop Oct 28, 2019
18 checks passed
18 checks passed
buildkite/synapse Build #5153 passed (20 minutes, 24 seconds)
Details
buildkite/synapse/check-sample-config Passed (1 minute, 37 seconds)
Details
buildkite/synapse/check-style Passed (1 minute, 56 seconds)
Details
buildkite/synapse/isort Passed (39 seconds)
Details
buildkite/synapse/mypy Passed (25 seconds)
Details
buildkite/synapse/newspaper-newsfile Passed (16 seconds)
Details
buildkite/synapse/packaging Passed (21 seconds)
Details
buildkite/synapse/pipeline Passed (10 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-postgres-9-dot-5 Passed (17 minutes, 16 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-sqlite Passed (6 minutes, 50 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-sqlite-slash-old-deps Passed (9 minutes, 11 seconds)
Details
buildkite/synapse/python-3-dot-6-slash-sqlite Passed (6 minutes, 38 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-postgres-11 Passed (17 minutes, 45 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-postgres-9-dot-5 Passed (17 minutes, 30 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-sqlite Passed (6 minutes, 32 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-postgres-9-dot-6-slash-monolith Passed (12 minutes, 51 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-postgres-9-dot-6-slash-workers Passed (14 minutes, 8 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-sqlite-slash-monolith Passed (13 minutes, 41 seconds)
Details
Homeserver Task Board automation moved this from Review to Done Oct 28, 2019
richvdh added a commit to matrix-org/pipelines that referenced this pull request Oct 28, 2019
Paired with matrix-org/synapse#6140

Defines the Synapse CI jobs to run the tox environment added in matrix-org/synapse#6140
@richvdh richvdh mentioned this pull request Oct 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
3 participants
You can’t perform that action at this time.