Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#12048] Update liquibase configuration #12930

Merged
merged 23 commits into from
Apr 6, 2024
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
21cf9c8
Update gradle config
NicolasCwy Mar 23, 2024
1f64184
Update liquibase config for v9
NicolasCwy Mar 23, 2024
8151bde
Turn off table generate for prod
NicolasCwy Mar 24, 2024
4765e3e
Add back wrongly deleted properties
NicolasCwy Mar 24, 2024
862821f
Merge branch 'master' into liquibase-configuration
NicolasCwy Mar 24, 2024
5a5ffc8
Update of changelog file
NicolasCwy Mar 24, 2024
217e9cf
Fix lint
NicolasCwy Mar 24, 2024
08bc71d
Add configuration for generating changelog
NicolasCwy Mar 26, 2024
5a76894
Add schema migration docs
NicolasCwy Mar 26, 2024
4b86ed4
Merge branch 'master' into liquibase-configuration
NicolasCwy Mar 26, 2024
4c8ac0d
Update process.md
NicolasCwy Mar 26, 2024
951f9a3
Temporary fix to build gradle
NicolasCwy Mar 26, 2024
f5067f3
Add fix to choose liquibase activity
NicolasCwy Mar 26, 2024
14f79f3
Amend documentation for schema migration
NicolasCwy Mar 27, 2024
bd665ad
Merge branch 'master' into liquibase-configuration
NicolasCwy Mar 27, 2024
21d0db5
Add connection instructions to documentation
NicolasCwy Mar 27, 2024
a31ff40
Merge branch 'master' into liquibase-configuration
NicolasCwy Apr 1, 2024
628f274
Revert naming changes and update schema process
NicolasCwy Apr 1, 2024
d952f33
Remove PR naming requirement
NicolasCwy Apr 1, 2024
60946f9
Fix changelog renaming
NicolasCwy Apr 1, 2024
2672b43
Update schema migration to only be done by release leader
NicolasCwy Apr 4, 2024
736670a
Merge branch 'master' into liquibase-configuration
NicolasCwy Apr 4, 2024
78a15df
Add section on gradle activities
NicolasCwy Apr 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,9 @@ dependencies {
exclude group: "org.apache.jmeter", module: "bom"
}

liquibaseRuntime("org.liquibase:liquibase-core:4.19.0")
liquibaseRuntime("info.picocli:picocli:4.7.1")
liquibaseRuntime("org.postgresql:postgresql:42.7.2")
liquibaseRuntime(sourceSets.main.output)
}

Expand Down Expand Up @@ -135,6 +137,10 @@ sourceSets {
}
}

if (!project.hasProperty("runList")) {
project.ext.set("runList", "main")
}

liquibase {
activities {
main {
Expand All @@ -144,7 +150,23 @@ liquibase {
username project.properties['liquibaseUsername']
password project.properties['liquibasePassword']
}
snapshot {
url project.properties['liquibaseDbUrl']
username project.properties['liquibaseUsername']
password project.properties['liquibasePassword']
snapshotFormat "json"
outputFile "liquibase-snapshot.json"
}
diffMain {
searchPath "${projectDir}"
changeLogFile "src/main/resources/db/changelog/db.changelog-new.xml"
referenceUrl project.properties['liquibaseDbUrl']
referenceUsername project.properties['liquibaseUsername']
referencePassword project.properties['liquibasePassword']
url "offline:postgresql?snapshot=liquibase-snapshot.json"
}
}
runList = project.ext.runList
}

tasks.withType(cz.habarta.typescript.generator.gradle.GenerateTask) {
Expand Down
1 change: 1 addition & 0 deletions docs/process.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,7 @@ Make the changes to the code, tests, and documentations as needed by the issue.
npm run lint
```

* **Database schema changes** have a corresponding changelog (see [schema migration](schema-migration.md))
* **All affected tests are passing** on your dev server.<br>
You are more than welcome to also ensure that all tests are passing on your dev server.
* **Staging-tested (if need be)**: If your new code might behave differently on a remote server than how it behaves on the dev server, ensure that the affected tests are passing against the updated app running on your own GAE staging server.
Expand Down
27 changes: 27 additions & 0 deletions docs/schema-migration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
<frontmatter>
title: "Schema Migration"
</frontmatter>

# SQL Schema Migration

Teammates uses _[Liquibase]_(https://docs.liquibase.com/start/home.html), a database schema change management solution that enables developers to revise and release database changes to production. If you were to change the schema of any entity, you would have to create a _Liquibase_ changelog which a maintainer will run to keep the production databases schema in sync with the code.

## Liquibase in Teammates
_Liquibase_ is made available using the [gradle plugin](https://github.com/liquibase/liquibase-gradle-plugin), providing _liquibase_ functions as tasks. Try `gradle tasks | grep "liquibase"` to see all the tasks available. In teammates, change logs (more in the next section) are written in _XML_.

## Change logs, change sets and change types
A _change log_ is a file that contains a series of _change sets_ (analagous to a transaction) which applies _change types_ (actions). You can refer to this page on liquibase on the types of [change types](https://docs.liquibase.com/change-types/home.html) that can be used.

## How to use Liquibase in Teammates
1. Create an _XML_ change log file in `src/main/resources/db/changelog` naming convention is the `db.changelog-YYYY-MM-DD-entity.xml` e.g `db.changelog-2024-03-24-courses.xml`.
2. Add changelog file to be included in the `db.changelog-root` as the last entry

## Generating liquibase change logs
1. Delete `postgres-data` to clear any old database schemas
NicolasCwy marked this conversation as resolved.
Show resolved Hide resolved
2. Run `git checkout master` and
3. run the server using `./gradlew serverRun` to generate tables found on master
NicolasCwy marked this conversation as resolved.
Show resolved Hide resolved
4. Generate snapshot of database by running `./gradlew liquibaseSnapshot -PrunList=snapshot`, the snapshot will be output to `liquibase-snapshot.json`
NicolasCwy marked this conversation as resolved.
Show resolved Hide resolved
4. Checkout your branch and repeat steps 1 and 3 to generate the tables found on your branch
5. Run `./gradlew liquibaseDiffChangeLog -PrunList=diffMain` to generate changeLog to resolve database schema differences
6. Rename this file to the format `db.changelog-YYYY-MM-DD-entity.xml` and add it as a changelog in `db.changelog-root`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's then change this back to follow the release version 🙏
Regarding guide to prof, yes pls add it to teammates-ops. Thanks!


6 changes: 5 additions & 1 deletion src/main/java/teammates/common/util/HibernateUtil.java
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ public static void buildSessionFactory(String dbUrl, String username, String pas
.setProperty("hibernate.connection.username", username)
.setProperty("hibernate.connection.password", password)
.setProperty("hibernate.connection.url", dbUrl)
.setProperty("hibernate.hbm2ddl.auto", "update")
.setProperty("hibernate.hbm2ddl.auto", "validate")
.setProperty("show_sql", "true")
.setProperty("hibernate.current_session_context_class", "thread")
// Uncomment only during migration for optimized batch-insertion, batch-update, and batch-fetch.
Expand All @@ -124,6 +124,10 @@ public static void buildSessionFactory(String dbUrl, String username, String pas
// .setProperty("hibernate.jdbc.fetch_size", "50")
.addPackage("teammates.storage.sqlentity");

if (Config.IS_DEV_SERVER) {
config.setProperty("hibernate.hbm2ddl.auto", "update");
}

for (Class<? extends BaseEntity> cls : ANNOTATED_CLASSES) {
config = config.addAnnotatedClass(cls);
}
Expand Down
Loading
Loading