Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Following install guide: Metabase crash loop, re: database migration lock, recommend starter-plus as min instance size? #2

Open
sherbondy opened this issue Jan 11, 2021 · 1 comment

Comments

@sherbondy
Copy link

Hey folks,

I just tried setting up a Metabase instance alongside a fresh Render Postgres DB instance per the install guide.
(https://render.com/docs/deploy-metabase)

I ran into the following deployment crash loop in the logs, where it seems like the Metabase server was trying to run its initial DB migrations on the first launch and complained it could not because the database was holding onto a migration lock:

Jan 11 03:53:05 PM  2021-01-11 07:53:05,573 INFO db.liquibase :: Checking if Database has unrun migrations...
Jan 11 03:53:16 PM  2021-01-11 07:53:16,268 INFO db.liquibase :: Database has unrun migrations. Waiting for migration lock to be cleared...
Jan 11 03:53:18 PM  2021-01-11 07:53:18,274 WARN metabase.util :: auto-retry metabase.db.liquibase$wait_for_migration_lock_to_be_cleared$fn__20958@4df6d39c: Database has migration lock; cannot run migrations. You can force-release these locks by running `java -jar metabase.jar migrate release-locks`.
Jan 11 03:53:20 PM  2021-01-11 07:53:20,277 WARN metabase.util :: auto-retry metabase.db.liquibase$wait_for_migration_lock_to_be_cleared$fn__20958@4df6d39c: Database has migration lock; cannot run migrations. You can force-release these locks by running `java -jar metabase.jar migrate release-locks`.
Jan 11 03:53:22 PM  2021-01-11 07:53:22,279 WARN metabase.util :: auto-retry metabase.db.liquibase$wait_for_migration_lock_to_be_cleared$fn__20958@4df6d39c: Database has migration lock; cannot run migrations. You can force-release these locks by running `java -jar metabase.jar migrate release-locks`.
Jan 11 03:53:24 PM  2021-01-11 07:53:24,281 WARN metabase.util :: auto-retry metabase.db.liquibase$wait_for_migration_lock_to_be_cleared$fn__20958@4df6d39c: Database has migration lock; cannot run migrations. You can force-release these locks by running `java -jar metabase.jar migrate release-locks`.
Jan 11 03:53:26 PM  2021-01-11 07:53:26,284 WARN metabase.util :: auto-retry metabase.db.liquibase$wait_for_migration_lock_to_be_cleared$fn__20958@4df6d39c: Database has migration lock; cannot run migrations. You can force-release these locks by running `java -jar metabase.jar migrate release-locks`.
Jan 11 03:53:28 PM  2021-01-11 07:53:28,289 ERROR metabase.core :: Metabase Initialization FAILED
Jan 11 03:53:28 PM  liquibase.exception.LockException: Database has migration lock; cannot run migrations. You can force-release these locks by running `java -jar metabase.jar migrate release-locks`.

Has anyone else run into this issue on first deployment?

I was able to jump into the Shell in the Render dashboard for the Metabase service and run the java -jar metabase.jar migrate release-locks command.

But the migration lock seems to be a red herring, and the real underlying issue seems to be that the 512 MB of memory available for starter instances is simply not enough to feed the hungry, hungry JVM. It seems like Metabase crashes due to OOM in the middle of the initial migration process.

Maybe the Deploy guide should mention starter-plus as the minimum advisable plan to deploy a Metabase instance?

@sherbondy
Copy link
Author

Related thread from Metabase issues:
metabase/metabase#8288 (comment)

After upgrading the instance to starter-plus was able to deploy without a hitch after manually releasing the migration locks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant