New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SQlite databases stopped working after migrating #3753
Comments
Can you try adding the same database again with a different name and seeing if that works? Don't delete the old one, as that will delete cards you've created against it, I'm just curious if it's all sqlite databases aren't working or if we somehow got your database entry into an unworkable state somehow... |
Sounds like it's a SQLite issue with the Java temp dir? @gseva is this the first time you've upgraded metabase? |
@salsakran When i'm trying to add a new database, this error appears: @camsaul No, i'm using metabase since version 16 (i guess), upgrading on every major release. |
Hmm... the only thing that seems to have changed is that we updated our sqlite driver from 3.8.11.2 -> 3.14.2.1 |
@salsakran Do i have any easy way of trying the previous version of driver with the last docker image? |
@gseva you can try an older version on Metabase, like 0.19.3, and see if things are working there. That should help us diagnose your issue. Make sure you use a new internal database though because a 0.20.x DB won't work on older versions. |
Download As an aside, can you share any info on how you're running docker? Are you using our dockerfile directly? What's the underlying host? Were there any changes other than upgrading that occurred? |
We're using a docker-compose entrance for metabase that looks like this: metabase_instance:
image: metabase/metabase
ports:
- 3000
container_name: metabase_instance
volumes:
- ./meta_instance:/tmp
environment:
- MB_DB_FILE=/tmp/metabase.db To upgrade, i'm pulling the new image, stopping and lifting up the docker-compose. There we no changes aside from that. |
Ok, so I'm not able to replicate this. I didn't have a sqlite db handy so I pulled down http://www.sqlitetutorial.net/sqlite-sample-database/ I was able to connect to it and run queries without any kind of error on both v0.20.0 and v0.20.3. As per @camsaul's SO link, are you able to see if ./metabase_instance/tmp has NOEXEC on it? |
Seems like it does not:
I reviewed the logs, and this is the first error to appear (if it helps):
|
Also, this seems to be the same problem: http://discourse.metabase.com/t/error-adding-sqlite-database/807 |
Weird. That guy is also running Metabase via Docker. Sounds like this is a Docker + SQLite issue. @salsakran did you try running via Docker or did you just run the JAR directly? |
ran the mac app in my case. |
Hi there I have since migrated everything to Postgres, but here are the details of the sqlite version: docker-compose:
I had a separate container which scraped some data and put it into sqlite databases in the mb/Dockerfile
mb-entrypoint.sh #!/usr/bin/env bash
java -jar /app/metabase.jar migrate release-locks;
echo "db locks released";
./app/run_metabase.sh (p.s. not sure if that db lock release makes sense, but it did the trick for me) I was running on a Ubuntu 16.04 box, with docker version 1.12.2, and docker-compose version 1.8.1 (though it might have been v1.9.0-rc1) You should be able to put that Hope this helps. --p edit: should also note that the latest docker image as 0.20.0 at the time - I may have also tried 0.20.1 as well... |
That's strange. I'll try to replicate this again, but on a docker image. In theory, you could migrate your data to another db, and just manually change the database type. |
That is more or less what I did... the script that I had scraping data and putting it into a sqlite db now puts it into a postgresdb. |
Looks like this is an error with the SQLite library we're using: |
@salsakran did you manage to repro this on Docker? There's a new version of the SQLite library we use available, I think we should check if it's fixed in the new version. If not, we should downgrade back to the old version that was known to work. |
repro'd this on the v0.20.3 published image. |
@gseva SQLite doesn't have proper datetime types, but uses strings instead, so if you kept them as strings when you migrated to Postgres that would explain this behavior. You'll want to use something like |
@camsaul
There is not much in logs as well:
I'm running:
Edit: I tried to recreate metabase docker(openjdk:8-jre + metabase.jar) and it is working just fine |
I also noticed/reverted to the old driver to keep going, can this be addressed again @salsakran? Thanks! |
At any rate the upstream bug was allegedly fixed in March 2016 🤔 |
It looks like they're still having issues with this: |
I'll downgrade us back to the last know working version and add it to our upcoming 0.25.2 patch release |
Re-fixed by #5720 |
Thanks! Working fine now on 0.25.2 👏 |
@camsaul It should be fixed with sqlite-jdbc 3.20.0 (fixed by xerial/sqlite-jdbc/pull/233). There's also fix for #4826. |
@starhel xerial/sqlite-jdbc#225 and xerial/sqlite-jdbc#197 are still open. I'm extremely wary of updating to a new version of the SQLite JDBC driver because doing so has broken SQLite support entirely for people the last couple of times I tried it. Last time the bug was supposedly fixed too |
@camsaul There's probable a need of dev version for tests. Currently I'm using Metabase 0.25.2 with sqlite-jdbc 3.20.0 without any problem, so if you need some tests just tell me what to do :) |
I'm still having this issue with docker, and then think I found an easy way to reproduce.
What happens: Reproduced consistently on AWS on local machine. Works fine on Metabase for Mac, only a problem through Docker. Happening on 0.26.2 |
I have this error too when I add sqlite database : metabase version: v0.26.2 |
+1 |
I'm getting the same error on a docker-installed Metabase version v0.27.1 (055e448 release-0.27.0). Whenever I try to add a SQLite database, the following error is logged:
Note that the last line occurs every time I try to add my database, whereas the first and second only occur once. To me, this suggests something is wrong with loading the library in the first place. @starhel you mentioned you're using Metabase 0.25.2 with sqlite-jdbc 3.20.0. How come I'm on Metabase 0.27.1 with sqlite-3.16.1? Has the driver used by Metabase been downgraded? |
@Simbul I've compiled Metabase on my own with newer sqlite driver. |
+1 same error w/ docker and alpine linux (3.20.1) |
@starhel did the newer SQLite driver fix the issue? |
I've never had problem like this. My database wasn't supported by default driver, so I decided to update. You should probably follow discussion in this thread: xerial/sqlite-jdbc#197 |
After updating to 0.20 my sqlite databases stopped working.
Dashboards show "There was a problem displaying this chart." and questions after a while of "Doing science" return "Your question took too long".
In logs i'm finding a lot of java tracebacks like this one:
The text was updated successfully, but these errors were encountered: