Data wiped due to database corruption #2463
Comments
oskarkook
changed the title
|
Is it possible that this is an external storage issue? It is not uncommon for SD cards to corrupt. I don't see how updating the app could have removed some of the files while keeping others in place. |
|
Just had the same issue. I'm not using an external storage/SD card and all my podcast files are still in /Android/data/de.danoeh.antennapod/files/ but the app is showing no subscriptions, downloads, or download history. The data is still on my phone but the app isn't recognizing it. Edit: Should have mentioned Android version. I'm using Android 6.0.1 on a Galaxy S5 |
|
Same issue here: all my feeds gone. Using Android 6.0.1 on a Nexus 5. |
|
I'm on a Galaxy 5, Android 6.0.1 |
|
Same problem as thelstew. Went to download some new episodes and lost everything |
|
Oh. I am on a galaxy 5 |
|
This is an additional report of all data being wiped from the application. The phone is running LineageOS 14.1 (Android 7.1.2) on a Galaxy S4 with AntennaPod 1.6.4.1 from F-Droid. The sequence of steps was the following:
The following are the logs from that time. Unfortunately, nothing looks that out of place. |
|
When it happened to me, I was watching it update in the Episodes view. As I was watching, one of the downloads stopped updating, and then all three episodes disappeared. I opened the drawer, and no subscriptions were visible, with the queue empty. |
|
There seem to be more and more users affected by this.
|
|
I am also affected. I did also watch my subscriptions getting updated and suddenly AntennaPod is completely empty - no downloads, no subscriptions - everything gone. App version: 1.6.4.1 I checked Is there anything I can do to help debug this? |
|
Yeah, all my files are there. I don't wanna uninstall and loose them. But..
Yeah. This really blows.
…On Nov 27, 2017 5:27 PM, "basti2342" ***@***.***> wrote:
I am also affected. I did also watch my subscriptions getting updated and
suddenly AntennaPod is completely empty - no downloads, no subscriptions -
everything gone.
App version: 1.6.4.1
Android version: 8.0.0
Device: Nexus 5x
I checked /Android/data/de.danoeh.antennapod/files/ and the files are
still there.
Is there anything I can do to help debug this?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#2463 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AfrKawkpX6cQ7X0InTEUctMjlv3yxoT3ks5s6zdSgaJpZM4QC7Wz>
.
|
|
To be clear @mfietz, I'm on a Nexus 6P with LineageOS. A temporary solution would be to make an OPML export every time subscriptions change. At least that would allow us to restore that much. |
|
I've hit this a second time. Very soon after after hitting the refresh menu option to download updated podcast information the database was wiped. This time I captured a bug report right after the issue, and found the following in the logs around the time of the issue: So, it would appear that SQLite believed the database was corrupt and deleted it as a result. The app's storage was set to an external SD card, which is less than 1 year old. No other app is reporting issues with that storage, but it cannot be ruled out. In response, I've changed the apps storage to internal. |
|
A quick note: in my case the potential database corruption happened on the internal storage of my Nexus 5x. I didn't notice any other storage/file system problems on my device. |
|
I was also running on internal storage. I don't think storage corruption is the root cause here. |
|
moto g 2014 titan lineageos, happened here on external card |
Just to be sure, has anyone hit this who is not running LineageOS? |
|
Yes, @brarcher. I had the issue on stock Android 6.0.1 on a Nexus 5. |
|
I'm also seeing this. It has happened twice. LG v20, Android 7 stock |
|
@ejstacey can you export your subscriptions to OPML and send me the file to martin.fietz 'at' gmail.com ? |
|
I have emailed you an OPML file. Here's some activity/other notes, if it helps: I had it on on the way home (Bluetooth through car) when I got interrupted by a call. Antennapod stopped/started on its own fine. I always have Bluetooth going through the car, but I very very rarely receive a call, so that's out of the ordinary. I don't know if it's related, but I thought I'd mention it just in case it is. I have Update Interval set to every 4 hours. I have Enqueue Downloaded enabled. I have 4 parallel downloads set up. I have Automatic Download enabled, and have it set to Download when not charging. |
|
I am also affected. CM13 / 6.0.1 OnePlus One, so internal storage only. Version 1.6.4.1 |
|
Galaxy S6 running Android 7.0 - something happened while updating and all the subscriptions, history, queue, etc. vanished. I am guessing it just got corrupted and that I should make it a point to regularly export an OPML file with my subscriptions. I am looking through the above comments which are actually helpful since I don't mind resubscribing but will need to look for the saved data manually wherever it is stored and it will probably take me a month to figure out everything I was subscribed too - maybe there is some limit on how much data the app can track? |
|
I just hit the issue again, but I'm not sure there was a subscription update going on. I opened the app, and a podcast I had been listening to earlier only had a minute left when I had paused it. I have the 'delete if stopped with only a few minutes left' option turned on, fwiw. Anyway, I went to the queue and long pressed on a different episode and said 'remove from queue'. Nothing disappeared (lock on sqlite database between the autodeleter and my action? No idea!). I then long pressed on a different podcast (the one that only had a minute left) and said 'remove from queue'. Then everything disappeared (queue, subscriptions, etc). |
|
oh noes, same here: I dont think feeds wee updating when everything crashed and deleted feeds, watched-status and so on. |
|
Just hit it again. I was partway through two podcasts. Podcast B I last listened to a couple hours ago when I paused it. Podcast A I last listened to a few days ago. I opened the app, which I think did a refresh operation. I hit podcast A while that was happening. It didn't play anything, just nothing. I hit podcast B. Nothing. I hit the hamburger icon to open the menu and everything disappeared. While it was in the broken/frozen state, I could scroll up+down through my queue. |
|
I'm also affected by this whole "everything vanished" thing. :( Google Pixel, stock Android version 8.1.0 Opened a subscription to download further episodes, the list didn't load and suddenly everything was gone! Was anyone of you guys able to restore their previous data? I welcome even the whackiest solution! |
|
The only restoring which I attempted was importing a previously exported OPLM file which backs up all the podcast subscriptions. Any previously downloaded podcasts might still be there, but they may not be usable by the app. |
|
Since I don't think there's a report for 2.1.2 and just to add more data, my database corrupted, I think while adding podcasts. I only noticed at ~20:45:03.522 but the error log states this: Ironically the two podcasts I added are now showing up in AntennaPod, but all my other podcasts are gone. For database size, I believe I had over 100 and I had ~450 episodes actively in my queue. I've had this many podcasts for multiple months and never run into an issue. Podcasts sorted by publication date. I also subscribed to some Youtube feeds, though only about 4. Nothing in my behaviour has changed for months regarding my usage. The podcast I was playing remained in the "notification" but nothing happened when I pressed play (I think this is where the PlayableUtils in the full-logs.txtcome from). Environment: Files stored in /sdcard/Android/data/de.danoeh.antennapod/ (internal storage). There's a 'CorruptedDatabaseBackup.db' created at 18:54 but its file size is 0. OPML export of my list only outputs the two 'new' podcasts. An export of the db also only includes those two. Just to be complete, in files/cache, I have some 'feed' files with no file ending (ie, feed-World Ocean-Radio-4.) which seem kind of wacky, given that feed-World Ocean-Radio-n exists for 1-33 with each number going up in creation time, from June to November last year. Attempting to OPML import shows nothing in them. My /data/user/0/de.danoeh.antennapod/databases only has the new .db file with the two podcasts. |
|
We disabled WriteAheadLogging (WAL) for the database at some point because it breaks database exports. Maybe we should write another way to export the database (using proper sqlite to copy the database instead of just copying the file) and re-enable WAL. If anyone is up for rewriting the backup feature, that would be greatly appreciated :) |
|
Hmm. Maybe the missing synchronization here actually is the problem? |
|
This issue has been mentioned on AntennaPod Forum. There might be relevant details there: https://forum.antennapod.org/t/i-lost-my-data-and-i-no-have-idea-about-i-lost-this/1009/2 |
|
This issue has been mentioned on AntennaPod Forum. There might be relevant details there: https://forum.antennapod.org/t/i-lost-my-data-and-i-no-have-idea-about-i-lost-this/1009/4 |
|
This happened to me today with 2.4.1—I started the app but everything in there is gone, as if the db was wiped. My 27GB of downloaded podcasts are still on the filesystem though. The full-log.txt on the bug report screen has no history before the wipe. The database and all app data was stored in the default location on internal storage. I don't have an SD card installed. Screenshots: Android version: 10 |
|
This issue has been mentioned on AntennaPod Forum. There might be relevant details there: https://forum.antennapod.org/t/all-gone-podcasts-and-episodes-as-if-i-just-installed-the-app/1593/2 |
|
@ByteHamster Should we add more custom bug reporting logic here when we detect a corrupt db when there is a corrupt database and ask users to share some data with us and directly into this specific ticket? |
|
And/or a link to instructions on the website on how to recover the data (there are some options/instructions both in this issue and on the forum). |
I don't know what data they could share that would help. I'm pretty much out of ideas here. A rather similar issue seems to happen in K-9 Mail. Maybe corrupted databases are just a thing that occasionally happens when running an app on thousands of devices? |
|
I had this issue occur a few times in a fairly short space of time (this was at least 2 years ago) but then haven't had the issue since (still on the same phone after about 4 years). I was once active in this thread (I was running a suspect version for some time) but thought I'd share that it does just seem random. I then have been updating to the latest version without issue. The backup feature is handy and I do that occasionally in case it strikes again. |
|
I have had this often, too, but not anymore since I switched to a phone without SD. Could have been some version that fixed it, too of course. One thing that could help users a bit is automatic backups, weekly for example. #4850 |
|
Hi, My CorruptedDatabaseBackup.db starts with Not sure what I can do from here. |
|
Looks like this issue is solved, meaning that no new databases are getting corrupted since about 4 years. Because of the past corruption, there might be later effects, like the database breaking years later on some unrelated operation. For this, we have issue #6212 to add a better error message and explain users how they can recover the database. Unfortunately, repairing (or even detecting) the quirks on a phone does not work on most Android versions. So we cannot really do more about this here in this issue. |
I dare say this is incorrect. The problem just happened to me. I started using the app in 2022. One funny thing: Even though the app itself has no content whatsoever, the widget (don't know whether that is the correct word) on top of the Android notifications still shows me the banner and title of the last episode I was listening to. If I touch it, the empty queue opens. One other thing to notice: I previously removed some elements from the sidebar in the app settings. Those settings are still in place. Are they not stored in the same db? I'm on a Galaxy S10 with LineageOS 20 (Android 13) and recently upgraded the app to 3.1.0. I guess I'll try to find a way to fix the db in the forum. |
|
@graue70 Thanks for the report. Guess we'd better keep making periodic backups of our databases. |
|
Sorry to hear @graue70. Please have a look here if you haven't already: https://antennapod.org/documentation/bugs-first-aid/database-error You might still be able to recover your data.
That's correct. See also #4089. |
I took a look at the CorruptedDatabaseBackup.db file with a hex editor. The first line in the file is fine so the first quoted suggestion does not apply. The second suggestion is confusing because my file has the 'db' extension, not 'sqlite'. Also, I did not find the "begin transaction" and "rollback" words in the file. In any case, the resulting file can be imported in the app but leads to a crash on the first start. (The error was 'table Feeds already exists'.) Funny thing: I can import the unchanged CorruptedDatabaseBackup.db file in the app and temporarily continue using it. It only seems to crash a little later after I update the feeds. I'm prepared to create a new db, fill it manually through the app and lose my history data in the process unless somebody here has some other advice for me? |
|
You can try something like this, which automates what is described here: #!/bin/python
import subprocess
import json
import shutil
corruptedPath = input("Enter file path to your corrupted AntennaPod database: ")
corruptedVersion = subprocess.run(["sqlite3", corruptedPath, "PRAGMA user_version;"], capture_output=True, text=True).stdout.strip()
if corruptedVersion == "0":
print("Error: File not found, not a database, or too corrupted for this script.")
exit()
print("Corrupted file version: " + corruptedVersion)
emptyPath = input("Enter file path to an EMPTY AntennaPod database with the same version. If needed, you can download old app versions on F-Droid and export an empty database: ")
emptyVersion = subprocess.run(["sqlite3", emptyPath, "PRAGMA user_version;"], capture_output=True, text=True).stdout.strip()
print("Empty file version: " + emptyVersion)
if corruptedVersion != emptyVersion:
print("Error: Version does not match")
exit()
outputPath = corruptedPath + "-repaired.db"
shutil.copyfile(emptyPath, outputPath, follow_symlinks=True)
def query(db, query):
result = ""
try:
result = subprocess.run(["sqlite3", "-json", db, query], capture_output=True, check=True, text=True).stdout
return json.loads(result)
except subprocess.CalledProcessError as err:
print(err.stderr)
exit()
except json.decoder.JSONDecodeError as err:
return result
tables = query(emptyPath, "SELECT name FROM sqlite_schema WHERE type ='table';")
for table in tables:
table = table["name"]
print("Copying " + table + "...")
columns = query(emptyPath, "SELECT GROUP_CONCAT(NAME,',') AS columns FROM PRAGMA_TABLE_INFO('" + table + "')")[0]["columns"]
query(outputPath, "DELETE FROM " + table)
query(outputPath, "attach '" + corruptedPath + "' AS old; INSERT INTO main." + table + " SELECT * from old." + table + ";")
print("Done. Output file: " + outputPath) |
|
Thanks for the hint. Running your script yields this: Running Details
Next I tried the solution from https://support.storj.io/hc/en-us/articles/360029309111-How-to-fix-a-database-disk-image-is-malformed- which is basically what you wrote before. It again led to a file that I could import, but the app crashes with this error on the first start: |
Try running that file through my script |
By the way: The file I get from the dump has 5.4 MB while the CorruptedDatabaseBackup file has 17,9 MB. |
|
Could you send me your CorruptedDatabaseBackup via email, please? My address is |
|
I added the steps described in #2463 (comment) to my script and tested it with your database. Seems to work. See https://github.com/ByteHamster/AntennaPodDbFixer for the updated script |
|
This issue has been mentioned on AntennaPod Forum. There might be relevant details there: https://forum.antennapod.org/t/clarification-on-wiped-database-post/5089/1 |
and others
App version: 1.6.4.1 (commit cef7006) (from Google Play)
Android version: 7.0
Current behaviour: Have been using AntennaPod for 1-2 months. I moved my data from Internal Storage to SD card ~1 week ago. Had subscription updates on (every 12hrs I think). It started automatically, I opened the app to check on it, it froze for 1-2 seconds and then most of my data was wiped (app has no subscriptions, playback history empty etc).
/Android/data/de.danoeh.antennapod/files/contains the following:cache/has 2 files of 2 subscriptions. I had 25 subscriptions.import/emptymedia/directories of 6 subscriptions. Still contains some pre-downloaded episodes.First occured: 22.10.2017
Steps to reproduce: Not sure
I still have the old data (1 week old) on my Internal Storage. The
cache/folder there seems to at least contain all my old feeds. Any way I can recover something from there?The text was updated successfully, but these errors were encountered: