Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handling Device Low Disk Storage #435

Closed
InI4 opened this issue Oct 6, 2016 · 10 comments
Closed

Handling Device Low Disk Storage #435

InI4 opened this issue Oct 6, 2016 · 10 comments

Comments

@InI4
Copy link

@InI4 InI4 commented Oct 6, 2016

This is actually kind of a feature request. I am observing below exception on an emulator, but it could well be on a real world device and I wonder, if osmdroid could do some checking of free memory or at least cleanup, when below the exception occurs:

10-06 04:27:01.710 7978-9239/de.spieleck.app.badgers.debug E/SQLiteDatabase: Error inserting expires=1475812252000 key=47358738 tile=[B@27aea82 provider=Mapnik android.database.sqlite.SQLiteFullException: database or disk is full (code 13) at android.database.sqlite.SQLiteConnection.nativeExecuteForLastInsertedRowId(Native Method) at android.database.sqlite.SQLiteConnection.executeForLastInsertedRowId(SQLiteConnection.java:780) at android.database.sqlite.SQLiteSession.executeForLastInsertedRowId(SQLiteSession.java:788) at android.database.sqlite.SQLiteStatement.executeInsert(SQLiteStatement.java:86) at android.database.sqlite.SQLiteDatabase.insertWithOnConflict(SQLiteDatabase.java:1471) at android.database.sqlite.SQLiteDatabase.insert(SQLiteDatabase.java:1341) at org.osmdroid.tileprovider.modules.SqlTileWriter.saveFile(SqlTileWriter.java:133) at org.osmdroid.tileprovider.modules.MapTileDownloader$TileLoader.loadTile(MapTileDownloader.java:230) at org.osmdroid.tileprovider.modules.MapTileModuleProviderBase$TileLoader.run(MapTileModuleProviderBase.java:296) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1113) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:588) at java.lang.Thread.run(Thread.java:818)

@spyhunter99
Copy link
Collaborator

@spyhunter99 spyhunter99 commented Oct 18, 2016

#448 related issue

and roger this. Great idea. What should the expected behavior be then?

Maybe on start up, set the cache max size to the current size + free disk space. This should force the trim mechanism however this is only triggered on application startup. As such, we should probably do some trickery to have the vacuum functions run periodically. It should be quick enough to attach to every tile insert event.

@spyhunter99 spyhunter99 changed the title Handling Device Low Storage Memory Handling Device Low Disk Storage Oct 22, 2016
spyhunter99 added a commit that referenced this issue Oct 23, 2016
…le app + the library not being disk space aware. Also fixes a bug in the sqlitetilewriter's clean up mechanism for trimming. Preferences for developers to turn on and off the debug settings without recompiling the sample app. User selection of cache location and how to work with reconfiguring the cache settings. This kind of highlights that we need to rework osmdroid to have some kind of configuration class to manage all the settings, storage locations and to bounce sqlite writers and more
spyhunter99 added a commit that referenced this issue Oct 23, 2016
…le app + the library not being disk space aware. Also fixes a bug in the sqlitetilewriter's clean up mechanism for trimming. Preferences for developers to turn on and off the debug settings without recompiling the sample app. User selection of cache location and how to work with reconfiguring the cache settings. This kind of highlights that we need to rework osmdroid to have some kind of configuration class to manage all the settings, storage locations and to bounce sqlite writers and more
@spyhunter99 spyhunter99 added this to the v5.6 milestone Oct 23, 2016
@spyhunter99
Copy link
Collaborator

@spyhunter99 spyhunter99 commented Oct 23, 2016

@InI4 might want to look at these changes. could always use another set of eyes

@InI4
Copy link
Author

@InI4 InI4 commented Oct 23, 2016

I had a glance at the code and it seems, that it is trying to do some async database cleaning on startup and on exception?! Vacuum still commented out. But looks like a step in the right direction.

I need to merge this into my (local) branch for unclear fixes of #411 and then build it into my product. I will keep you posted.

@InI4
Copy link
Author

@InI4 InI4 commented Oct 23, 2016

"That" emulator stays calm now. Can it be, that some memory to database code was also helping, tile files are gone now.

@spyhunter99
Copy link
Collaborator

@spyhunter99 spyhunter99 commented Oct 23, 2016

The vacuum command apparently isn;t supported by sqlite. But basically on start up, it will purge expired tiles, which may or may not be desirable. The old tile system writer did the same task on startup so i just mirrored the logic. Now i'm thinking the logic may be wrong.

On low disk space it will trim, also on every insert, it will check the database size and trim if needed. I also added a bit of code to auto resize the max database size based on the available free space on the storage media.

spyhunter99 added a commit that referenced this issue Oct 23, 2016
#455 adding extra checks to improve performance
spyhunter99 added a commit that referenced this issue Oct 23, 2016
# Conflicts:
#	OpenStreetMapViewer/src/main/AndroidManifest.xml
#	OpenStreetMapViewer/src/main/java/org/osmdroid/MainActivity.java
#	OpenStreetMapViewer/src/main/java/org/osmdroid/samplefragments/SampleFollowMe.java
spyhunter99 added a commit that referenced this issue Oct 23, 2016
@spyhunter99
Copy link
Collaborator

@spyhunter99 spyhunter99 commented Oct 23, 2016

merged

@spyhunter99 spyhunter99 reopened this Oct 23, 2016
@spyhunter99
Copy link
Collaborator

@spyhunter99 spyhunter99 commented Oct 23, 2016

and my fix causing all kinds of issues, disabling the auto trimming for now

spyhunter99 added a commit that referenced this issue Oct 23, 2016
…ing a looping condition.
@InI4
Copy link
Author

@InI4 InI4 commented Oct 29, 2016

Spontaneous question: Does a trim without vaccum shrink the database file size?

@spyhunter99
Copy link
Collaborator

@spyhunter99 spyhunter99 commented Oct 29, 2016

From what I've seen, yes. But it may vary by android version

On Oct 29, 2016 6:04 AM, "InI4" notifications@github.com wrote:

Spontaneous question: Does a trim without vaccum shrink the database file
size?


You are receiving this because you modified the open/close state.

Reply to this email directly, view it on GitHub
#435 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AB4kGWYw4wK6TQEui5CeqlKi4t1tz6A7ks5q4xoIgaJpZM4KPiZO
.

@spyhunter99
Copy link
Collaborator

@spyhunter99 spyhunter99 commented Oct 29, 2016

Actually this ticket needs more work. App procure storage isn't working

spyhunter99 added a commit that referenced this issue Oct 29, 2016
…there's no external storage available and when there storage permission is denied. woohoo!. Also added some hacks for androidx86 functionality and examples for keyboard/mouse wheel events.

#236 zoom to bounds example
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants