Program Database & Archive Library #1542
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This is pretty much just the database2 branch stuff without the zscript parsing stuff. That will come later, first I'd like to implement more of the library stuff like archive/entry metadata, etc.
…ArchiveEntry data Using xxHash, it's much much faster than the md5 implementation i was using. In preparation for keeping entry info in the database I found that calculating md5 for all entries in large archives was just too slow for my liking
Register a database context on a thread to use it when calling database::global() instead of the main thread's database context
- Start on some archive entry info in the library (unused for now, it'll likely be a tricky thing to get right, so will focus on just archive info to start) - Add some basic database models and handling functions in library - ArchiveManager now keeps track of the library id of open archives - Various other minor changes
This should be done anyway, but also will ensure save as works correctly in the library Also a bit of a start on saving entry list configuration per-archive
Column visibility, width and sorting are now remembered per-archive between sessions Wanted to do the filter visibility too but that's a bit trickier since it's more tied to the cvar
Now it's archive_ui_config and contains any general ui configuration for the archive. Splitter position is now remembered per-archive
Also has a bunch of refactoring of the library code, moving it to its own source folder and splitting by table/model since library.cpp was getting very large Also also some minor changes to some database handling stuff, including a new custom transaction class similar to the sqlitecpp one but with a bit more control over when it starts
Currently only text editor settings but will expand later where it makes sense
Will now be remembered between sessions
Currently only available via console (will eventually add a library UI of some sort)
Added a UI for the archive library, opens in a tab. - Added new toolbar to main window for 'general' stuff, has script manager, archive library and preferences - Updated run dialog to take an archive path rather than Archive* so it can just run anything - Bunch of other minor changes
It's such a simple change I'm not sure why it hasn't been added already, so I've just done it myself. Much better.
Moved a bunch of cvars to it, basically anything that was keeping track of ui state (cvars should generally be for options)
- Created SDataViewCtrl to split out some common functionality, used for ArchiveEntryTree and the library archive list - Library archive list columns configurable and saved between sessions
When reading from the library, give matching entries a type hint to help with type detection. Added a parameter to disable detecting entry types on load, so that the archive manager can detect types after reading info from the library (and use the type hints from the library if any)
Further modified SQLiteCpp to add 'viewExists' utility function, now also will load view definitions from slade.pk3
- Add archive_map table to database - Add # of maps column to library panel - Add a bunch of const to various Archive functions
Eventually we need to support selecting an iwad and using library archives as resources in the run dialog, but this will do for now
A temporary measure to keep recent files from <3.3.0 intact without having to go through and scan them all for the library. Will likely remove this in 3.4.0
Filters by filename currently, not sure there is anything else you'd want to filter by text but might add some other filter controls eventually like archive type, has maps, etc.
By default it'll select the current base resource
It's a bit annoying but at least now I don't have to re-add string_view support every time I update SQLiteCpp
To be used in the future when database updates are required with new SLADE versions, will eventually run scripts to update the database from what is in db_info.version to the database version required by SLADE
Not fully tested so there may be some situations where it breaks, but seems to work fine in basic cases so far
Doesn't do much yet but it'll be used in future for a few features eg. detecting and hopefully recovering from a previous crash, among other things
- Add function to get the database session id - Need to specify a busy timeout for database connections or else concurrent read/write won't work
It probably should be in cmake/ instead of dist/ but this will do for now
Moved Database.cpp/.h into a separate Database/ folder, split the classes out to separate files Plus a bunch of other misc. cleanup
Hopefully macOS too, will see how it goes
- Include xxhash in thirdparty instead of using system lib - Cast time_t to int64_t in bindDateTime
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This (WIP) branch/pr adds a SLADE program database for storing various data, info and settings that don't really belong in user-editable text files.
A basic overview of what is saved to the database (so far):
This also adds an archive library UI which lists all previously opened archives and allows opening or running them directly
I'm not 100% sure about going ahead with merging this for 3.3.0 since there is potential to introduce a whole new class of bugs :P However there are also many big improvements it could lead to such as simplifying and extending the undo/redo system, or potentially even recovering data after a crash.
Planned/potential additions: