Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DB Browser Crashes When Importing Its Own .sql dump #1045

Closed
ghost opened this Issue Jun 19, 2017 · 53 comments

Comments

Projects
None yet
3 participants
@ghost
Copy link

ghost commented Jun 19, 2017

Hi

I can get DB Browser to crash with the following steps:

  1. Open a 120MB .db database.
  2. Export the db to .sql using the File\Export option.
  3. Close the project and create a new database by clicking the New Database button.
  4. Open the Execute SQL tab
  5. Click the icon for loading a .sql file
  6. Click the > arrow to execute the .sql
  7. Crash

My question is: what is the upperbound MB size of a .sql file that can be imported and has 120MB exceeded it?

Thanks
Graham

@justinclift

This comment has been minimized.

Copy link
Member

justinclift commented Jun 19, 2017

Oops, just saw your mailing list message too. 😉

Which version of our software are you using, and on what OS?

People have reported using multi-GB files with DB4S (our acronym for DB Browser for SQLite), but I'm not remembering what kind of sizes people have reported for the SQL export/import part of things.

@ghost

This comment has been minimized.

Copy link
Author

ghost commented Jun 19, 2017

Version 3.9.1

Qt Version 5.7.0

SQLCipher Version 3.11.0

Win7-32bit

@justinclift

This comment has been minimized.

Copy link
Member

justinclift commented Jun 19, 2017

Interesting. Would you be ok to try this ~recent nightly build instead?

    https://nightlies.sqlitebrowser.org/win32/DB%20Browser%20for%20SQLite-2017-05-19-win32.exe

Note - There are newer nightly builds than that one, but please ignore them (for now), there's a crash bug after ~19th May or so which we haven't yet tracked down. 😉

@chrisjlocke

This comment has been minimized.

Copy link
Contributor

chrisjlocke commented Jun 19, 2017

Also, have you tried File | Import | Database from SQL file...

image

@ghost

This comment has been minimized.

Copy link
Author

ghost commented Jun 19, 2017

#justinclift:

  1. Downloaded: DB Browser for SQLite-2017-05-19-win32.exe
  2. Clicked No when asked if want to uninstall "old" version
  3. Click Next and Agree buttons
  4. Click Browse...
  5. Installation hangs!!!!!

Wow - it eventually popped up a dialog to navigate - probably took more than 2 minutes.

Then pops up with a dialog stating: "Error when connecting to github.../currentrelease. Connection refused"
Clicking OK it goes away.

After waiting 5 minutes I killed it - sorry! I loaded a 112MB .sql file and who knows when it would finish. Really, really slow.

After it crashed on me before I gave up using the app and wrote a little Java program to do the merging myself and on 120+MB .db files it took seconds using batch execution. What's going on with this app? After typing this text it's still running - I give up.

Graham

@justinclift

This comment has been minimized.

Copy link
Member

justinclift commented Jun 19, 2017

Gah. Well, this is starting out bad and getting worse! 😉

Ok, maybe let it uninstall the old version instead. That might be causing the hang. 😄

@justinclift justinclift added the windows label Jun 19, 2017

@chrisjlocke

This comment has been minimized.

Copy link
Contributor

chrisjlocke commented Jun 19, 2017

Have you 7zip? This can 'unpack' the installation files. DB4S is happy to run 'portable' (in the sense that it doesn't need to be installed). The 'installation hangs' could be Windows popping up a shield in the taskbar (the consent thingy), but you do sound like you know what you're doing, so I don't like saying, "are you sure you've checked the taskbar....." ;) 🐔

image

@ghost

This comment has been minimized.

Copy link
Author

ghost commented Jun 19, 2017

Although I selected "No" to not uninstall the previous version it DID uninstall it as I can't see the previous version. Bizarre.

@chrisjlocke

This comment has been minimized.

Copy link
Contributor

chrisjlocke commented Jun 27, 2017

@gmseed - any chance you could email me the file causing you problems? I presume it should ZIP up to something smaller than 120 MB?

@ghost

This comment has been minimized.

Copy link
Author

ghost commented Jun 27, 2017

@chrisjlocke - thanks but as I said in a previous post I've given up with this app. I moved to using SQLite Expert Personal. I had too many issues using this app. Also, the .db file is confidential and can't post. I'm sure similar results would be experience wit other large files, which could be easily created by repeatedly looping over a set of values and adding. Also, be shortly moving to using HSQLDB, which has a neat and robust browser.

@justinclift

This comment has been minimized.

Copy link
Member

justinclift commented Jun 27, 2017

No worries. Very much understand how frustrating it gets when something-that-needs-to-work... just doesn't. ☹️

@chrisjlocke If you've the time/interest to investigate more, maybe grab some of the larger databases from dev1, export them to sql, and see if they crash when being imported back?

If we can find some way to reliably trigger the crash, then we'll be able to fix it. Everything else would just be guesswork (in this instance).

@chrisjlocke

This comment has been minimized.

Copy link
Contributor

chrisjlocke commented Jun 27, 2017

Not sure about the crashing, but I'm assuming the delay is due to the individual transactions. I posted ages ago about importing taking ages... can't remember if that got fixed. I'll take another look.
I'll close this issue on the request of the originator for the time being, and will re-open a new issue if I find out anything.

@justinclift

This comment has been minimized.

Copy link
Member

justinclift commented Jun 27, 2017

@chrisjlocke Hmmm, I'm more thinking we should leave this open until we've had a chance to try out some import/export of larger sized files. It's possible something has gone weird that causes the problem in a generic way, which would let us capture and fix it. If it's something specific to @gmseed's data set though - which we don't have access to - we might as well close it as that's one challenge too far at the moment. 😉

@chrisjlocke chrisjlocke reopened this Jun 27, 2017

@chrisjlocke

This comment has been minimized.

Copy link
Contributor

chrisjlocke commented Nov 9, 2017

The importing has been overhauled considerably in the latest nightlies, so I'll close this.
@gmseed - if you're able to try one of the latest nightlies, importing should be OOOOOOODLES quicker by a magnitude of MILLLLLLIONS.*

* Not guaranteed.

If you're still getting an issue whatsoever, please comment back (you can still comment on closed issues)

@chrisjlocke chrisjlocke closed this Nov 9, 2017

@justinclift

This comment has been minimized.

Copy link
Member

justinclift commented Nov 9, 2017

Re-opening this, as the importing was only changed for CSV imports, not SQL imports. It's a different set of code. 😉

@justinclift justinclift reopened this Nov 9, 2017

@chrisjlocke

This comment has been minimized.

Copy link
Contributor

chrisjlocke commented Nov 9, 2017

Doh, good point. Sorry!

@justinclift

This comment has been minimized.

Copy link
Member

justinclift commented Nov 9, 2017

No worries at all. 😄

@chrisjlocke

This comment has been minimized.

Copy link
Contributor

chrisjlocke commented Feb 12, 2019

Lol. Importing a 52 MB sql file is taking a while. After 15 minutes, I'm still on 0%.

image

I assume its processing each line in its own transaction. I know there is multithreading code in the CSV import, but in the 'short term', wrapping up the import in just one transaction should speed this up immensely.

@justinclift

This comment has been minimized.

Copy link
Member

justinclift commented Feb 12, 2019

Hmmm, 52MB should happen pretty quickly. As in a few seconds. 😦

@justinclift

This comment has been minimized.

Copy link
Member

justinclift commented Feb 12, 2019

Oh... that's right. It's a different bit of code, so maybe not. Oops.

@MKleusberg

This comment has been minimized.

Copy link
Member

MKleusberg commented Feb 13, 2019

Yeah, the code is far from ideal for files that large. We could think about rewriting this piece of code. However, I don't think the performance gain would be as large as with the CSV import. We're by the way only using one transaction already, so no easy way to speed it up like this 😉 It's also interesting to see that the performance varies a lot depending on the structure of the database, i.e. many short statements are much more problematic compared to few long statements it seems.

@MKleusberg

This comment has been minimized.

Copy link
Member

MKleusberg commented Feb 13, 2019

Looking closer at this... well, the performance probably can be improved a bit. Somebody wants to prepare a benchmark while I try to speed this function up? 😄

@chrisjlocke

This comment has been minimized.

Copy link
Contributor

chrisjlocke commented Feb 13, 2019

I had a 52 MB file the other day, which I had to cancel in the end. I'll reduce that down tomorrow and give some timings, and how many lines in the file. I wanted to compare whether multiple rows per insert was quicker/slower as well.

@MKleusberg

This comment has been minimized.

Copy link
Member

MKleusberg commented Feb 14, 2019

Here's what we do: we read the entire file into a QString (which is always UTF16 encoded as far as I remember). So from whatever the file encoding is, we have a conversion to UTF16. We then convert that to an UTF8 byte array which we can pass to SQLite. That's the second conversion which crashes in your case. Probably because it is using too much memory after all (including memory fragmentation maybe).

I have an idea though which I'll try later today 😄 In the meantime it's worth checking if the import has already become faster for smaller files which don't crash the application.

@justinclift

This comment has been minimized.

Copy link
Member

justinclift commented Feb 14, 2019

Cool. I'm going to get some sleep soon, but will take a look back at this tomorrow. 😄

MKleusberg added a commit that referenced this issue Feb 14, 2019

Further performance improvements in the SQL import
This eliminates some unneeded conversions in the SQL import code which
should speed up the import and reduce its memory consumption.

See issue #1045.
@MKleusberg

This comment has been minimized.

Copy link
Member

MKleusberg commented Feb 14, 2019

Just committed my attempt to fix this. I didn't get the crash before, but I can definitely say it's taking a lot less time now until the progress dialog shows up. So I guess that's a good sign, maybe 🤞

@justinclift

This comment has been minimized.

Copy link
Member

justinclift commented Feb 14, 2019

Cool. Starting my day now, so will look at this shortly. With that code piece which reads the whole SQL file into a single string first, would it be feasible to instead break it into a loop for chunking the file?

eg processing (say) 10k lines at a time

That should stop the crashing, and it might not be terribly slower either. Obviously, that last bit would need testing. 😄

@justinclift

This comment has been minimized.

Copy link
Member

justinclift commented Feb 14, 2019

Oh... testing your new code now, with the commit that uses a QByteArray instead of a QString, no longer crashes. Instead, the progress dialog is displayed and goes through to 100% (smoothly), importing the whole file. At the end, a new pop up asking if we wish to save the changes happens (that could probably be skipped).

So, import no longer crashes. The whole "single" file (database size of ~858MB) is created and looks ok. Major win. 😄

@justinclift

This comment has been minimized.

Copy link
Member

justinclift commented Feb 14, 2019

This code change should be simple enough to add to the 3.11.x branch too yeah? eg that'd stop the crashing (for larger files) there as well

@MKleusberg

This comment has been minimized.

Copy link
Member

MKleusberg commented Feb 15, 2019

Wow, didn't expect it to be that much of a difference 😄 And yeah, should be safe enough to include it in the 3.11.1 release.

@chrisjlocke Can you try again with the latest nightly build too? Would be interesting to see how it compares to your VB program 😉

@MKleusberg

This comment has been minimized.

Copy link
Member

MKleusberg commented Feb 15, 2019

@justinclift

eg processing (say) 10k lines at a time

I though about that too but I think this would be pretty complicated and definitely not worth the effort for now. The reason is that we can't expect one SQL statement per line and we cannot simply check for a semicolon at the end of a line either because that might be a comment, a string, inside a trigger definition, etc. So there would be a lot of parsing involved to be able to load the file in chunks.

MKleusberg added a commit that referenced this issue Feb 15, 2019

Don't ask for saving changes when importing a SQL file
When importing a SQL file into a new database, we would create a
database file, import everything, then close the database and open it in
the GUI. While closing, however, the application asks you whether you
want to save the changes you made which can be confusing. This is solved
by creating an empty database, then closing it immediately without any
changes just to then open it in the GUI. Only then the import takes
place.

See comment in issue #1045.
@MKleusberg

This comment has been minimized.

Copy link
Member

MKleusberg commented Feb 15, 2019

At the end, a new pop up asking if we wish to save the changes happens (that could probably be skipped).

That should be solved now too 😄

@chrisjlocke

This comment has been minimized.

Copy link
Contributor

chrisjlocke commented Feb 15, 2019

@chrisjlocke Can you try again with the latest nightly build too? Would be interesting to see how it compares to your VB program

Nah, my program is better! 😉

Well, not really. DB4S now loads the 52 MB import in approx. 13 seconds, so a lot quicker than my reading it all line by line!

Just one 'note'. If I read in a .sql file from 'multiple rows' turned off, then the progress bar is smooth. Takes ~13 seconds. If I read in a .sql file where 'multiple rows' was turned on, then the progress bar doesn't move (and I get a 'Not Responding' mouse cursor after about 8 seconds) then it jumps to 99%, then 100%. Still takes ~13 seconds. It makes sense, as its just one sql statement (736,000+ lines of it), but I was wondering where the delay was. Is it 10 seconds reading that lot, then 5 seconds for SQLite to process it? Wonder if the dialog could be amended - flipping between 'reading statement...' then 'processing statement .... (734,432 lines)', etc. Just a thought.

But thanks for looking into this and speeding it up - a great addition. Appeciate it, while there are 290+ other issues in DB4S.

@MKleusberg

This comment has been minimized.

Copy link
Member

MKleusberg commented Feb 15, 2019

Oh cool, I guess we can close this issue then 😄 Any objections?

@chrisjlocke The entire reading happens before the progress dialog is shown. It also seems to happen reasonably fast, so probably no point flipping between 'reading statement' and 'processing statement'.
Before my first patch, instead of actually doing the work the import was spending most of the time on checking how much work there is left to do (which coincidentally is exactly what I do as a person too). The remaining ~13 seconds are almost entirely spent inside SQLite functions now. Because we can't check the progress of these we can only calculate the progress between two statements. Now in case of the 'multiple rows' option there are only a hand full of statements in the file (one of them having 736k lines) which makes the progress bar jump around. So in short, I don't think we have any chance to change this unless SQLite adds some functions which let us peek inside its inner workings and check the progress 😄

@chrisjlocke

This comment has been minimized.

Copy link
Contributor

chrisjlocke commented Feb 15, 2019

Aah, My thinking was 'OK, I'm going to pass this SQL statement to SQLite to process. Ooh, its quite long - contains a lot of lines. Thats gonna take a while. I'll just let the user know its 736,000 lines worth...' rather than a progress bar at 0%. I can understand why its that - as you say, only two statements, etc. But before we pass SQLite those statements, we know they're pretty large.

@justinclift

This comment has been minimized.

Copy link
Member

justinclift commented Feb 15, 2019

Looks good to me. 😄

@MKleusberg Should I cherry-pick the commits across to v3.11.x, or will you?

@MKleusberg

This comment has been minimized.

Copy link
Member

MKleusberg commented Feb 15, 2019

@chrisjlocke Ah, I see what you mean 😄 There's another issue then. We only know how many bytes are left in total before handing them over to SQLite. But we don't know where the next statement ends because SQLite determines that for us. So we get to know whether a statement is particularly long or not only when it's too late.

@justinclift I'll cherry-pick them in a second 😄

@chrisjlocke

This comment has been minimized.

Copy link
Contributor

chrisjlocke commented Feb 15, 2019

But we don't know where the next statement ends because SQLite determines that for us.

Well cocksticks. I've used up all my good ideas for the day. I'll leave this with you to resolve. 😜

MKleusberg added a commit that referenced this issue Feb 15, 2019

Speed up the SQL import a bit
This commit attempts to speed up the SQL import a bit by removing
unnecessary conversions and counting of characters. It's still far from
perfect though.

See issue #1045.

MKleusberg added a commit that referenced this issue Feb 15, 2019

Further performance improvements in the SQL import
This eliminates some unneeded conversions in the SQL import code which
should speed up the import and reduce its memory consumption.

See issue #1045.
@MKleusberg

This comment has been minimized.

Copy link
Member

MKleusberg commented Feb 15, 2019

@chrisjlocke Heh heh, I guess I'll tend to the other issue then 😉

@justinclift Cherry-picking is done 😄

@justinclift justinclift added this to the 3.11.1 milestone Feb 15, 2019

@MKleusberg

This comment has been minimized.

Copy link
Member

MKleusberg commented Feb 15, 2019

Thanks again to @chrisjlocke for bringing this issue to my attention 😄

@chrisjlocke

This comment has been minimized.

Copy link
Contributor

chrisjlocke commented Feb 15, 2019

Always a pleasure. Now get on and fix #1725 so the nightlies can be used again. 😆 💥

@MKleusberg

This comment has been minimized.

Copy link
Member

MKleusberg commented Feb 15, 2019

Actually, I was already working on that issue before you mentioned it 😉

@chrisjlocke

This comment has been minimized.

Copy link
Contributor

chrisjlocke commented Feb 15, 2019

🥇

@justinclift justinclift referenced this issue Feb 16, 2019

Closed

3.11.1 - outstanding pieces #1747

12 of 13 tasks complete

deepsidhu1313 added a commit to deepsidhu1313/sqlitebrowser that referenced this issue Feb 20, 2019

Changes from upstream (#8)
* Replace link style from pragma labels to an embedded icon with link

Even after having removed the hard-coded style from the link texts, this
combination of hyperlink over regular window background is not giving good
results in some dark theme colours. It is uncommon to have this combination
in GUI applications.

The link in the label has been replaced by a link in an embedded icon.

In order to be still recognisable as a link (apart from the hand icon) the
link hover signal is connected to a slot showing the link in the status bar
for 5 seconds.

See issue sqlitebrowser#1493

* Align Mac shortcut in Scintilla to standard and Qt for Alt+Backspace

In Mac the standard shortcut for deleting word to the left is Alt+Backspace
so we set this key sequence in Scintilla editors as alternate for Mac.

Reference:

http://doc.qt.io/qt-5/qkeysequence.html
Section Standard Shortcuts:
DeleteStartOfWord	is Alt+Backspace in macOS column.

See issue sqlitebrowser#815

* Cmd+Backspace should delete from cursor to the beginning of line on macOS

See issue sqlitebrowser#815

* Merge updated Spanish translation for v3.11.x

- New SQLcipher dialog strings
- Extensions filter
- Remove translation for Alt+Del, since it has to match the official Qt
translation and it changes between versions. When the shortcut isn't
changed, it is better to not translate the shortcut at all.

# Conflicts:
#	src/translations/sqlb_es_ES.ts

* Merge updated Spanish translation for v3.11.x (Fix possible resource leaks)

This is an update of the translation file followed by a manual merge of
0ab9f44

See issue sqlitebrowser#1691

* Merge e completamento traduzione italiana (sqlitebrowser#1705)

* Nuke the AppImage stuff until we have time to investigate the failures

* Add source code locations to Portuguese translation file

Otherwise linguist doesn't show the widgets or source code references.

See PR sqlitebrowser#1707

* "Save Project" to remember filename and new "Save Project As" action

"Save Project" remembers used filename, at second and successive calls
the user is not asked for the filename. Ellipsis are removed, since it
only opens a dialog the first time.

A Wait Cursor is set while saving so the user does not get the impression
that pressing "Save Project" does nothing.

"Save Project As" action is added, so the user can still save to another
filename when saved for the first time. Added icon combining package.png
and textfield_rename.png from the Silk icon set.

The base filename for a new project does not include the DB suffix.

Problems saving the project file are detected and the user warned.

See issue sqlitebrowser#1706

* Added icon for foreign-key fields

Fields having a foreign-key constraint are shown with a distinctive icon.

New icon based on page_green.png and bullet_key.png from the Silk icon set.

See comment in issue sqlitebrowser#192

* New Czech translations (sqlitebrowser#1710)

* dbhub: Attempt to fix the OK button in the Push dialog

The OK button in the Remote Push dialog sometimes needed to be clicked
twice in order to work. That's because whenever the Database Name field
loses focus, we would send out a request for the updated branch list.
While processing this request the OK button is effectively disabled by
Qt. This commit aims for an easy fix by changing the request to happen
whenever the text in the Name field changes. This way the timing is a
lot less sensitive. We will need to see however if this introduces other
timing issues for users with slow Internet connections.

* Add a comment

* Fix executing selection only in Execute SQL tab

When executing only a part of a query by selecting the start of it in
the Execute SQL tab, SQLite would execute it until the next semicolon,
even when the selection does not go as far. This commit changes this
back to the old behaviour to only ever execute exactly the selected part
of a query.

See issue sqlitebrowser#1708.

* Ask user about saving any modified data in SQL tabs

User is asked once for saving modified SQL tabs not attached to a file to
the current project file (or to a new project file, when there isn't any
yet).

For each tab linked to a file, the user is asked to save the changes to the
file.

The asking is performed when the tab is closed, or when the application
is closing.

The modified attribute of the editor is set to false when the data is
saved or has just been loaded in the editor.

See issues sqlitebrowser#1386 and sqlitebrowser#1706

* New action "Save All"

New action "Save All" for saving all the files currently opened by the
application: DB file, SQL files and project file. Can be activated through
shortcut (Ctrl+Shift+S) and File menu.

Icon composed from package.png and database_save.png.

See issues sqlitebrowser#871 and sqlitebrowser#1706

* Fix shortcut items in the MSI installer

See issue sqlitebrowser#1713.

* refactor: find correct QScintilla package

* Building with CMake on macOS (sqlitebrowser#1644)

* Fix override warning in selectAll

This change fixes a cmpile warning on Mac OS: (10.12.6) + Clang

warning: 'selectAll' overrides a member function but is not marked
'override'

Fixes sqlitebrowser#1718

* Rectify the sort indicator after multiple column selection

When more than one column is selected we abort the sorting, but Qt has
already assigned the sort indicator to the last selected column, so we
have to restore the saved settings to fix the sort indicator column.

See issue sqlitebrowser#1717

* Double click a column for selecting it

Since the mouse release triggers the sorting, we were unable to select
only one column. Connecting the double click over the header to a column
selection, we are able to select that column. However, we cannot select
one column without sorting by it, since the sorting is triggered before,
on the first mouse click.

See issue sqlitebrowser#1717

* CMake installation on macOS should also copy the icon & desktop files

Should address sqlitebrowser#1723

* Homebrew seems to have removed all support for formula options

* Use our new, slightly customised, SQLite3 Homebrew tap

* Enable/disable "Save All" button

The  "Save All" button is now disabled and enabled with the same criteria
than the save project buttons.

See issues sqlitebrowser#871 and sqlitebrowser#1706

* Fix ambiguous shortcut for Ctrl+/

The shortcut was ambiguous because it was active in the button and in the
SQL editor itself. In the button was only documentation and was made
ambiguous when enabling the feature and fixing the shortcut in
6425fb1

Now its scope is reduced to widget like similar button shortcuts so it can
be unambiguous in the editor.

See issue sqlitebrowser#1614

* Use Qt 5.12.1 for our Win64 nightly builds

* qtquick1_*.qm doesn't seem to be included with Qt 5.12.1

* Switch Win64 nightly builds back to Qt 5.11.3

This is due to a weird word wrap bug in Qt 5.12.1

* Update DB4S version numbers in our issue templates

* Added 3.11.0 to the README.md Releases section

* Update BUILDING.md

Grammar tweak.

* Updated currentrelease file for 3.11.0

* Improve the size policy (toolbars and improved icons) (sqlitebrowser#1684)

* Improve the size policy (and more and better icons)

These changes improve the size policy for translations having long texts
in some buttons by:

* Converting the text buttons in the Edit Database Cell to icons
* Making the "Type of data" label wrappable and expandable
* Converting the text buttons in the Browse Data tab to icons
* Allowing the Plot combo-boxes to shrunk

All this allows both the Browse Data and the docks to grow and shrink with
more freedom.

New icons for buttons are reused when appropriate in context menus.

Added icon for filter and improve icon for docks (has been mirrored so it
matches the actual dock position).

Added Print icon in Edit Database Cell using the extra free space, so the
print action there is more visible.

This continues the effort started in sqlitebrowser#1324.

* Convert the embedded buttons to actual toolbars

This provides more flexibility, like the way how toolbars are compacted
when they have not enough space.

The QToolButtons in Browse Data tab and in Edit Cell dialog are converted
to QActions and inserted in new toolbars embedded in the same place as
the old buttons. Everything else should stay the same (shortcuts, tool-tips
and what's-this information).

* Set style for all toolbars in Preferences and minor adjustments

The combo-box used for the main toolbar is replicated for all the toolbars
in application. In this way, users with high resolutions can use the styles
with both icons and text, while users with lower resolutions can leave the
default styles, which should be better for them.

Some icon texts has been abbreviated from their default values, so they fit
better in the toolbars when they are visible.

The print icon in Edit Cell has been moved to the right, where it would be
the first to be collapsed.

The original what's-this info for Set as NULL in Edit Cell toolbar has been
restored.

* Remove no longer used overloaded function

The addShortcutsTooltip function applying to QWidget was no longer used
after having converted all the buttons to actions, so it is removed.

* grammar: Disable window function parsing

Remove the window function grammar rules. I think they are not needed
anyway as long as we only parse CREATE TABLE and CREATE INDEX
statements and unexpectedly they do seem to cause problems. It it still
worth investigating how this is possible but for now removing them seems
like the best option.

See issue sqlitebrowser#1733.

* tests: Add test case for ee70a34

Add a test case for the bug which motivated disabling the window
function rules in ee70a34.

See issue sqlitebrowser#1733.

* Make Add Record button work again with a single click

In 5f4d0ee the Add Record button in the
Browse Table tab was inadvertently changed to only open a popup menu
after a long click but not triggering the Add Record action after a
normal click. This restores the old behaviour.

* Fix text detection check

Truncating the text in bytes boundaries for the quick test was breaking
the text detection for Russian and probably any script encoded in more than
one byte. The problem occurred probably when a multibyte character was
truncated at the 512 boundary. This is a bit improbable in latin-based
languages like German or Spanish, whose most characters are a byte, but
very easy in other scripts, like Cyrillic, whose characters are encoded in
more than one.

The new approach is based in QTextCodec finding invalid characters using
the current encoding, which seems immune to the truncation problem.
According to callgrind, it has also better performance, probably because it
does not involve memory comparison.

See issue sqlitebrowser#1731

* Detect XML data starting with a declaration

Checking for "<?xml" at the beginning of a text is a quick and simple test
that will detect most of the XML cases.

This fills an obvious gap in sqlitebrowser#1537.

* Fix static analyzer issues (sqlitebrowser#1727)

* Fix `barsGroup` memory leak

* Remove unused values

* Revert currentrelease to 3.10.1

Our encryption of new databases with SQLCipher 4.0.1 is broken :(,
so we'll need to fix it and do a 3.11.1 release.

* Class 'NullLineEdit' lacks Q_OBJECT macro

Qt Creator and lupdate are giving warnings about this class lacking the
Q_OBJECT macro. Although it seems to not be needed, adding it will silence
the warnings without drawbacks.

See issue sqlitebrowser#1740.

* sqlcipher: Fix editing the encryption for SQLCipher4

With SQLCipher4 the encryption was not working as expected because the
KDF and HMAC algorithms were not set properly. This is fixed in this
commit so it should work now with SQLCipher4 as well as SQLCipher3.

See issues sqlitebrowser#1690 and sqlitebrowser#1732.

* Add warnings to cmake compilation (sqlitebrowser#1721)

* Add warnings to cmake compilation

Use the same set of warning flags that are used for qmake compilation.

See comments in sqlitebrowser#1718.

* Add condition for warning flags not supported by GCC 5.5

Satisfy Travis build by adding the unrecognised warning flags only when
the compiler version is greater or equal 7.0. Maybe those flags are
available in previous versions, but I don't know when they were introduced.

Tested with GCC 7.3.

* CMake option for enabling GCC warnings

This option follows the qmake configuration, where the same all_warnings
option exist. This allows users to select compiling with or without
warnings using "cmake -DALL_WARNINGS=ON" or OFF.

* Speed up the SQL import a bit

This commit attempts to speed up the SQL import a bit by removing
unnecessary conversions and counting of characters. It's still far from
perfect though.

See issue sqlitebrowser#1045.

* Allow downgrading from nightly versions to stable releases

  sqlitebrowser#1743 (comment)

* DowngradeErrorMessage needed removing too

* AllowSameVersionUpgrades also needed removing

* Fix build on some platforms

Fix build issues caused by 13a8a1f.

* Further performance improvements in the SQL import

This eliminates some unneeded conversions in the SQL import code which
should speed up the import and reduce its memory consumption.

See issue sqlitebrowser#1045.

* Add more of our macOS packaging files

* Don't ask for saving changes when importing a SQL file

When importing a SQL file into a new database, we would create a
database file, import everything, then close the database and open it in
the GUI. While closing, however, the application asks you whether you
want to save the changes you made which can be confusing. This is solved
by creating an empty database, then closing it immediately without any
changes just to then open it in the GUI. Only then the import takes
place.

See comment in issue sqlitebrowser#1045.

* Fix alterTable() function

Fix a few issues in the alterTable() code. One type of issue would
happen if there are any keys or constraints in the table. Because the
check whether more changes are needed did not work as expected, we would
try to edit the table again, even though it is already correct. The
second type of issue (which can be triggered independently but which can
also be a follow-up issue to the first one) tries to access the table by
its old name even though it might already have been renamed.

See issue sqlitebrowser#1725.

* Allow custom display formats (sqlitebrowser#1720)

Allow user to define their own display formats. The SQL code is editable
and the combo box changes automatically to custom if the user edits one
of the predefined formats.

The display format (either custom or predefined) is validated with a 
case-insensitive regex so at least it contains a function applied to the column
name. 

Added a new callback for executeSQL following the model of sqlite3_exec.
Using this, the number of columns is got from a checking query execution. If
it is not one, the custom display format is also rejected.

Note that these validations might still be fooled in some unforeseen way, but
the users should know what they're doing.

See issue sqlitebrowser#573

* Add SQLITE_ENABLE_STAT4 to the nightly Windows builds

For the windows component of sqlitebrowser#1161.

* Make early network accesses more reliable when using WLAN

The automatic update check is performed early during the application
start. It turns out that, when using a Wifi connection, the Qt
networking code is not ready yet at this point which leads to an
"Network inaccessible" error. This commit delays the automatic update
check until the network configuration is loaded completely.

See issue sqlitebrowser#1595.

* Add 3.11.1 to README.md and to the issue templates

* Move the automatic update check back into the main window class

See issue sqlitebrowser#1595.

* Update currentrelease file for 3.11.1

* Simplify code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.