Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring #73

Merged
merged 52 commits into from Jun 22, 2023
Merged

Refactoring #73

merged 52 commits into from Jun 22, 2023

Conversation

splitbrain
Copy link
Member

Deprecate the old helper class in favor of a new dokuwiki\plugin\sqlite\SQLiteDB with a cleaner API and proper use of exceptions. Removal of all sqlite2 support, only PDO::sqlite is supported. Major code cleanup. Merge and cleanup of #65

@splitbrain
Copy link
Member Author

@annda FYI you might want to look at this when debugging the struct migration stuff.

solewniczak and others added 7 commits March 17, 2023 13:18
Multiple statements are only executed with PDO::exec() not with a
prepared statement
This ensures that the helper is working as intended for backwards
compatibility. New tests for the new class still have to be added.
@splitbrain

This comment was marked as resolved.

@fjf2002

This comment was marked as outdated.

This is supported natively in sqlite now
@splitbrain
Copy link
Member Author

This should now be ready to merge.

@annda I would like to run this branch on a bunch of our wikis first, to see if there are any issues with the backwards compatibility. Once we're satisfied that it works, we can merge and start migrating our plugins to use the new class.

@Klap-in
Copy link
Contributor

Klap-in commented May 16, 2023

Could you do please also a update of the documentation if it merged, with some guiding what is updated?

@splitbrain
Copy link
Member Author

@Klap-in absolutely! docs will be updated as soon as this is merged.

@splitbrain
Copy link
Member Author

The new code wraps each migration (regardless if it's a normal file based one or an event based one) into a transaction and properly rolls it back if an exception happens. This however means that migrations may not have any transactions themselves because nested transactions are not supported by sqlite.

Currently I am aware of the following plugins doing their own transactions in migrations:

  • struct
  • structstatus
  • structpublish

The current docs do not mention to use transactions in migrations and it seems no other plugins used them. So I would suggest to adjust the plugins above instead of changing the behaviour of the SQLiteDB class.

@annda
Copy link
Contributor

annda commented Jun 1, 2023

This PR dropped the custom implementation of group_concat but it was slightly different than the native one. Namely, it applied array_unique() on the data, which the native function does not do.

We rely on group_concat in the struct plugin when building queries. I haven't noticed any side effects of the change, except that a few tests fail. I'm not yet sure to handle this, probably by enforcing uniqueness somewhere in the code just after data has been fetched from the database.

Some other plugins also use the function and may have to be adjusted.

Even though mutliple statements can be passed to $sth->execute(),
failures will only be checked in the very first statement. Failures
later on will lead to the rest of the statements to be silently ignored
rendering our rollback strategy moot.

We now always split multiline statements and run them separately
(affects import and migrations).

The export was adjusted to not include the transaction handling and
fix entry exports.
PDO::prepare sometimes throws ValueErrors (and maybe other Throwables)
so we convert them to PDOExceptions
This is clearer in what to expect as a return value
@splitbrain
Copy link
Member Author

The correct syntax to use when you don't want duplicates would be GROUP_CONCAT(DISTINCT column). Unfortunately this is broken when you want to also set a delimiter as discussed here:

Workarounds on this limitation directly in the SQL are outlined above (sub selects) but I guess fixing it in code after retrieving the results is often easier.

I don't think it's worth to fix backwards compatibility for this and would rather have plugin authors adjust their code should it be affected.

splitbrain added a commit to cosmocode/starred that referenced this pull request Jun 13, 2023
This is a common use case in plugins
splitbrain added a commit to cosmocode/starred that referenced this pull request Jun 13, 2023
Since using GROUP_CONCAT with a custom separator and the DISTINCT
keyword is still broken in sqlite (and might never be fixed), this
readds our custom implementation back. Using a different name than the
default allows developers to pick the native implementation when
adequate.

See also #73 (comment)

This reverts commit ff1cb7a.
This is what the old query methods allowed. Passing in an array is still
somewhat cleaner, but allowing separate arguments should make porting
older code a bit more convenient.
@splitbrain splitbrain merged commit f84dbd1 into master Jun 22, 2023
8 checks passed
@splitbrain splitbrain deleted the devel branch June 22, 2023 06:04
@gerardnico
Copy link

Hy Andi,

I just made the migration. I had no time to check the new beauty structure.

During the update, just for your information, I got it by the cursor cache data.
I ended up copying the whole code because the cache variable is private.

Cheers
Nico

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants