Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

This fixes #6176 and #6180 - Major Upgrade Bug Fixes #6177

Closed

Conversation

Butterflysaway
Copy link
Contributor

@Butterflysaway Butterflysaway commented Jul 17, 2020

Fixes issue I published.

fixes #6176
fixes #6180

@Butterflysaway Butterflysaway changed the title Update upgrade_2-1_mysql.sql Update upgrade_2-1_mysql.sql Fixes #6176 Jul 17, 2020
@Butterflysaway Butterflysaway changed the title Update upgrade_2-1_mysql.sql Fixes #6176 Update upgrade_2-1_mysql.sql #6176 Jul 17, 2020
@Butterflysaway Butterflysaway changed the title Update upgrade_2-1_mysql.sql #6176 This fixes #6176 Jul 17, 2020
@Butterflysaway
Copy link
Contributor Author

Hello @live627 please make sure this gets included in the next SMF 2.1 if one ever gets released. I think the developers need to add another upgrade which fixes all past upgrades to 2.1 for attachments. This bug looks like it's been there since the beginning of 2.1. I think you could easily add a function to recheck for attachments/avatars which failed to update.

@Butterflysaway Butterflysaway changed the title This fixes #6176 This fixes #6176 and #6180 Jul 20, 2020
@Butterflysaway Butterflysaway changed the title This fixes #6176 and #6180 This fixes #6176 and #6180 - Major Upgrade Bug Fix Jul 20, 2020
@Butterflysaway Butterflysaway changed the title This fixes #6176 and #6180 - Major Upgrade Bug Fix This fixes #6176 and #6180 - Major Upgrade Bug Fixes Jul 20, 2020
@albertlast
Copy link
Collaborator

albertlast commented Jul 20, 2020

First it's not smart two combine to fixes in one pr
secondly why you extend the query about attachment_type and later on you skip attachment_type 1 ?

Copy link
Contributor

@Arantor Arantor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of doing this, you should be able to fix the problem by way of instead forcing attachments to be handled in a consistent order (i.e. applying a ORDER BY to the processing query that steps through in groups). I don't believe this fixes whatever problem you actually have.

@albertlast
Copy link
Collaborator

  1. you will fine pr from me how more than one year old...

  2. you second fix is none fix, i guess you run this so many time that it at some point got done or you database is too small for the amount of data,
    would be also a good explaination why you got Upgrade Script Error #6176 since your setup didn't match the size of your forum.

@albertlast
Copy link
Collaborator

i'm not part of the smf team, should be easy to since the collaborator text beside my name.

The order by hint from arantor looks more in right direction to possible fix the issue,
also you forget postgres upgrade here to fix too.

@Butterflysaway
Copy link
Contributor Author

Instead of doing this, you should be able to fix the problem by way of instead forcing attachments to be handled in a consistent order (i.e. applying a ORDER BY to the processing query that steps through in groups). I don't believe this fixes whatever problem you actually have.

I guess if you would like to develop a different fix that works the same way then that's fine. I have confirmed for myself at least that this fixes the problem. This bug affects every prior version of smf when upgrading attachments/avatars to 2.1. You can test this if you have a 1.1.21 or 2.0.17 forum with a few hundred attachments/avatars.

This bug does not happen when you do not have any avatars and just a bunch of attachments. It has to be a mix of attachments/avatars in the attachment folder/database.

@Butterflysaway
Copy link
Contributor Author

In other words it works by accident for you because you have MyISAM tables and lowering the block size is why it works. You could, of course, just change the query to include ORDER BY id_attach and get all of the fix to work without any other changes.

Pretty sure that will not work as you are later setting attachment_type = 1 which changes the total size of the rows being selected each time the updated avatar is changed to attachment_type = 1. Either way I don't want to spend more time retesting something that works just fine in this pr.

fyi my tables are InnoDB

@jdarwood007
Copy link
Member

So the upgrade logic here is to rename all non avatars (avatars are attachment_type = 1)

First the attachment_type logic seems to be
0 == attachment
1 == avatar
3 == thumbnail

I can't find a usage of 2.

A way to fix this would be to the search where file name NOT LIKE '%.dat' as well, and then walk over that until we get no results. Only issue is with a large forum there would be no index on the file name and this could be slow.

@sbulen sbulen added Upgrader and removed Installer labels Jun 20, 2021
@sbulen
Copy link
Contributor

sbulen commented Jun 23, 2021

... you are later setting attachment_type = 1 which changes the total size of the rows being selected each time the updated avatar is changed to attachment_type = 1.

Butterflysaway is correct here. This explains why the check for type makes more sense within the processing loop than in the query.

But others are also correct here, in that using a LIMIT without an ORDER BY may yield questionable navigation thru the set as well. Lots of articles out there on that (& in fact I think that has bitten us before).

So... I think we need a companion PR to this that adds an ORDER BY here, and we should test the two PRs together. I'll label this as final & work on doing so.

@sbulen sbulen added this to the Final milestone Jun 23, 2021
BrickOzp added a commit to BrickOzp/SMF2.1 that referenced this pull request Jul 2, 2021
During the legacy attachments upgrade, batches of 100
rows that do not have attachment_type of 1, are queried.
But in the prosessing of these rows, attachment_type is
sometimes change to 1. When this happens, rows will be
skipped and those attachments not processed. This can
lead to file not found errors. Fixed this problem by
moving the attachment_type check, from the query and
into the loop.

Fixes SimpleMachines#6181
Fixes SimpleMachines#6810
Closes SimpleMachines#6177

Co-authored-by: Butterflysaway
Signed-off-by: Oscar Rydhé <oscar.rydhe@gmail.com>
BrickOzp added a commit to BrickOzp/SMF2.1 that referenced this pull request Jul 5, 2021
During the legacy attachments upgrade, batches of 100
rows that do not have attachment_type of 1, are queried.
But in the prosessing of these rows, attachment_type is
sometimes change to 1. When this happens, rows will be
skipped and those attachments not processed. This can
lead to file not found errors. Fixed this problem by
decreasing the index when changing attachment_type.

Fixes SimpleMachines#6181
Fixes SimpleMachines#6810
Closes SimpleMachines#6177

Signed-off-by: Oscar Rydhé <oscar.rydhe@gmail.com>
MissAllSunday pushed a commit that referenced this pull request Jul 6, 2021
During the legacy attachments upgrade, batches of 100
rows that do not have attachment_type of 1, are queried.
But in the prosessing of these rows, attachment_type is
sometimes change to 1. When this happens, rows will be
skipped and those attachments not processed. This can
lead to file not found errors. Fixed this problem by
decreasing the index when changing attachment_type.

Fixes #6181
Fixes #6810
Closes #6177

Signed-off-by: Oscar Rydhé <oscar.rydhe@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Avatar/Attachment Upgrade failure - only some convert correctly Upgrade Script Error
6 participants