Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

1.7 datalists schema prevents upgrade to 1.8 #6814

Closed
iionly opened this issue May 3, 2014 · 24 comments

Comments

@iionly
Copy link
Contributor

commented May 3, 2014

Happened now twice when upgrading test installations from Elgg 1.7 to 1.8. Error output is:

Fatal Error.

Data too long for column 'name' at row 1

QUERY: INSERT into elgg_datalists set name = 'simplecache_lastupdate_installation', value = '0' ON DUPLICATE KEY UPDATE value='0'

DatabaseException Object
(
[message:protected] => Data too long for column 'name' at row 1

QUERY: INSERT into elgg_datalists set name = 'simplecache_lastupdate_installation', value = '0' ON DUPLICATE KEY UPDATE value='0'
[string:Exception:private] => exception 'DatabaseException' with message 'Data too long for column 'name' at row 1

QUERY: INSERT into elgg_datalists set name = 'simplecache_lastupdate_installation', value = '0' ON DUPLICATE KEY UPDATE value='0'' in /xxx/engine/lib/database.php:274
Stack trace:
#0 /xxx/engine/lib/database.php(468): execute_query('INSERT into elg...', Resource id #56)
#1 /xxx/engine/lib/configuration.php(311): insert_data('INSERT into elg...')
#2 /xxx/engine/lib/cache.php(386): datalist_set('simplecache_las...', 0)
#3 /xxx/engine/lib/upgrades/2011010101.php(68): elgg_invalidate_simplecache()
#4 /xxx/engine/lib/upgrade.php(63): include('/xxx...')
#5 /xxx/engine/lib/upgrade.php(256): upgrade_code(2011052803, false)
#6 /xxx/upgrade.php(36): version_upgrade()
#7 {main}

[code:protected] => 0
[file:protected] => /xxx/engine/lib/database.php
[line:protected] => 274
[trace:Exception:private] => Array
(
[0] => Array
(
[file] => /xxx/engine/lib/database.php
[line] => 468
[function] => execute_query
[args] => Array
(
[0] => INSERT into elgg_datalists set name = 'simplecache_lastupdate_installation', value = '0' ON DUPLICATE KEY UPDATE value='0'
[1] => Resource id #56
)

)

[1] => Array
(
[file] => /xxx/engine/lib/configuration.php
[line] => 311
[function] => insert_data
[args] => Array
(
[0] => INSERT into elgg_datalists set name = 'simplecache_lastupdate_installation', value = '0' ON DUPLICATE KEY UPDATE value='0'
)

)

[2] => Array
(
[file] => /xxx/engine/lib/cache.php
[line] => 386
[function] => datalist_set
[args] => Array
(
[0] => simplecache_lastupdate_installation
[1] => 0
)

)

[3] => Array
(
[file] => /xxx/engine/lib/upgrades/2011010101.php
[line] => 68
[function] => elgg_invalidate_simplecache
[args] => Array
(
)

)

[4] => Array
(
[file] => /xxx/engine/lib/upgrade.php
[line] => 63
[args] => Array
(
[0] => /xxx/engine/lib/upgrades/2011010101.php
)

[function] => include
)

[5] => Array
(
[file] => /xxx/engine/lib/upgrade.php
[line] => 256
[function] => upgrade_code
[args] => Array
(
[0] => 2011052803
[1] =>
)

)

[6] => Array
(
[file] => /xxx/upgrade.php
[line] => 36
[function] => version_upgrade
[args] => Array
(
)

)

)

[previous:Exception:private] =>
)

@ewinslow

This comment has been minimized.

Copy link
Member

commented May 13, 2014

If you go 1.7.x => 1.8.0 => 1.8.19 does that work?

@mrclay

This comment has been minimized.

Copy link
Member

commented May 13, 2014

name is only 32 chars in the 1.7 schema. The schema update needs to happen earlier in the upgrade I guess. For now you could manually increase the size of that column before upgrading.

@iionly

This comment has been minimized.

Copy link
Contributor Author

commented May 13, 2014

I think it also has something to do with simplecache, more precisely never enabling / flushing simplecache on Elgg 1.7 before upgrading to 1.8. I've set up the Elgg 1.7 test installations only to add some test content with some 3rd party plugins then upgrading immediately to Elgg 1.8 as I wanted to test the upgrading of these 3rd party plugins and their content. Therefore, I hadn't bothered with simplycache.

I've tested to upgrade from 1.7.22 to different versions of Elgg 1.8 as I first suspected some issue might have been introduced in later versions of Elgg 1.8 but the error occured each time when I upgraded without touching simplecache settings on Elgg 1.7. When simplecache had been in use on Elgg 1.7 the error did not occur on upgrading to Elgg 1.8 regardless which version of Elgg 1.8 I've upgraded to.

@ewinslow

This comment has been minimized.

Copy link
Member

commented May 13, 2014

OK, so the solution is "turn on simplecache first, then upgrade, then all is well" is that right?

@iionly

This comment has been minimized.

Copy link
Contributor Author

commented May 13, 2014

I think solution is not the right word. Maybe it's rather a "workaround".

@ewinslow

This comment has been minimized.

Copy link
Member

commented May 13, 2014

;). Indeed. So the workaround is to turn on simplecache first. Leaving this open so we can add this to docs or something, but I doubt if anyone will bother actually fixing this since the workaround is trivial.

@iionly

This comment has been minimized.

Copy link
Contributor Author

commented Oct 18, 2014

I don't know why this problem seems to happen only for me... but it happens to me each time I try to upgrade a site from 1.7 to 1.8 and I've given up on trying to figure out which steps in turning on/off caching or deleting the cache folder in the data directory or whatever might help to avoid the problem to occur.

As @mrclay already said the problem is with the 32 characters limit of Elgg 1.7 for the datalist table and the corresponding upgrade script is executed too late during the upgrade.

Today I've tried to "repair" the damage / stuck upgrade by executing the corresponding upgrade script manually:

<?php
$db_prefix = elgg_get_config('dbprefix');
$q = "ALTER TABLE {$db_prefix}datalists CHANGE name name VARCHAR(255)";
update_data($q);
$q = "ALTER TABLE {$db_prefix}config CHANGE name name VARCHAR(255)";
update_data($q);

The upgrade could be completed afterwards. Still this workaround seems not be best way to solve the problem.

@iionly iionly added bug engine labels Oct 18, 2014
@ewinslow

This comment has been minimized.

Copy link
Member

commented Oct 20, 2014

@mrclay, any more insights here?

@jeabakker jeabakker removed their assignment Dec 4, 2014
@mrclay mrclay changed the title Fatal error on upgrading a site from Elgg 1.7.22 to Elgg 1.8.19 1.7 datalists schema prevents upgrade to 1.8 Jan 12, 2015
@mrclay

This comment has been minimized.

Copy link
Member

commented Jan 12, 2015

Simplest solution would be to release a 1.7.23 that does nothing but update the schema for those who want to update.

@iionly

This comment has been minimized.

Copy link
Contributor Author

commented Jan 12, 2015

Would 1.7 continue to work with the updated data scheme when not updated to Elgg 1.8? If yes, it's most likely the easiest (best?) solution. If not, it might happen that some people would just update to 1.7.23 without updating to 1.8 right away - even if it would be clearly stated that the only purpose of the release is to prepare the site for 1.8 - and then might have other issues sooner or later.

The solution with a release 1.7.23 would surely work for me. I'm just worried that it might be a problem for others.

@juho-jaakkola

This comment has been minimized.

Copy link
Member

commented Jan 12, 2015

I ran into similar problem when attempting to upgrade straight from 1.8 to 1.10. My guess is that changes made with ALTER TABLE statement won't take effect until shutdown. We're trying to run all database upgrades within a single request, so all queries using to the new schema will fail.

@iionly

This comment has been minimized.

Copy link
Contributor Author

commented Jan 12, 2015

I've kind of changed my mind regarding "easiest / best" solution after thinking about it a bit more today - especially after reading that it's maybe not a singular issue when upgrading from 1.7 to 1.8 only but possibly occuring on updating to later releases, too.

The problem is not in Elgg 1.7 but it had been introduced with Elgg 1.8. Therefore, I would think that a solution within Elgg 1.8 (and in later versions wherever necessary) is the logical way to fix it - even if this is more complicated from the Elgg core dev point of view. Think about the following: if someone who still runs Elgg 1.7 (e.g. 1.7.22) decides to do the upgrade to Elgg 1.8, how likely is it that he first checks for any new Elgg 1.7 version - especially not knowing that there is a problem in the first place? It's much more likely that he starts directly with taking the latest Elgg 1.8 release available. A pre-upgrade script added to Elgg 1.8 (and later version) to be executed manually and separately before running the real upgrade script could be kind of a workaround, too. But this would surely lack elegance and has the same disadvantage as an additional Elgg 1.7 release would have: how to make people aware that they have to do something different for this specific new version?

Regarding helping with the fix I can't be of much help though not knowing enough about how the upgrade process actually works. Would it be possible to introduce kind of "breaking points" within the chain of upgrade scripts if it fails to work in a single request? Then the upgrade could get continued within a new request when necessary.

What about the upgrade process in general? Is it supposed to be possible to skip minor releases when upgrading (e.g. from 1.8 directly to 1.10 as @juho-jaakkola did)?

If skipping releases should be a possibility, my suggestion would be to make sure that upgrading from Elgg 1.7 to 1.8 or 1.9 or 1.10 is possible and even make it a blocker for the release of the final 1.10.0 to be sure that this works without problems. Why starting from 1.7? Because Elgg 1.8 is currently still supported and in my opinion this support should include the option to upgrade from the previous minor version of Elgg at any time during the support period.

Bonus points if an upgrade path from Elgg 1.0 to any currently supported Elgg version would be possible, too. ;-)

@juho-jaakkola

This comment has been minimized.

Copy link
Member

commented Jan 13, 2015

We do not currently support skipping minor versions (e.g. straight from 1.8 to 1.10).

@iionly

This comment has been minimized.

Copy link
Contributor Author

commented Jan 13, 2015

@juho-jaakkola: I'm aware that the docs say to upgrade only to the next minor release. I just thought it might be a goal already defined / in the works for Elgg to be able to skip minor releases when I read that you upgraded directly from Elgg 1.8 to 1.10.

Nevertheless, the issue here is about the upgrade from Elgg 1.7 to 1.8 not being possible at all. At least this should still made to work somehow.

@iionly

This comment has been minimized.

Copy link
Contributor Author

commented Feb 21, 2015

Any chance for getting a solution on this? Right now the upgrade path from Elgg 1.7 to 1.8 is basically broken for everyone who's not able to fix this problem on his own. And I doubt very much that any non-dev would look for the error in the Elgg upgrade itself. More likely the user thinks that he made something wrong himself.

@mrclay

This comment has been minimized.

Copy link
Member

commented Feb 21, 2015

Let's add a prominent note in the docs that 1.7 needs first to be upgraded to 1.7.23. Obviously no one is interested in putting energy into this and making another 1.8 release.

@ewinslow

This comment has been minimized.

Copy link
Member

commented Feb 22, 2015

+1

On Sat, Feb 21, 2015, 8:48 AM Steve Clay notifications@github.com wrote:

Let's add a prominent note in the docs that 1.7 needs first to be upgraded
to 1.7.23. Obviously no one is interested in putting energy into this and
making another 1.8 release.


Reply to this email directly or view it on GitHub
#6814 (comment).

@iionly

This comment has been minimized.

Copy link
Contributor Author

commented Feb 22, 2015

So, the thing to do is to make a 1.7.23 then instead of a 1.8.21.

What's the filename convention (date?) for upgrade scripts of Elgg 1.7?

@juho-jaakkola

This comment has been minimized.

Copy link
Member

commented Feb 22, 2015

The date is the creation time. The script should be created with lib/uprades/create_upgrade.php.

@iionly

This comment has been minimized.

Copy link
Contributor Author

commented Feb 22, 2015

This script doesn't seem to exist for Elgg 1.7 yet. Also, wouldn't setting a recent date for an upgrade script to be included in Elgg 1.7 collide with the Elgg release number of any newer Elgg version? If the date of the newly added update script would be newer than the "oldest" script of the newer Elgg version, wouldn't it result in the upgrade scripts in between not getting executed?

@juho-jaakkola

This comment has been minimized.

Copy link
Member

commented Feb 23, 2015

wouldn't setting a recent date for an upgrade script to be included in Elgg 1.7 collide with the Elgg release number of any newer Elgg version?

No, the name of the upgrade will be saved to the database. Later when the same Elgg site is being updated to 1.8, the upgrading feature will check the database and run only the ones that haven't been processed yet: https://github.com/Elgg/Elgg/blob/1.8/engine/lib/upgrade.php#L213

@jdalsem

This comment has been minimized.

Copy link
Member

commented Mar 11, 2015

did #8027 fix this issue?

@juho-jaakkola

This comment has been minimized.

Copy link
Member

commented Mar 11, 2015

Yes, I believe so.

@iionly

This comment has been minimized.

Copy link
Contributor Author

commented Mar 17, 2015

It seems it has been forgotten to actually make an Elgg 1.7.23 release. ;-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.