Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrader - 1.x issues #5752

Closed
wants to merge 10 commits into from
Closed

Upgrader - 1.x issues #5752

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
660b8db
Address no default allowed problems
sbulen Jul 25, 2019
018b5be
log_activity date fixes
sbulen Jul 25, 2019
ebb3eca
Detect prior run
sbulen Jul 25, 2019
60d5488
Check if reserved names exists
sbulen Jul 26, 2019
5082274
Notifies & dates fixes
sbulen Jul 26, 2019
3c32e15
Date fix test
sbulen Jul 26, 2019
4d61a96
Date fixes
sbulen Jul 26, 2019
38ad524
Fix check for message icons
sbulen Jul 26, 2019
bd88331
Fix birthdates
sbulen Jul 26, 2019
edcf776
More year fixes
sbulen Jul 26, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
157 changes: 98 additions & 59 deletions other/upgrade_1-0.sql
View file
Open in desktop
Expand Up @@ -8,16 +8,16 @@
CREATE TABLE IF NOT EXISTS {$db_prefix}themes (
ID_MEMBER mediumint(8) NOT NULL default '0',
ID_THEME tinyint(4) unsigned NOT NULL default '1',
variable tinytext NOT NULL default '',
value text NOT NULL default '',
variable tinytext NOT NULL,
value text NOT NULL,
PRIMARY KEY (ID_MEMBER, ID_THEME, variable(30))
) ENGINE=MyISAM;

ALTER TABLE {$db_prefix}themes
CHANGE COLUMN ID_MEMBER ID_MEMBER mediumint(8) NOT NULL default '0';

ALTER TABLE {$db_prefix}themes
CHANGE COLUMN value value text NOT NULL default '';
CHANGE COLUMN value value text NOT NULL;

INSERT IGNORE INTO {$db_prefix}themes
(ID_MEMBER, ID_THEME, variable, value)
Expand Down Expand Up @@ -108,7 +108,7 @@ CREATE TABLE IF NOT EXISTS {$db_prefix}attachments (
ID_ATTACH int(11) unsigned NOT NULL auto_increment,
ID_MSG int(10) unsigned NOT NULL default '0',
ID_MEMBER int(10) unsigned NOT NULL default '0',
filename tinytext NOT NULL default '',
filename tinytext NOT NULL,
size int(10) unsigned NOT NULL default '0',
downloads mediumint(8) unsigned NOT NULL default '0',
PRIMARY KEY (ID_ATTACH),
Expand Down Expand Up @@ -141,9 +141,9 @@ CREATE TABLE IF NOT EXISTS {$db_prefix}log_actions (
ID_ACTION int(10) unsigned NOT NULL auto_increment,
logTime int(10) unsigned NOT NULL default '0',
ID_MEMBER mediumint(8) unsigned NOT NULL default '0',
IP tinytext NOT NULL default '',
IP tinytext NOT NULL,
action varchar(30) NOT NULL default '',
extra text NOT NULL default '',
extra text NOT NULL,
PRIMARY KEY (ID_ACTION),
KEY logTime (logTime),
KEY ID_MEMBER (ID_MEMBER)
Expand All @@ -154,7 +154,7 @@ CREATE TABLE IF NOT EXISTS {$db_prefix}log_actions (
CREATE TABLE IF NOT EXISTS {$db_prefix}poll_choices (
ID_POLL mediumint(8) unsigned NOT NULL default '0',
ID_CHOICE tinyint(4) unsigned NOT NULL default '0',
label tinytext NOT NULL default '',
label tinytext NOT NULL,
votes smallint(5) unsigned NOT NULL default '0',
PRIMARY KEY (ID_POLL, ID_CHOICE)
) ENGINE=MyISAM;
Expand Down Expand Up @@ -237,7 +237,7 @@ CREATE TABLE {$db_prefix}log_online (
logTime timestamp,
ID_MEMBER mediumint(8) unsigned NOT NULL default '0',
ip int(11) unsigned NOT NULL default '0',
url text NOT NULL default '',
url text NOT NULL,
PRIMARY KEY (session),
KEY online (logTime, ID_MEMBER),
KEY ID_MEMBER (ID_MEMBER)
Expand All @@ -247,7 +247,7 @@ CREATE TABLE {$db_prefix}log_online (
---# Converting "log_floodcontrol"...
DROP TABLE IF EXISTS {$db_prefix}log_floodcontrol;
CREATE TABLE {$db_prefix}log_floodcontrol (
ip tinytext NOT NULL default '',
ip tinytext NOT NULL,
logTime int(10) unsigned NOT NULL default '0',
PRIMARY KEY (ip(16)),
KEY logTime (logTime)
Expand Down Expand Up @@ -311,7 +311,10 @@ if (!empty($replaceRows))

---# Converting "log_activity"...
ALTER TABLE {$db_prefix}log_activity
ADD date date NOT NULL default '0001-01-01';
ADD date date NOT NULL default '1004-01-01';
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

smf 1.0,1.1 and 2.0o can't handle 1004 in the way which 2.1 does,
the years 0000,0001 and in 2.1 1004 are somekind of special "year" which get handled differently.

Copy link
Contributor Author

@sbulen sbulen Jul 26, 2019 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are upgrading to the 2.1 code & logic...

While at the same time trying to navigate a bunch of silly rules... For example, mysql supports a DATE range of '1000-01-01' to '9999-12-31'.
https://dev.mysql.com/doc/refman/5.7/en/datetime.html

So... Where '0000-00-00' used to be an acceptable "null" value for a DATE, it is no longer with all these new STRICT rules of NO_ZERO_DATE and NO_ZERO_IN_DATE. Which, by the way, are going to be deprecated & folded into the definition of STRICT - so they will no longer be independent switches... They will be part of STRICT:
https://dev.mysql.com/doc/refman/5.7/en/sql-mode.html#sqlmode_no_zero_in_date

All the while, they hate '0000-00-00' so much, that you can't even use it in syntax to correct your OLD data (as we have seen in this SMF issue):
https://bugs.mysql.com/bug.php?id=85918

So... '0000-00-00' won't work, and '0001-01-01' is on the way out, we needed a "not specified" value. My understanding of the special SMF 2.1 treatment is that when it sees '1004-01-01' it doesn't display the date. That's why it's the new default.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well you can see this from two side,
the upgrader 1.0 and 1.1 should uprade to his related version number or
since we are here in 2.1 upgrade process we can leave the rule and targeting for 2.1 in every upgrade script.

The laste one would be the best way,
but i understand from other smf members that they don't like the way.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting dilemma, isn't it?

That's why I thought we should consider maybe operating different steps of the upgrader under different sql_modes. Only apply STRICT to the 2.1 step. If we do that, we could get by with many fewer updates. (This would allow us to leave all the 0000-00-00 logic in the 1.0 & 1.1 scripts.)

But that wouldn't fix all of the issues. To me, it appears that upgrade_query() has also changed behavior over the years. A rewrite of upgrade_query would result in many fewer changes needed. (The code appears to assume it will return false when tables/columns aren't found...)

It we want "untouched", universally-sharable 1.0 & 1.1 scripts, we would have to make the changes above to the upgrader.

Even then, there are still changes needed to the 1.0/1,1 scripts, due to confusion between the install & the code (e.g., mostOn vs most_on...).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm not sure how long the non sql mode hack works,
but i see the why this could be a valid way,

another way again would be to drop the 1.x upgrade path in 2.1,
So the admin had to use 2.0.x upgrade to move from 1.x to 2.0 and then use the upgrade of 2.1 to move again...


ALTER TABLE {$db_prefix}log_activity
CHANGE COLUMN date date DATE NOT NULL default '1004-01-01';

ALTER TABLE {$db_prefix}log_activity
DROP PRIMARY KEY;
Expand All @@ -329,6 +332,9 @@ ADD INDEX hits (hits);
ALTER TABLE {$db_prefix}log_activity
ADD PRIMARY KEY (date);

ALTER TABLE {$db_prefix}log_activity
CHANGE COLUMN mostOn most_on smallint(5) unsigned NOT NULL default '0';

ALTER TABLE {$db_prefix}log_activity
CHANGE COLUMN hits hits mediumint(8) unsigned NOT NULL default '0',
CHANGE COLUMN topics topics smallint(5) unsigned NOT NULL default '0',
Expand Down Expand Up @@ -385,7 +391,7 @@ DROP PRIMARY KEY,
CHANGE COLUMN ID_CAT ID_CAT tinyint(4) unsigned NOT NULL default '0',
CHANGE COLUMN numTopics numTopics mediumint(8) unsigned NOT NULL default '0',
CHANGE COLUMN numPosts numPosts mediumint(8) unsigned NOT NULL default '0',
CHANGE COLUMN description description text NOT NULL default '',
CHANGE COLUMN description description text NOT NULL,
CHANGE COLUMN ID_BOARD ID_BOARD smallint(5) unsigned NOT NULL auto_increment PRIMARY KEY;
---#

Expand Down Expand Up @@ -875,7 +881,18 @@ ADD ID_GROUP smallint(5) unsigned NOT NULL default 0,
ADD is_activated tinyint(3) unsigned NOT NULL default '1',
ADD validation_code varchar(10) NOT NULL default '',
ADD ID_MSG_LAST_VISIT int(10) unsigned NOT NULL default '0',
ADD additionalGroups tinytext NOT NULL default '';
ADD additionalGroups tinytext NOT NULL;
---#

---# Fixing birthdates...
UPDATE {$db_prefix}members
SET birthdate = DATE(CONCAT(IF(YEAR(birthdate) < 1004, 1004, YEAR(birthdate)), '-', IF(MONTH(birthdate) < 1, 1, MONTH(birthdate)), '-', IF(DAY(birthdate) < 1, 1, DAY(birthdate))))
WHERE YEAR(birthdate) < 1004;
---#

---# Fixing birthdate default...
ALTER TABLE {$db_prefix}members
CHANGE COLUMN birthdate birthdate DATE NOT NULL default '1004-01-01';
---#

---# Updating columns on "members"...
Expand Down Expand Up @@ -945,28 +962,28 @@ DROP INDEX realName;
ALTER TABLE {$db_prefix}members
CHANGE COLUMN AIM AIM varchar(16) NOT NULL default '',
CHANGE COLUMN YIM YIM varchar(32) NOT NULL default '',
CHANGE COLUMN ICQ ICQ tinytext NOT NULL default '',
CHANGE COLUMN realName realName tinytext NOT NULL default '',
CHANGE COLUMN emailAddress emailAddress tinytext NOT NULL default '',
CHANGE COLUMN ICQ ICQ tinytext NOT NULL,
CHANGE COLUMN realName realName tinytext NOT NULL,
CHANGE COLUMN emailAddress emailAddress tinytext NOT NULL,
CHANGE COLUMN dateRegistered dateRegistered int(10) unsigned NOT NULL default '0',
CHANGE COLUMN passwd passwd varchar(64) NOT NULL default '',
CHANGE COLUMN personalText personalText tinytext NOT NULL default '',
CHANGE COLUMN websiteTitle websiteTitle tinytext NOT NULL default '';
CHANGE COLUMN personalText personalText tinytext NOT NULL,
CHANGE COLUMN websiteTitle websiteTitle tinytext NOT NULL;
---#

---# Changing column sizes on "members" (part 3)...
ALTER TABLE {$db_prefix}members
DROP INDEX lngfile;

ALTER TABLE {$db_prefix}members
CHANGE COLUMN websiteUrl websiteUrl tinytext NOT NULL default '',
CHANGE COLUMN location location tinytext NOT NULL default '',
CHANGE COLUMN avatar avatar tinytext NOT NULL default '',
CHANGE COLUMN im_ignore_list im_ignore_list tinytext NOT NULL default '',
CHANGE COLUMN usertitle usertitle tinytext NOT NULL default '',
CHANGE COLUMN lngfile lngfile tinytext NOT NULL default '',
CHANGE COLUMN MSN MSN tinytext NOT NULL default '',
CHANGE COLUMN memberIP memberIP tinytext NOT NULL default '',
CHANGE COLUMN websiteUrl websiteUrl tinytext NOT NULL,
CHANGE COLUMN location location tinytext NOT NULL,
CHANGE COLUMN avatar avatar tinytext NOT NULL,
CHANGE COLUMN im_ignore_list im_ignore_list tinytext NOT NULL,
CHANGE COLUMN usertitle usertitle tinytext NOT NULL,
CHANGE COLUMN lngfile lngfile tinytext NOT NULL,
CHANGE COLUMN MSN MSN tinytext NOT NULL,
CHANGE COLUMN memberIP memberIP tinytext NOT NULL,
ADD INDEX lngfile (lngfile(24));
---#

Expand Down Expand Up @@ -1082,11 +1099,20 @@ if ($do_it)
$adv_im = smf_mysql_num_rows($request) == 0;
smf_mysql_free_result($request);

upgrade_query("
INSERT IGNORE INTO {$db_prefix}im_recipients
(ID_PM, ID_MEMBER, bcc, is_read, deleted)
SELECT ID_PM, ID_MEMBER_TO, 0, IF(" . (!$adv_im ? 'readBy' : 'alerted') . " != 0, 1, 0), IF(deletedBy = '1', 1, 0)
FROM {$db_prefix}instant_messages");
$request = upgrade_query("
SHOW COLUMNS
FROM {$db_prefix}instant_messages
LIKE 'ID_MEMBER_TO'");
$do_it2 = smf_mysql_num_rows($request) > 0;

if ($do_it2)
{
upgrade_query("
INSERT IGNORE INTO {$db_prefix}im_recipients
(ID_PM, ID_MEMBER, bcc, is_read, deleted)
SELECT ID_PM, ID_MEMBER_TO, 0, IF(" . (!$adv_im ? 'readBy' : 'alerted') . " != 0, 1, 0), IF(deletedBy = '1', 1, 0)
FROM {$db_prefix}instant_messages");
}
}
---}

Expand Down Expand Up @@ -1216,7 +1242,7 @@ if ($result === false)
onlineColor varchar(20) NOT NULL default '',
minPosts mediumint(9) NOT NULL default '-1',
maxMessages smallint(5) unsigned NOT NULL default '0',
stars tinytext NOT NULL default '',
stars tinytext NOT NULL,
PRIMARY KEY (ID_GROUP),
KEY minPosts (minPosts)
) ENGINE=MyISAM");
Expand Down Expand Up @@ -1326,11 +1352,15 @@ if ($result === false)

---# Converting "reserved_names"...
---{
$request = upgrade_query("
SELECT setting, value
FROM {$db_prefix}reserved_names");
if ($request !== false)
$result = upgrade_query("SHOW TABLES LIKE '{$db_prefix}reserved_names'");
$tableExists = smf_mysql_num_rows($result) > 0;

if ($tableExists)
{
$request = upgrade_query("
SELECT setting, value
FROM {$db_prefix}reserved_names");

$words = array();
$match_settings = array();
while ($row = smf_mysql_fetch_assoc($request))
Expand Down Expand Up @@ -1437,12 +1467,14 @@ if (!isset($modSettings['censor_vulgar']) || !isset($modSettings['censor_proper'

---# Converting topic notifications...
---{
$result = upgrade_query("
SELECT COUNT(*)
FROM {$db_prefix}topics
WHERE notifies != ''");
if ($result !== false)
$request = upgrade_query("SHOW COLUMNS FROM {$db_prefix}topics LIKE 'notifies'");
if (smf_mysql_num_rows($request) > 0)
{
$result = upgrade_query("
SELECT COUNT(*)
FROM {$db_prefix}topics
WHERE notifies != ''");

list ($numNotifies) = smf_mysql_fetch_row($result);
smf_mysql_free_result($result);

Expand Down Expand Up @@ -1473,12 +1505,14 @@ DROP notifies;

---# Converting "banned"...
---{
$request = upgrade_query("
SELECT type, value
FROM {$db_prefix}banned
WHERE type = 'ip'");
if ($request !== false)
$request = upgrade_query("SHOW COLUMNS FROM {$db_prefix}banned LIKE 'type'");
if (smf_mysql_num_rows($request) > 0)
{
$request = upgrade_query("
SELECT type, value
FROM {$db_prefix}banned
WHERE type = 'ip'");

$insertEntries = array();
while ($row = smf_mysql_fetch_assoc($request))
{
Expand All @@ -1502,14 +1536,14 @@ if ($request !== false)
ip_high3 tinyint(3) unsigned NOT NULL default '0',
ip_low4 tinyint(3) unsigned NOT NULL default '0',
ip_high4 tinyint(3) unsigned NOT NULL default '0',
hostname tinytext NOT NULL default '',
email_address tinytext NOT NULL default '',
hostname tinytext NOT NULL,
email_address tinytext NOT NULL,
ID_MEMBER mediumint(8) unsigned NOT NULL default '0',
ban_time int(10) unsigned NOT NULL default '0',
expire_time int(10) unsigned,
restriction_type varchar(30) NOT NULL default '',
reason tinytext NOT NULL default '',
notes text NOT NULL default '',
reason tinytext NOT NULL,
notes text NOT NULL,
PRIMARY KEY (id_ban)
) ENGINE=MyISAM");

Expand Down Expand Up @@ -1554,6 +1588,9 @@ ADD INDEX logTime (logTime);
---#

---# Updating columns on "calendar"...
ALTER TABLE {$db_prefix}calendar
CHANGE COLUMN eventDate eventDate DATE NOT NULL default '1004-01-01';

ALTER TABLE {$db_prefix}calendar
DROP PRIMARY KEY,
CHANGE COLUMN id ID_EVENT smallint(5) unsigned NOT NULL auto_increment PRIMARY KEY,
Expand All @@ -1564,7 +1601,7 @@ ALTER TABLE {$db_prefix}calendar
CHANGE COLUMN title title varchar(48) NOT NULL default '';

ALTER TABLE {$db_prefix}calendar
ADD eventDate date NOT NULL default '0000-00-00';
ADD eventDate date NOT NULL default '1004-01-01';
---#

---# Updating indexes on "calendar"...
Expand All @@ -1588,7 +1625,7 @@ ADD INDEX eventDate (eventDate);
---# Updating structure on "calendar_holidays"...
CREATE TABLE IF NOT EXISTS {$db_prefix}calendar_holidays (
ID_HOLIDAY smallint(5) unsigned NOT NULL auto_increment,
eventDate date NOT NULL default '0000-00-00',
eventDate date NOT NULL default '1004-01-01',
title varchar(30) NOT NULL default '',
PRIMARY KEY (ID_HOLIDAY),
KEY eventDate (eventDate)
Expand All @@ -1608,13 +1645,13 @@ if (empty($size))
upgrade_query("
INSERT INTO {$db_prefix}calendar_holidays
(eventDate, title)
SELECT IF(year IS NULL, CONCAT('0000-', month, '-', day), CONCAT(year, '-', month, '-', day)), title
SELECT IF(year IS NULL, CONCAT('1004-', month, '-', day), CONCAT(year, '-', month, '-', day)), title
FROM {$db_prefix}calendar_holiday");

upgrade_query("
INSERT INTO {$db_prefix}calendar_holidays
(eventDate, title)
VALUES ('0000-06-06', 'D-Day')");
VALUES ('1004-06-06', 'D-Day')");
}
---}

Expand Down Expand Up @@ -1671,11 +1708,13 @@ FROM {$db_prefix}polls;

---# Converting data to "log_polls"...
---{
$query = upgrade_query("
SELECT ID_POLL, votedMemberIDs
FROM {$db_prefix}polls");
if ($query !== false)
$request = upgrade_query("SHOW COLUMNS FROM {$db_prefix}polls LIKE 'votedMemberIDs'");
if (smf_mysql_num_rows($request) > 0)
{
$query = upgrade_query("
SELECT ID_POLL, votedMemberIDs
FROM {$db_prefix}polls");

$setStringLog = '';
while ($row = smf_mysql_fetch_assoc($query))
{
Expand Down Expand Up @@ -1705,14 +1744,14 @@ DROP votedMemberIDs,
DROP PRIMARY KEY,
CHANGE COLUMN ID_POLL ID_POLL mediumint(8) unsigned NOT NULL auto_increment PRIMARY KEY,
CHANGE COLUMN votingLocked votingLocked tinyint(1) NOT NULL default '0',
CHANGE COLUMN question question tinytext NOT NULL default '',
CHANGE COLUMN question question tinytext NOT NULL,
ADD maxVotes tinyint(4) unsigned NOT NULL default '1',
ADD expireTime int(10) unsigned NOT NULL default '0',
ADD hideResults tinyint(4) unsigned NOT NULL default '0';

ALTER TABLE {$db_prefix}polls
ADD ID_MEMBER mediumint(8) unsigned NOT NULL default '0',
ADD posterName tinytext NOT NULL default '';
ADD posterName tinytext NOT NULL;
ALTER TABLE {$db_prefix}polls
ADD changeVote tinyint(4) unsigned NOT NULL default '0';
---#
Expand Down