Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correct issue 9509 missing index and column changes in utf8mb4 conversion #9510

Closed

Conversation

@richard67
Copy link
Contributor

commented Mar 22, 2016

Pull Request for Issue #9509
.

Summary of Changes

Add changing of columns and indexes to utf8(mb4) conversion scripts which have been forgotten before.

  • Column user_id of table #__user_keys downsized to length of 150 so it fits to column username of table #__users.
  • Changed column length of column name in table #__users to 400.
  • Changed follwoing indexes (keys) to include column only with 1st 100 chars:
    • TABLE #__menu, KEY idx_path
    • TABLE #__users, KEY idx_name

Column path of table #__menu has already 1024 and was not enlarged in 3.5.0's joomla.sql, and so it is also not enlarged with this PR.

Note: This PR does not solve the problem that the utf8mb4 or utf8 combersion has to be executed again after an update e.g. from 3.5.0 to the 3.5.x in which this PR will be included. This can be manually done by the SQL statement provided in testing instructions below.

A solution for that has to be developed with another PR.

Testing Instructions

Step 1: Apply this patch on latest staging or a 3.5.0 final release, e.g. using the patchtester.

Step 2: In phpMyAdmin, execute the SQL statement from the new schema update script included in this PR:

UPDATE #__utf8_conversion SET converted = 0;

The reason for this step is that this PR does not solve the problem that the utf8mb4 or utf8 combersion has to be executed again after an update e.g. from 3.5.0 to the 3.5.x in which this PR will be included, see description above.

Step 3: Go to "Extensions -> Manage -> Database".

Result: Database is not up to date, utf8mb4 (or utf8) conversion is shown as to be done.

Click the "Fix" button.

Result: Database up to date.

Check with phpMyadmin if the changes mentioned in the Summary of Changes above have been applied.

Result: Changes have been applied.

@richard67 richard67 changed the title Correct utf8 conversion 2016 03 21 Correct issue 9509 missing index and colummn changes in utf8mb4 conversion Mar 22, 2016
@richard67 richard67 changed the title Correct issue 9509 missing index and colummn changes in utf8mb4 conversion Correct issue 9509 missing index and column changes in utf8mb4 conversion Mar 22, 2016
@richard67 richard67 changed the title Correct issue 9509 missing index and column changes in utf8mb4 conversion Correct issue 9509 missing index and column changes in utf8mb4 conversion Mar 22, 2016
@richard67 richard67 referenced this pull request Mar 22, 2016
0 of 4 tasks complete
@ltraveler

This comment has been minimized.

Copy link

commented Mar 22, 2016

I've made update as you u said using custom url the update wasn't finished and at the and i've got a message like that:


ERROR 504 - GATEWAY TIMEOUT

Why am I seeing this page?

The server that your request has reached is acting as a gateway or proxy to fulfil the request made by your client.

Web Browser => Web Front-End => Web Back-End

This server (Web Front-End) received an invalid response from an upstream (Web Back-End) server it accessed to fulfil the request.

In most cases this will not mean that the upstream server is down, but rather that the upstream server and the gateway/proxy do not agree on the protocol for exchanging data.

This problem is most commonly caused when there is a problem with IP communications between the Web Front and Back-Ends. Before you attempt to resolve this problem you should clear your browser cache completely.

Our support staff will be happy to assist you in resolving this issue. Please contact our Live Support or reply to any Tickets you may have received from our technicians for further assistance.


So then I go to Joomla update again and the problem
The Joomla! Core database tables have not been converted yet to UTF-8 Multibyte (utf8mb4).
Still exists and actually I do not know how to fix it...


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9510.

@ltraveler

This comment has been minimized.

Copy link

commented Mar 22, 2016

When I am clicking Fix in Joomla Upgrade after some waiting I am getting an error
405 Not Allowed

nginx/1.8.1

I increased in my php.ihi default_socket_timeout from 90 to 360
and in mysql.connect_timeout from 90 to 360
But it doesn't help. Please help me....


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9510.

@ltraveler

This comment has been minimized.

Copy link

commented Mar 22, 2016

The problem still exists:
The Joomla! Core database tables have not been converted yet to UTF-8 Multibyte (utf8mb4).


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9510.

@wilsonge

This comment has been minimized.

Copy link
Contributor

commented Mar 22, 2016

So this is definitely the fix we need - the problem is that if people upgrade via the file upload approach then we are in trouble cause the schema update (obviously) doesn't check upgrade or insert statements. meaning they won't run this query :(

@richard67

This comment has been minimized.

Copy link
Contributor Author

commented Mar 22, 2016

@wilsonge "So this is definitely the fix we need" .. what do you mean with that? This PR? Or a fix for schema manager detecting inserts and updates, too (I would not recommend this then but maybe a special handling of updates on the utf8_conversion table)?

@wilsonge

This comment has been minimized.

Copy link
Contributor

commented Mar 22, 2016

I mean it fixed the forum post issue that started all these problems :P

@mbabker mbabker referenced this pull request Mar 22, 2016
@andrepereiradasilva

This comment has been minimized.

Copy link
Contributor

commented Mar 22, 2016

I have tested this item successfully on d2bb748

used Method 1: For lazy testers for tests


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9510.

@@ -49,6 +52,8 @@ ALTER TABLE `#__tags` MODIFY `alias` varchar(400) NOT NULL DEFAULT '';
ALTER TABLE `#__ucm_content` MODIFY `core_type_alias` varchar(400) NOT NULL DEFAULT '' COMMENT 'FK to the content types table';
ALTER TABLE `#__ucm_content` MODIFY `core_title` varchar(400) NOT NULL;
ALTER TABLE `#__ucm_content` MODIFY `core_alias` varchar(400) NOT NULL DEFAULT '';
ALTER TABLE `#__users` MODIFY `name` varchar(400) NOT NULL DEFAULT '';
ALTER TABLE `#__user_keys` MODIFY `user_id` varchar(400) NOT NULL;

This comment has been minimized.

Copy link
@wilsonge

wilsonge Mar 22, 2016

Contributor

So this matches the username column which is limited to 150 characters - let's just fix this now and edit this in the joomla.sql and here to make everything "add up"

This comment has been minimized.

Copy link
@richard67

richard67 Mar 22, 2016

Author Contributor

Sorry?

This comment has been minimized.

Copy link
@andrepereiradasilva

andrepereiradasilva Mar 22, 2016

Contributor

yeah joomla.sql needs the change in fields size too.

This comment has been minimized.

Copy link
@richard67

richard67 Mar 22, 2016

Author Contributor

It does not have these sizes yet?

This comment has been minimized.

Copy link
@andrepereiradasilva

andrepereiradasilva Mar 22, 2016

Contributor

oh, you're right sorry.

This comment has been minimized.

Copy link
@richard67

richard67 Mar 22, 2016

Author Contributor

The sizes of the 2 columns above are already 400 in joomla.sql, and the indexes also are already limited to 100. That was the issue, to make the conversion script equal to joomla.sql

This comment has been minimized.

Copy link
@richard67

richard67 Mar 22, 2016

Author Contributor

@wilsonge Do you mean I shall make the username column same as the name column? Why so? It was not same before 3.5.

@joomla-cms-bot

This comment has been minimized.

Copy link

commented Mar 22, 2016

This PR has received new commits.

CC: @andrepereiradasilva


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9510.

For forcing an execution of the utf8 or utf8mb4 conversion after an
update, we have to find another way. Via schema update is not
practicable.
@joomla-cms-bot

This comment has been minimized.

Copy link

commented Mar 23, 2016

This PR has received new commits.

CC: @andrepereiradasilva


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9510.

@richard67

This comment has been minimized.

Copy link
Contributor Author

commented Mar 23, 2016

Removed reset of the conversion status from the schema update because this is 1. not helping in case of update method with file copy and database fix used, and 2. will be disturbing a soon to be developed solution for forcing conversion after an update.

So the former 2nd test method it not requred anymore, and so I have removed it from testing instructions. The remaining test which was former test method 1 is sufficient.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9510.

@richard67

This comment has been minimized.

Copy link
Contributor Author

commented Mar 23, 2016

Ready for re-test.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9510.

@andrepereiradasilva

This comment has been minimized.

Copy link
Contributor

commented Mar 23, 2016

I have tested this item successfully on b914f7d


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9510.

@richard67

This comment has been minimized.

Copy link
Contributor Author

commented Mar 23, 2016

Thanks for testing, @andrepereiradasilva

richard67 added a commit to richard67/joomla-cms that referenced this pull request Mar 23, 2016
These are the changes from my PR joomla#9510
@richard67

This comment has been minimized.

Copy link
Contributor Author

commented Mar 23, 2016

Closed in favour of #9549 .

@richard67 richard67 closed this Mar 23, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.