New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

General error: 2006 MySQL server has gone away #121

Closed
ghost opened this Issue Oct 31, 2016 · 41 comments

Comments

Projects
None yet
@ghost
Copy link

ghost commented Oct 31, 2016

Im getting alot of these 'General error: 2006 MySQL server has gone away' on my logs and the nextcloud desktop client give me 'Internal Server error' any fix?

Exception: {"Message":"HTTP/1.1 503 Doctrine\DBAL\Exception\DriverException: An exception occurred while executing 'SELECT * FROM oc_storages WHERE id = ?' with params ["home::xxx"]:\n\nSQLSTATE[HY000]: General error: 2006 MySQL server has gone away","Exception":"Sabre\DAV\Exception\ServiceUnavailable","Code":0,"Trace":"#0 /snap/nextcloud/134/htdocs/3rdparty/sabre/dav/lib/DAV/Auth/Plugin.php(163): OCA\DAV\Connector\Sabre\Auth->check(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))\n#1 [internal function]: Sabre\DAV\Auth\Plugin->beforeMethod(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))\n#2 /snap/nextcloud/134/htdocs/3rdparty/sabre/event/lib/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\n#3 /snap/nextcloud/134/htdocs/3rdparty/sabre/dav/lib/DAV/Server.php(446): Sabre\Event\EventEmitter->emit('beforeMethod', Array)\n#4 /snap/nextcloud/134/htdocs/3rdparty/sabre/dav/lib/DAV/Server.php(248): Sabre\DAV\Server->invokeMethod(Object(Sabre\HTTP\Request), Object(Sabre\HTTP\Response))\n#5 /snap/nextcloud/134/htdocs/apps/dav/appinfo/v1/webdav.php(60): Sabre\DAV\Server->exec()\n#6 /snap/nextcloud/134/htdocs/remote.php(165): require_once('/snap/nextcloud...')\n#7 {main}","File":"/snap/nextcloud/134/htdocs/apps/dav/lib/Connector/Sabre/Auth.php","Line":162,"User":"xxx"}<

@oparoz

This comment has been minimized.

Copy link
Member

oparoz commented Oct 31, 2016

That's just MySQL crashing because it runs out of memory.

@ghost

This comment has been minimized.

Copy link

ghost commented Oct 31, 2016

How do you give it more memory?

@oparoz

This comment has been minimized.

Copy link
Member

oparoz commented Oct 31, 2016

You can't because the Pi2 only has 1GB of RAM. The only option is to use different hardware once Ubuntu Core supports it.

@ghost

This comment has been minimized.

Copy link

ghost commented Oct 31, 2016

I am using nextcloud snap on my ubuntu computer that has 8gb of ram so should i remove the snap and use the normal nextcloud with lamp?

@enoch85

This comment has been minimized.

Copy link
Member

enoch85 commented Oct 31, 2016

@Sehlee You could try the VM: https://github.com/nextcloud/vm/releases

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Oct 31, 2016

@Sehlee the snap should be running fine on such a system. Is there any chance you're running many things on there and thus still running low on ram? I'd like to take a look at why MySQL might be dying. Can I see the output of the following command, please?

$ journalctl -u snap.nextcloud.mysql.service

@kyrofa kyrofa added the needs info label Oct 31, 2016

@MariusBluem

This comment has been minimized.

Copy link
Member

MariusBluem commented Nov 5, 2016

A user in our forum told about the same experience with an Nextcloud Box:
https://help.nextcloud.com/t/mysql-server-has-gone-away-on-nextcloud-box/4841/5

@MariusBluem

This comment has been minimized.

Copy link
Member

MariusBluem commented Nov 5, 2016

Here is the output provided by the user in the Nextcloud forums @kyrofa : https://help.nextcloud.com/t/mysql-server-has-gone-away-on-nextcloud-box/4841/8

@ghost

This comment has been minimized.

Copy link

ghost commented Nov 6, 2016

Sorry i already moved from the snap to the normal nextcloud setup. everything running fine now.

@kyrofa i was not running out of memory i always had around 4gb+ available memory

@eppfel

This comment has been minimized.

Copy link
Member

eppfel commented Nov 10, 2016

So, I have the box with RPi2 and the (same) issue, although the Internal Server Error only occurs occasionally. Cannot really pin down one particular use-case.
If found "MySQL server has gone away" in the log, but on different requests.
And journalctl -u snap.nextcloud.mysql.service repeatedly shows something like this:

Nov 10 08:57:26 ubuntu-standard systemd[1]: snap.nextcloud.mysql.service: State 'stop-sigterm' timed out. Killing.
Nov 10 08:57:26 ubuntu-standard systemd[1]: snap.nextcloud.mysql.service: Unit entered failed state.
Nov 10 08:57:26 ubuntu-standard systemd[1]: snap.nextcloud.mysql.service: Failed with result 'timeout'.
Nov 10 08:57:26 ubuntu-standard systemd[1]: snap.nextcloud.mysql.service: Service hold-off time over, scheduling restart.
Nov 10 08:57:26 ubuntu-standard systemd[1]: Stopped Service for snap application nextcloud.mysql.
Nov 10 08:57:26 ubuntu-standard systemd[1]: Started Service for snap application nextcloud.mysql.
Nov 10 08:57:26 ubuntu-standard ubuntu-core-launcher[20542]: 2016-11-10T08:57:26.453959Z 0 [Warning] Changed limits: max_open_files: 1024 (requested 5000)
Nov 10 08:57:26 ubuntu-standard ubuntu-core-launcher[20542]: 2016-11-10T08:57:26.454647Z 0 [Warning] Changed limits: table_open_cache: 431 (requested 2000)
Nov 10 08:57:26 ubuntu-standard ubuntu-core-launcher[20542]: 2016-11-10T08:57:26.456062Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use 
Nov 10 08:57:26 ubuntu-standard ubuntu-core-launcher[20542]: 2016-11-10T08:57:26.465592Z 0 [ERROR] --initialize specified but the data directory has files in it. Abo
Nov 10 08:57:26 ubuntu-standard ubuntu-core-launcher[20542]: 2016-11-10T08:57:26.465861Z 0 [ERROR] Aborting
Nov 10 08:57:26 ubuntu-standard ubuntu-core-launcher[20542]: Starting MySQL
Nov 10 08:57:26 ubuntu-standard ubuntu-core-launcher[20542]: .
Nov 10 08:57:26 ubuntu-standard ubuntu-core-launcher[20542]: Waiting for server...
Nov 10 08:57:26 ubuntu-standard ubuntu-core-launcher[20542]: Checking/upgrading mysql tables if necessary...
Nov 10 08:57:26 ubuntu-standard ubuntu-core-launcher[20542]: mysql_upgrade: Got error: 2002: Can't connect to local MySQL server through socket '/var/snap/nextcloud/
Nov 10 08:57:26 ubuntu-standard ubuntu-core-launcher[20542]: Upgrade process encountered error and will not continue.
Nov 10 08:57:26 ubuntu-standard ubuntu-core-launcher[20567]: MySQL server process #20452 is not running! ... failed!

If this is really due to the RAM, that's pretty bad user experience.

@hixhupf

This comment has been minimized.

Copy link

hixhupf commented Nov 10, 2016

I'm the one with this same problem mentioned by MariusBluem. Also on RPi2. I have these error messages now since nearly a week, setup is not useable any more. Are there any settings / config files that I can tweak so that MySQL gets more resources? Or any logs I can check why it fails to start? It worked fine without these problems for about 3 weeks.

I'm happy to provide any log file if this helps.

I'm kind of familiar with Linux, but not with snaps. What seems weird to me is this:

ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/snap/nextcloud/48/mysql/mysql.sock' (2)

It mentions directory ../48/... but if I go there I can see there are more recent ones:

ubuntu@ubuntu-standard:/var/snap/nextcloud$ ls -rtl
total 16
drwxr-xr-x 3 root root 4096 Aug 24 04:05 common
drwxr-xr-x 8 root root 4096 Aug 24 04:05 48
drwxr-xr-x 8 root root 4096 Aug 24 04:05 136
drwxr-xr-x 8 root root 4096 Aug 24 04:05 128
lrwxrwxrwx 1 root root    3 Oct 17 17:10 current -> 136

Is something else broken and it is just trying to launch the wrong MySQL instance? Shouldn't it do something in the current directory (136 in my case)?

Sascha

@hixhupf

This comment has been minimized.

Copy link

hixhupf commented Nov 14, 2016

I have restarted from scratch and followed these guidelines:
https://github.com/nextcloud/nextcloud-snap/wiki/How-to-restart-from-scratch

It looks like my setup works again, will verify it in the next hours. I still see this error for MySQL:

Nov 14 09:19:58 ubuntu-standard systemd[1]: Started Service for snap application nextcloud.mysql.
Nov 14 09:20:05 ubuntu-standard snap[956]: 2016-11-14T09:20:04.610974Z 0 [Warning] Changed limits: max_open_files: 1024 (requested 5000)
Nov 14 09:20:05 ubuntu-standard snap[956]: 2016-11-14T09:20:04.611587Z 0 [Warning] Changed limits: table_open_cache: 431 (requested 2000)
Nov 14 09:20:05 ubuntu-standard snap[956]: 2016-11-14T09:20:04.685378Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
Nov 14 09:20:05 ubuntu-standard snap[956]: 2016-11-14T09:20:05.328725Z 0 [ERROR] --initialize specified but the data directory has files in it. Aborting.
Nov 14 09:20:05 ubuntu-standard snap[956]: 2016-11-14T09:20:05.328969Z 0 [ERROR] Aborting
Nov 14 09:20:06 ubuntu-standard snap[956]: Starting MySQL
Nov 14 09:20:10 ubuntu-standard snap[956]: .....
Nov 14 09:20:10 ubuntu-standard snap[956]: Waiting for server...
Nov 14 09:20:10 ubuntu-standard snap[956]: Checking/upgrading mysql tables if necessary...
Nov 14 09:20:11 ubuntu-standard snap[956]: mysql_upgrade: Got error: 2002: Can't connect to local MySQL server through socket '/var/snap/nextcloud/48/mysql/mysql.sock' (2) while connecting to the MySQL server
Nov 14 09:20:11 ubuntu-standard snap[956]: Upgrade process encountered error and will not continue.

And my syslog is full with this (every second a new message):

Nov 14 09:29:15 ubuntu-standard kernel: [  520.829371] audit: type=1400 audit(1479115755.736:715): apparmor="DENIED" operation="open" profile="snap.nextcloud.mysql" name="/etc/ld.so.preload" pid=2223 comm="sleep" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Nov 14 09:29:16 ubuntu-standard kernel: [  521.838219] audit: type=1400 audit(1479115756.744:716): apparmor="DENIED" operation="open" profile="snap.nextcloud.mysql" name="/etc/ld.so.preload" pid=2224 comm="sleep" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Nov 14 09:29:17 ubuntu-standard kernel: [  522.847088] audit: type=1400 audit(1479115757.752:717): apparmor="DENIED" operation="open" profile="snap.nextcloud.mysql" name="/etc/ld.so.preload" pid=2225 comm="sleep" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Nov 14 09:29:18 ubuntu-standard kernel: [  523.856136] audit: type=1400 audit(1479115758.760:718): apparmor="DENIED" operation="open" profile="snap.nextcloud.mysql" name="/etc/ld.so.preload" pid=2226 comm="sleep" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Nov 14 09:29:19 ubuntu-standard kernel: [  524.865044] audit: type=1400 audit(1479115759.772:719): apparmor="DENIED" operation="open" profile="snap.nextcloud.mysql" name="/etc/ld.so.preload" pid=2227 comm="sleep" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Nov 14 09:29:20 ubuntu-standard kernel: [  525.873778] audit: type=1400 audit(1479115760.780:720): apparmor="DENIED" operation="open" profile="snap.nextcloud.mysql" name="/etc/ld.so.preload" pid=2228 comm="sleep" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Nov 14 09:29:21 ubuntu-standard kernel: [  526.882580] audit: type=1400 audit(1479115761.788:721): apparmor="DENIED" operation="open" profile="snap.nextcloud.mysql" name="/etc/ld.so.preload" pid=2229 comm="sleep" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Nov 14 09:29:22 ubuntu-standard kernel: [  527.891353] audit: type=1400 audit(1479115762.796:722): apparmor="DENIED" operation="open" profile="snap.nextcloud.mysql" name="/etc/ld.so.preload" pid=2230 comm="sleep" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Nov 14 09:29:23 ubuntu-standard kernel: [  528.900290] audit: type=1400 audit(1479115763.808:723): apparmor="DENIED" operation="open" profile="snap.nextcloud.mysql" name="/etc/ld.so.preload" pid=2231 comm="sleep" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Nov 14 09:29:24 ubuntu-standard kernel: [  529.909190] audit: type=1400 audit(1479115764.816:724): apparmor="DENIED" operation="open" profile="snap.nextcloud.mysql" name="/etc/ld.so.preload" pid=2232 comm="sleep" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Nov 14 09:29:25 ubuntu-standard kernel: [  530.918183] audit: type=1400 audit(1479115765.824:725): apparmor="DENIED" operation="open" profile="snap.nextcloud.mysql" name="/etc/ld.so.preload" pid=2234 comm="sleep" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Nov 14 09:29:26 ubuntu-standard kernel: [  531.927338] audit: type=1400 audit(1479115766.832:726): apparmor="DENIED" operation="open" profile="snap.nextcloud.mysql" name="/etc/ld.so.preload" pid=2235 comm="sleep" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Nov 14 09:29:27 ubuntu-standard kernel: [  532.935633] audit: type=1400 audit(1479115767.840:727): apparmor="DENIED" operation="open" profile="snap.nextcloud.mysql" name="/etc/ld.so.preload" pid=2237 comm="sleep" requested_mask="r" denied_mask="r" fsuid=0 ouid=0
Nov 14 09:29:28 ubuntu-standard kernel: [  533.942459] audit: type=1400 audit(1479115768.848:728): apparmor="DENIED" operation="open" profile="snap.nextcloud.mysql" name="/etc/ld.so.preload" pid=2238 comm="sleep" requested_mask="r" denied_mask="r" fsuid=0 ouid=0

Regards,
Sascha

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Nov 21, 2016

I still see this error for MySQL

Note that this is not an error:

Nov 14 09:20:05 ubuntu-standard snap[956]: 2016-11-14T09:20:05.328725Z 0 [ERROR] --initialize specified but the data directory has files in it. Aborting.
Nov 14 09:20:05 ubuntu-standard snap[956]: 2016-11-14T09:20:05.328969Z 0 [ERROR] Aborting

It's just mysql noticing it doesn't need to initialize itself. This, though:

Nov 14 09:20:10 ubuntu-standard snap[956]: Waiting for server...
Nov 14 09:20:10 ubuntu-standard snap[956]: Checking/upgrading mysql tables if necessary...
Nov 14 09:20:11 ubuntu-standard snap[956]: mysql_upgrade: Got error: 2002: Can't connect to local MySQL server through socket '/var/snap/nextcloud/48/mysql/mysql.sock' (2) while connecting to the MySQL server
Nov 14 09:20:11 ubuntu-standard snap[956]: Upgrade process encountered error and will not continue.

Is weird. The "waiting for server" bit actually waits for the PID file and socket to be back in place and then runs some commands. It's odd that even though mysql places the PID file and creates the socket it's not available. I'm not sure what the deal is, there, I need to investigate.

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Nov 21, 2016

And my syslog is full with this (every second a new message):

Please make sure your system is completely up-to-date. Your snap-confine might be old.

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Nov 21, 2016

I'm kind of familiar with Linux, but not with snaps. What seems weird to me is this:

ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/snap/nextcloud/48/mysql/mysql.sock' (2)

It mentions directory ../48/... but if I go there I can see there are more recent ones:

snip

Is something else broken and it is just trying to launch the wrong MySQL instance? Shouldn't it do something in the current directory (136 in my case)?

Great catch @hixhupf! Can you please paste in the contents of /var/snaps/nextcloud/current/mysql/root.ini? Note that there's a password there, mask it out.

@hixhupf

This comment has been minimized.

Copy link

hixhupf commented Nov 22, 2016

Here is the contents of my root.ini:

[client]
socket=/var/snap/nextcloud/48/mysql/mysql.sock
user=root
password=*****

Looks wrong to me. My current mysqld instance is running here:

root 1498 0.7 14.2 1218680 134244 ? Sl Nov21 10:05 /snap/nextcloud/173/bin/mysqld --basedir=/snap/nextcloud/173 --datadir=/var/snap/nextcloud/173/mysql --plugin-dir=/snap/nextcloud/173/lib/plugin --lc-messages-dir=/snap/nextcloud/173/share --log-error=/var/snap/nextcloud/173/mysql/ubuntu-standard.err --pid-file=/var/snap/nextcloud/173/mysql/ubuntu-standard.pid --socket=/var/snap/nextcloud/173/mysql/mysql.sock

Shall I change this file or do I risk to break something?

@eppfel

This comment has been minimized.

Copy link
Member

eppfel commented Nov 22, 2016

Same here:
root.ini

[client]
socket=/var/snap/nextcloud/48/mysql/mysql.sock
...

nextcloud versions:

ubuntu@ubuntu-standard:/var/snap/nextcloud$ ls
136  162  173  common  current

@eppfel eppfel added the bug label Nov 22, 2016

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Nov 22, 2016

@hixhupf @eppfel indeed, that file is wrong. It looks like in an old revision somewhere that file was generated with SNAP_DATA instead of using the current symlink. It's been fixed since then, but apparently with no suitable migration which surprises me. The fix is easy though: change socket=/var/snap/nextcloud/48/mysql/mysql.sock to socket=/var/snap/nextcloud/current/mysql/mysql.sock

@eppfel

This comment has been minimized.

Copy link
Member

eppfel commented Nov 23, 2016

Still this:

Nov 23 23:11:37 ubuntu-standard systemd[1]: snap.nextcloud.mysql.service: State 'stop-sigterm' timed out. Killing.
Nov 23 23:11:37 ubuntu-standard systemd[1]: snap.nextcloud.mysql.service: Unit entered failed state.
Nov 23 23:11:37 ubuntu-standard systemd[1]: snap.nextcloud.mysql.service: Failed with result 'timeout'.
Nov 23 23:11:37 ubuntu-standard systemd[1]: snap.nextcloud.mysql.service: Service hold-off time over, scheduling restart.
Nov 23 23:11:37 ubuntu-standard systemd[1]: Stopped Service for snap application nextcloud.mysql.
Nov 23 23:11:37 ubuntu-standard systemd[1]: Started Service for snap application nextcloud.mysql.
Nov 23 23:11:37 ubuntu-standard snap[18936]: 2016-11-23T23:11:37.964267Z 0 [Warning] Changed limits: max_open_files: 1024 (requested 5000)
Nov 23 23:11:37 ubuntu-standard snap[18936]: 2016-11-23T23:11:37.965065Z 0 [Warning] Changed limits: table_open_cache: 431 (requested 2000)
Nov 23 23:11:37 ubuntu-standard snap[18936]: 2016-11-23T23:11:37.966513Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see docume
Nov 23 23:11:37 ubuntu-standard snap[18936]: 2016-11-23T23:11:37.975960Z 0 [ERROR] --initialize specified but the data directory has files in it. Aborting.
Nov 23 23:11:37 ubuntu-standard snap[18936]: 2016-11-23T23:11:37.976229Z 0 [ERROR] Aborting
Nov 23 23:11:38 ubuntu-standard snap[18936]: Starting MySQL
Nov 23 23:11:38 ubuntu-standard snap[18936]: .
Nov 23 23:11:38 ubuntu-standard snap[18936]: Waiting for server...
Nov 23 23:11:38 ubuntu-standard snap[18936]: Checking/upgrading mysql tables if necessary...
Nov 23 23:11:38 ubuntu-standard snap[18936]: mysql_upgrade: Got error: 2002: Can't connect to local MySQL server through socket '/var/snap/nextcloud/current/mysql/mysql.sock' (111) while connecting to the MyS
Nov 23 23:11:38 ubuntu-standard snap[18936]: Upgrade process encountered error and will not continue.
Nov 23 23:11:38 ubuntu-standard snap[18970]: MySQL server process #18904 is not running! ... failed!
@nehrka

This comment has been minimized.

Copy link

nehrka commented Dec 2, 2016

I'm new to Snaps but gave it a try on my home server. After todays upgrade of the underlying ubuntu 16.04 VM im getting the following, when calling: nextcloud.mysql-client

mysql: [ERROR] Could not open required defaults file: /var/snap/nextcloud/169/mysql/root.ini
mysql: [ERROR] Fatal error in defaults handling. Program aborted!

When calling

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Dec 2, 2016

@nehrka did you try this as well? Also, did you run that command with sudo?

@nehrka

This comment has been minimized.

Copy link

nehrka commented Dec 2, 2016

@kyrofa Ok thanks, with sudo I can call the mysql console. Sorry my fault.

@BodoWitcha

This comment has been minimized.

Copy link

BodoWitcha commented Dec 7, 2016

My Nextcloud Box is still a brick for more then a month with the above mentioned issue.
This post states it is a 'bug' and 'needs info'.
What kind of info is needed and who should provide this info to resolve this issue?

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Dec 10, 2016

@BodoWitcha me, I suppose. My first recommendation is to make sure the .ini file is correct as above. If it's still crashing it's likely due to RAM, as also pointed out. The solution is likely an optimization to lower RAM utilization. I'm not a MySQL pro though, I'm afraid, so I'm a bit slow on it.

@kyrofa kyrofa removed the needs info label Dec 10, 2016

@BodoWitcha

This comment has been minimized.

Copy link

BodoWitcha commented Dec 10, 2016

I have changed the init file as suggested, but the too much RAM usage is probably still the reason of the issue.
Because the Nextcloud-Box is targeting the non ICT literate user, a correct working update process should be preferred as the solution.

@adamlove86

This comment has been minimized.

Copy link

adamlove86 commented Dec 12, 2016

Hi there, I've managed to fix the problem I was having with this error - see nextcloud/server#2607

Strange. It suddenly worked today when I tried to get the error logs to show - after many tries & restarts yesterday. I'd also found the /snap/nextcloud/169/my.cnf and changed max_allowed_packet= to 100M (instead of 10). Perhaps it needed some time to think & grumble about it before it actually implemented the 100M?
My question now - shall I drop it back down to 10, or even another amount?

Hope that helps you guys too!

@adamlove86

This comment has been minimized.

Copy link

adamlove86 commented Dec 16, 2016

Update -
I've had to jump back to a previous installation, and then upgrade again. This way seems a lot less hassle free:

to update nextcloud:
nextcloud.occ maintenance:mode --on
sudo nextcloud upgrade
It seems that "nextcloud upgrade" as root user is not enough, I needed to write sudo nextcloud upgrade as root, and then I get no "2006 MySQL server has gone away" error
Now to finish off:
nextcloud.occ maintenance:mode --off

I also upgraded the ram temporarily for the instance I run, and then dropped it down again post upgrade.

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Dec 20, 2016

For those of you with a log like this:

Nov 23 23:11:38 ubuntu-standard snap[18936]: Waiting for server...
Nov 23 23:11:38 ubuntu-standard snap[18936]: Checking/upgrading mysql tables if necessary...
Nov 23 23:11:38 ubuntu-standard snap[18936]: mysql_upgrade: Got error: 2002: Can't connect to local MySQL server through socket '/var/snap/nextcloud/current/mysql/mysql.sock' (111) while connecting to the MyS
Nov 23 23:11:38 ubuntu-standard snap[18936]: Upgrade process encountered error and will not continue.
Nov 23 23:11:38 ubuntu-standard snap[18970]: MySQL server process #18904 is not running! ... failed!

Where it specifically waits for mysql and then complains about mysql not running: try this and tell me if you have more luck:

$ sudo snap disable nextcloud
$ rm -f /var/snap/nextcloud/<latest rev>/mysql/mysql.sock
$ rm -f /var/snap/nextcloud/<latest rev>/mysql/*.pid
$ sudo snap enable nextcloud
@vayan

This comment has been minimized.

Copy link

vayan commented Dec 22, 2016

try this and tell me if you have more luck:

Yeah this work for me, I have to do it on every reboot, duplicated of #151 I think

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Dec 22, 2016

@vayan excellent, thank you!

Then there are two issues here: MySQL using too much RAM (which I hope can be fixed with some optimization) and MySQL not coming back after dying because it used too much RAM (i.e. it should respawn immediately). I'll treat this bug as the first, and #151 as the second.

@adamlove86

This comment has been minimized.

Copy link

adamlove86 commented Jan 3, 2017

My earlier post in this thread was as a result of experiencing issues upgrading to 10.0.2 (was receiving error whilst upgrading in my console).

My approach now has been to remove the snap and fresh install any time I get an error I can't seem to easily resolve, but it seems that a few hours into syncing the new instance the 2006 error is popping up in my log again. The nextcloud mac app then stops syncing (says server replied - Internal Server Error).

Here's my log from my nextcloud admin:

Error	index	Exception: {"Exception":"Doctrine\\DBAL\\Exception\\DriverException","Message":"An exception occurred while executing 'UPDATE `oc_filecache` SET `size` = `size` + ? WHERE (`storage` = ?) AND (`path_hash` IN ('d41d8cd98f00b204e9800998ecf8427e', '3b8779ba05b8f0aed49650f3ff8beb4b', '51ac8400ec84acc670e1691540980c57')) AND (`size` > '-1')' with params [1746995, 1]:\n\nSQLSTATE[HY000]: General error: 2006 MySQL server has gone away","Code":0,"Trace":"#0 \/snap\/nextcloud\/383\/htdocs\/3rdparty\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/DBALException.php(116): Doctrine\\DBAL\\Driver\\AbstractMySQLDriver->convertException('An exception oc...', Object(Doctrine\\DBAL\\Driver\\PDOException))\n#1 \/snap\/nextcloud\/383\/htdocs\/3rdparty\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Connection.php(996): Doctrine\\DBAL\\DBALException::driverExceptionDuringQuery(Object(Doctrine\\DBAL\\Driver\\PDOMySql\\Driver), Object(Doctrine\\DBAL\\Driver\\PDOException), 'UPDATE `oc_file...', Array)\n#2 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/DB\/Connection.php(210): Doctrine\\DBAL\\Connection->executeUpdate('UPDATE `oc_file...', Array, Array)\n#3 \/snap\/nextcloud\/383\/htdocs\/3rdparty\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Query\/QueryBuilder.php(208): OC\\DB\\Connection->executeUpdate('UPDATE `oc_file...', Array, Array)\n#4 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/DB\/QueryBuilder\/QueryBuilder.php(142): Doctrine\\DBAL\\Query\\QueryBuilder->execute()\n#5 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/Files\/Cache\/Propagator.php(100): OC\\DB\\QueryBuilder\\QueryBuilder->execute()\n#6 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/Files\/Cache\/HomePropagator.php(49): OC\\Files\\Cache\\Propagator->propagateChange('thumbnails\/9690...', 1483478732, 1746995)\n#7 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/Files\/Cache\/Updater.php(138): OC\\Files\\Cache\\HomePropagator->propagateChange('thumbnails\/9690...', 1483478732, 1746995)\n#8 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/Files\/View.php(315): OC\\Files\\Cache\\Updater->update('thumbnails\/9690...', 1483478732)\n#9 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/Files\/View.php(1135): OC\\Files\\View->writeUpdate(Object(OCA\\FilesAccessControl\\StorageWrapper), 'thumbnails\/9690...')\n#10 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/Files\/View.php(672): OC\\Files\\View->basicOperation('file_put_conten...', '\/thumbnails\/969...', Array, '\\x89PNG\\r\\n\\x1A\\n\\x00\\x00\\x00\\rIHD...')\n#11 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/Preview.php(1220): OC\\Files\\View->file_put_contents('thumbnails\/9690...', '\\x89PNG\\r\\n\\x1A\\n\\x00\\x00\\x00\\rIHD...')\n#12 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/Preview.php(1162): OC\\Preview->storeMaxPreview('thumbnails\/9690...')\n#13 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/Preview.php(777): OC\\Preview->generatePreview(9690)\n#14 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/PreviewManager.php(120): OC\\Preview->getPreview()\n#15 \/snap\/nextcloud\/383\/htdocs\/apps\/files\/lib\/Controller\/ApiController.php(102): OC\\PreviewManager->createPreview('files\/Cognitive...', 512, 512, true)\n#16 [internal function]: OCA\\Files\\Controller\\ApiController->getThumbnail(512, 512, 'Cognitive Bias....')\n#17 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/AppFramework\/Http\/Dispatcher.php(160): call_user_func_array(Array, Array)\n#18 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/AppFramework\/Http\/Dispatcher.php(90): OC\\AppFramework\\Http\\Dispatcher->executeController(Object(OCA\\Files\\Controller\\ApiController), 'getThumbnail')\n#19 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/AppFramework\/App.php(111): OC\\AppFramework\\Http\\Dispatcher->dispatch(Object(OCA\\Files\\Controller\\ApiController), 'getThumbnail')\n#20 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/AppFramework\/Routing\/RouteActionHandler.php(47): OC\\AppFramework\\App::main('APIController', 'getThumbnail', Object(OC\\AppFramework\\DependencyInjection\\DIContainer), Array)\n#21 [internal function]: OC\\AppFramework\\Routing\\RouteActionHandler->__invoke(Array)\n#22 \/snap\/nextcloud\/383\/htdocs\/lib\/private\/Route\/Router.php(293): call_user_func(Object(OC\\AppFramework\\Routing\\RouteActionHandler), Array)\n#23 \/snap\/nextcloud\/383\/htdocs\/lib\/base.php(983): OC\\Route\\Router->match('\/apps\/files\/api...')\n#24 \/snap\/nextcloud\/383\/htdocs\/index.php(48): OC::handleRequest()\n#25 {main}","File":"\/snap\/nextcloud\/383\/htdocs\/3rdparty\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Driver\/AbstractMySQLDriver.php","Line":116}	2017-01-03T21:25:32+00:00	adam

Error	PHP	PDOStatement::execute(): Error reading result set's header at /snap/nextcloud/383/htdocs/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php#91	2017-01-03T21:25:32+00:00	adam

Error	PHP	PDOStatement::execute(): MySQL server has gone away at /snap/nextcloud/383/htdocs/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php#91	2017-01-03T21:25:32+00:00	adam
Fatal	webdav	Exception: {"Message":"HTTP\/1.1 503 Doctrine\\DBAL\\Exception\\DriverException: An exception occurred while executing 'SELECT `uid`, `displayname` FROM `oc_users` WHERE LOWER(`uid`) = LOWER(?)' with params [\"adam\"]:\n\nSQLSTATE[HY000]: General error: 2006 MySQL server has gone away","Exception":"Sabre\\DAV\\Exception\\ServiceUnavailable","Code":0,"Trace":"#0 [internal function]: {closure}(Object(Sabre\\HTTP\\Request), Object(Sabre\\HTTP\\Response))\n#1 \/snap\/nextcloud\/383\/htdocs\/3rdparty\/sabre\/event\/lib\/EventEmitterTrait.php(105): call_user_func_array(Object(Closure), Array)\n#2 \/snap\/nextcloud\/383\/htdocs\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php(446): Sabre\\Event\\EventEmitter->emit('beforeMethod', Array)\n#3 \/snap\/nextcloud\/383\/htdocs\/3rdparty\/sabre\/dav\/lib\/DAV\/Server.php(248): Sabre\\DAV\\Server->invokeMethod(Object(Sabre\\HTTP\\Request), Object(Sabre\\HTTP\\Response))\n#4 \/snap\/nextcloud\/383\/htdocs\/remote.php(70): Sabre\\DAV\\Server->exec()\n#5 \/snap\/nextcloud\/383\/htdocs\/remote.php(168): handleException(Object(Doctrine\\DBAL\\Exception\\DriverException))\n#6 {main}","File":"\/snap\/nextcloud\/383\/htdocs\/remote.php","Line":68,"User":"adam"}	2017-01-03T21:19:56+00:00	adam

Error	PHP	Error while sending QUERY packet. PID=11238 at /snap/nextcloud/383/htdocs/3rdparty/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php#91	2017-01-03T21:19:56+00:00

This is probably the best I've done so far, with only 4 errors in this new snap, and running on 512MB RAM (am running a DreamCompute instance which allows you to upgrade/downgrade RAM easily). Have just stepped it up to 1024MB RAM but have received the same error with the app trying to sync a different file.

Hopefully this is useful for you; let me know if there's any other info I can provide.

@ernesst

This comment has been minimized.

Copy link

ernesst commented Mar 1, 2017

I got the issue after an hard reboot of the raspberry pi.
After deleting the .sock and the .pid, no more problem with mysql server.

I'm currently monitoring the mysql server with nagios, i'll keep update.

Thanks for the help @kyrofa

@ernesst

This comment has been minimized.

Copy link

ernesst commented Mar 10, 2017

The issue comes back :

Mar 09 15:09:22 nextcloud systemd[1]: Stopping Service for snap application nextcloud.mysql...
Mar 09 15:09:22 nextcloud /usr/bin/snap[27982]: cmd.go:107: DEBUG: not restarting into "/snap/core/current/usr/bin/snap" ([VERSION=2.22.5 2.22.
Mar 09 15:09:23 nextcloud snap[27982]: Shutting down MySQL
Mar 09 15:09:30 nextcloud snap[27982]: ....... *
Mar 09 15:09:30 nextcloud systemd[1]: Stopped Service for snap application nextcloud.mysql.
Mar 09 15:10:31 nextcloud systemd[1]: Started Service for snap application nextcloud.mysql.
Mar 09 15:10:31 nextcloud /usr/bin/snap[28357]: cmd.go:107: DEBUG: not restarting into "/snap/core/current/usr/bin/snap" ([VERSION=2.22.5 2.22.
Mar 09 15:10:33 nextcloud snap[28357]: 2017-03-09T15:10:33.299442Z 0 [Warning] Changed limits: max_open_files: 1024 (requested 5000)
Mar 09 15:10:33 nextcloud snap[28357]: 2017-03-09T15:10:33.322966Z 0 [Warning] Changed limits: table_open_cache: 431 (requested 2000)
Mar 09 15:10:33 nextcloud snap[28357]: 2017-03-09T15:10:33.392567Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use
Mar 09 15:10:33 nextcloud snap[28357]: 2017-03-09T15:10:33.897806Z 0 [ERROR] --initialize specified but the data directory has files in it. Abo
Mar 09 15:10:33 nextcloud snap[28357]: 2017-03-09T15:10:33.898155Z 0 [ERROR] Aborting
Mar 09 15:10:34 nextcloud snap[28357]: Starting MySQL
Mar 09 15:10:40 nextcloud snap[28357]: ..... *
Mar 09 15:10:40 nextcloud snap[28357]: Checking/upgrading mysql tables if necessary...
Mar 09 15:10:40 nextcloud snap[28357]: mysql_upgrade: Got error: 2002: Can't connect to local MySQL server through socket '/var/snap/nextcloud/
Mar 09 15:10:40 nextcloud snap[28357]: Upgrade process encountered error and will not continue.

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Mar 10, 2017

@ernesst what version of the snap are you on (snap list nextcloud)?

@ernesst

This comment has been minimized.

Copy link

ernesst commented Mar 10, 2017

snap list nextcloud
Name Version Rev Developer Notes
nextcloud 11.0.2snap1 1161 nextcloud -

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Mar 10, 2017

It's looking in the wrong place for the MySQL socket. The snap upgrade should have migrated that for you (I tried to test that pretty thoroughly). Can I see the contents of /var/snap/nextcloud/current/nextcloud/config/config.php, please? Mask sensitive values.

@ernesst

This comment has been minimized.

Copy link

ernesst commented Mar 10, 2017

<?php
$CONFIG = array (
  'apps_paths' => 
  array (
    0 => 
    array (
      'path' => '/snap/nextcloud/current/htdocs/apps',
      'url' => '/apps',
      'writable' => false,
    ),
    1 => 
    array (
      'path' => '/var/snap/nextcloud/current/nextcloud/extra-apps',
      'url' => '/extra-apps',
      'writable' => true,
    ),
  ),
  'supportedDatabases' => 
  array (
    0 => 'mysql',
  ),
  'memcache.locking' => '\\OC\\Memcache\\Redis',
  'memcache.local' => '\\OC\\Memcache\\Redis',
'redis' => 
  array (
    'host' => '/tmp/sockets/redis.sock',
    'port' => 0,
  ),
  'instanceid' => 'XXX',
  'passwordsalt' => 'XX',
  'secret' => 'XX',
  'trusted_domains' => 
  array (
    0 => '192.168.1.101',
    1 => '127.0.1.1',
  ),
  'datadirectory' => '/var/snap/nextcloud/common/nextcloud/data',
  'overwrite.cli.url' => 'http://192.168.1.101',
  'dbtype' => 'mysql',
  'version' => '11.0.2.7',
  'dbname' => 'nextcloud',
  'dbhost' => 'localhost:/tmp/sockets/mysql.sock',
  'dbport' => '',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'nextcloud',
  'dbpassword' => 'xx','logtimezone' => 'UTC',
  'installed' => true,
  'loglevel' => 0,
  'maintenance' => false,
  'mail_from_address' => 'xx',
  'mail_smtpmode' => 'smtp',
  'mail_domain' => 'gmail.com',
  'mail_smtpauthtype' => 'LOGIN',
  'mail_smtphost' => 'smtp.gmail.com',
  'mail_smtpsecure' => 'ssl',
  'mail_smtpport' => '465',
  'mail_smtpauth' => 1,
  'mail_smtpname' => 'xxx',
  'mail_smtppassword' => 'xx'
);

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Mar 10, 2017

@ernesst that looks okay, so I bet the problem is in /var/snap/nextcloud/current/mysql/root.ini. Can I see the contents of that, please? Mask the password.

@ernesst

This comment has been minimized.

Copy link

ernesst commented Mar 10, 2017

Thanks for your time.

[client]
socket=/var/snap/nextcloud/current/mysql/mysql.sock
user=root
password=

This time, i cannot find any .sock in the relevant folder, however:

root@nextcloud:/usr/local/nagios/libexec# ps -aux |grep mysql
root     20523  3.2  0.1   1464  1012 ?        Ss   19:17   0:00 /bin/sh /snap/nextcloud/1161/bin/start_mysql
root     20552  1.5  0.1   1464  1100 ?        S    19:17   0:00 /bin/sh /snap/nextcloud/1161/bin/mysqld_safe --datadir=/var/snap/nextcloud/1161/mysql --pid-file=/tmp/pids/mysql.pid --lc-messages-dir=/snap/nextcloud/1161/share --socket=/tmp/sockets/mysql.sock
root     20722 25.0 13.8 1200380 130856 ?      Sl   19:17   0:01 /snap/nextcloud/1161/bin/mysqld --basedir=/snap/nextcloud/1161 --datadir=/var/snap/nextcloud/1161/mysql --plugin-dir=/snap/nextcloud/1161/lib/plugin --lc-messages-dir=/snap/nextcloud/1161/share --log-error=/var/snap/nextcloud/1161/mysql/nextcloud.err --pid-file=/tmp/pids/mysql.pid --socket=/tmp/sockets/mysql.sock

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Mar 10, 2017

Yeah, you want #228.

@kyrofa

This comment has been minimized.

Copy link
Member

kyrofa commented Mar 10, 2017

This bug is sort of turning into a catchall "mysql issues" bug. As of v11.0.2snap2, getting this message likely means something else is happening, so I'm going to close this. Please do log another bug if you continue seeing this when running v11.0.2snap2.

@kyrofa kyrofa closed this Mar 10, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment