Files freeze after ssh server drops connection

[janxkoci]

What Happened?

At my university we have a few ssh servers and clusters for running computations. I often connect to these servers from Files, because it's a convenient way to copy files between the servers and my local machine.

These particular servers have an annoying habit of dropping the ssh connection after few minutes of inactivity (unless I'm connected to university wifi - eduroam - which is quite weird in itself). When this happens, and I go to Files and try to interact with the remote directories (navigate folders or refresh current folder view), the Files app freezes. The only point of action afterwards is to kill Files.

One way to prevent this freezing is to disconnect the server from sidebar whenever I suspect the connection was dropped (e.g. coming back 5 minutes or more after I used it last time), then I can reconnect and everything works without issue. But obviously I forget to do that way too often.

I know that the ssh/sftp feature is handled by gvfs backend, however I also use another app - Taxi, the (s)ftp client from AppCenter - which is using the same backend, but doesn't freeze after the connection drops. I know it uses the same feature as it doesn't even ask for password if I already used the password in Files.

Steps to Reproduce

I admit this will not be easy to reproduce, but it goes something like this:

1. Connect to ssh server which tends to drop connection after some time of inactivity.
2. Wait some time without doing anything so the connection gets dropped.
3. Then try to browse folders or just refresh the current folder.
4. If it didn't freeze yet, click anywhere in the window with a mouse. Now you're there - Files is frozen.

Expected Behavior

Files should not freeze just because a connection to a server was dropped. Taxi uses the same backend to connect to the servers and does not freeze when connection is dropped.

OS Version

5.x (Hera)

Software Version

Latest release (I have run all updates)

Log Output

I tried launching Files from terminal, but the freeze doesn't produce any output there. Also `coredumpctl` says there are no coredumps to show.

Hardware Info

All the laptops that I ever had with elementary OS. Acer, Dell, Slimbook, all with Intel chips. I think the issue exists maybe since Loki, but I never paid enough attention to it, until my friend pointed out the dropping connections to our servers (I use a few other servers at different institutes and those are all fine). I then noticed that Files tends to freeze after the connection drops.

[Hedronmx]

This still applies to 6.1.

Same issue here, commented just to raise awareness. No way to debug this

carlosflores@Lenovo-ideapad-FLEX-5-1570-7d5362b2:~$ neofetch
         eeeeeeeeeeeeeeeee            carlosflores@Lenovo-ideapad-FLEX-5-1570-7d5362b2 
      eeeeeeeeeeeeeeeeeeeeeee         ------------------------------------------------ 
    eeeee  eeeeeeeeeeee   eeeee       OS: elementary OS 6.1 Jólnir x86_64 
  eeee   eeeee       eee     eeee     Host: 80XB Lenovo ideapad FLEX 5-1570 
 eeee   eeee          eee     eeee    Kernel: 5.11.0-46-generic 
eee    eee            eee       eee   Uptime: 20 hours, 50 mins 
eee   eee            eee        eee   Packages: 2618 (dpkg), 95 (flatpak) 
ee    eee           eeee       eeee   Shell: bash 5.0.17 
ee    eee         eeeee      eeeeee   Resolution: 1920x1080 
ee    eee       eeeee      eeeee ee   DE: Pantheon 
eee   eeee   eeeeee      eeeee  eee   WM: Mutter(Gala) 
eee    eeeeeeeeee     eeeeee    eee   Terminal: io.elementary.t 
 eeeeeeeeeeeeeeeeeeeeeeee    eeeee    CPU: Intel i5-7200U (4) @ 2.500GHz 
  eeeeeeee eeeeeeeeeeee      eeee     GPU: Intel HD Graphics 620 
    eeeee                 eeeee       Memory: 6577MiB / 7820MiB 
      eeeeeee         eeeeeee
         eeeeeeeeeeeeeeeee                                    
                                                              

[janxkoci]

I just realized one way to test this bug with any ssh server, as there is at least one other way to make the server drop the connection.

Have a tab with active connection to a ssh server and suspend the laptop / desktop for some time (e.g. over night, but even something like 15 minutes should be enough, or maybe a lunch break). After waking up, try to interact with the ssh server tab as described above (navigate folders or refresh the current folder view).

Edit: Forgot to mention I'm on OS 6.1 now.

[fayjie92]

This behavior is reproducible, when we let the system go to sleep while connected to a remote server. If the file was opened with a tab displaying remote location, on waking up, it crashes.

[jeremypw]

I have tried to reproduce this with version 6.5.0 (On Elementary 7) on an SFTP connection, simulating the "going to sleep" by going into aeroplane mode. I haven't experienced any crashing or freezing of the entire interface (the tab itself may block but other tabs and other elements of the UI continue to work). Reconnecting either completes the operation or asks for the password again. I'll try it with suspending the laptop.

Closing and opening Files such that a disconnected location is restored results in the tab continuing to be in a "working" state but the rest of the UI works as usual. Reconnecting results in the tab completing the load process.

A fair few changes have been made since this issue was first reported so a fresh report for Horus is required if it still happens under some circumstances.

[janxkoci]

I'll check it today

[janxkoci]

I can still get it to freeze. I connected to a server (my model finished overnight, cool!), then closed the laptop and went for breakfast. Now I opened it again, focused Files and clicked on a folder (in a column view) that I didn't load before - and Files are frozen now, clicking other tabs or anything else isn't working.

PS: upon switching back to Files after writing this comment I was presented with the "wait or kill" dialog.

PPS: relaunching Files again doesn't work and presents the same "wait or kill" dialog. Kiling the gvfsd process for the connection finally let me open Files again.

PPPS: BTW after all of this the path bar in Files looks like this:

Switching between tabs or opening new tabs doesn't help it, it just stays there like that with new paths projected over it. The background path seems to be the one to the server (closing connection from sidebar and reconnecting also doesn't help). Closing Files with Ctrl+Q finally shows normal breadcrumbs.

[jeremypw]

@janxkoci Thanks for the update. Looks like there is a blocking io process going on that needs to be made asynchronous. Either that or an infinite loop (in that case the cpu usage will be high). Either way something is stopping any other Files process from getting any cpu time. To help identify what it is, could you run Files from the terminal under gdb (you may need to install gdb) with the command gdb io.elementary.files followed by run. Make sure all existing Files processes are closed first. When Files freezes, go back to the terminal and enter Ctrl-C followed by bt. This should give information about what happened up to the problem. If you install the libpantheon-files-core-dev package first the information will be more useful.

[jeremypw]

The pathbar appearance is just a side-effect of the animation being interrupted in the middle of a transition.

By the way, could you confirm which version of Files you are running? The latest is 6.5.0.

[janxkoci]

Ok my is older:

$ io.elementary.files --version
io.elementary.files 6.4.1

(Btw issuing io.elementary.files -h closes files and I cannot open them again - I expected to see help instead 😮 )

[jeremypw]

@janxkoci Hmm that's a regression - not sure when that happened. I'll raise an issue and investigate.

[janxkoci]

Ah, the update to Files is in the queue, but Appcenter forces me to double-reboot to even install the update (it only downloads it now) and I didn't have time to reboot this week. 🤷

[jeremypw]

Hopefully it helps. I havent been able to freeze Files by simulating the server disconnecting by turning wifi off while suspended using a public SFTP server. The tab freezes but not the whole app. However, this is not exactly the same as the server disconnecting. When I have time Ill try to set up an SSH server I can control.

[jeremypw]

I thought AppCenter does install native apps like Files straight-away (although you need to close and restart Files for it to have an effect) however there do seem to be a lot of mandatory restarts recently - I assumed due to kernel updates - so maybe something changed.

[janxkoci]

Ok this is crazy. I did the two reboots to install the update, then after the second booting I opened Files and the SSH session in one of the tabs, closed the laptop and went out witm my second laptop. Now I returned, opened, focused Files and tried to navigate another folder on the SSH server - instant freeze.

I realized I forgot to check the version after the update, so I popped up a terminal and - well this is weird:

$ io.elementary.files --version
Failed to register: Časový limit vypršel

It timed out because Files were frozen 😮 I had to check in Appcenter, which shows the version as 6.5 now.

---

About the reboots - all "OS updates" (I think it means deb packages) are grouped under one item in Appcenter and when there is a kernel update, it only shows a "Download" button, not an "Update" button next to it (other updates show normal Update buttons, including flatpak apps and runtimes). So basically unless I reboot, all these updates get only downloaded but not installed, only the flatpaks do. 🤷

[jeremypw]

The --version option is working for me and showing "6.5.0". I am running the daily development version compiled from source but there havent been any recent commits that I would expect to affect this option. Weirdly, the --help option is now working as well 🤷 Maybe something to do with experiments with trying to get Files to freeze.

Might be worth running sudo killall io.elementary.files to ensure all Files processes are killed. They will restart with the installed version when needed.

[janxkoci]

The --version works when Files are not frozen 😉

[jeremypw]

The same applies to --help see #2305

[jeremypw]

@janxkoci I have tried setting up an OpenSSH server on my local machine but have not succeeded in getting a freeze so far. One thing occurred to me is that when Files is frozen, there might be a modal dialog somewhere (e.g. beneath a window) that has stolen focus. Unlikely, but worth asking. https://www.cyberciti.biz/tips/open-ssh-server-connection-drops-out-after-few-or-n-minutes-of-inactivity.html might be of help but only if you have access to the server config file.

[janxkoci]

There is definitely no dialog behind the window or anything like that. If there is, it appears normally on top, asking to wait or kill the app. In fact, killing the app and trying to re-launch will often show no window and only the wait-or-kill dialog appears.

Afaik, the only way to avoid the freeze is to eject the connection from the sidebar before it times-out (e.g. server dropping the connection, or me suspending the laptop), or to close the remote tab/not interact with it.

[fayjie92]

This issue still exists in files --version 6.5.3.