Accessing files on network drives is slow #1766

Open
izuzak opened this Issue Mar 18, 2014 · 62 comments

Comments

Projects
None yet
Owner

izuzak commented Mar 18, 2014

Reported via Halp:

  • support/6290a694aa0611e38d8242e11b72b07d

Accessing files on network drives appears to be much slower than Sublime for the same mount & file. Additionally, the "Keep Waiting" decision box pops up almost every time.

I don't have a network drive, so I didn't verify this. 🚗

Edit: more reports:

  • support/4e6fe392af5a11e38b62f01b0c57459b

@izuzak izuzak added the performance label Mar 18, 2014

This comment has been minimized.

Show comment Hide comment
@kevinsawicki

kevinsawicki Jul 21, 2014

Owner

After investigating this it looks like node's fs.realpathSync is pretty slow for network drives and most of the time is spent internally calling lstatSync.

Will re-profile after upgrading to atom-shell 0.14

Owner

kevinsawicki commented Jul 21, 2014

After investigating this it looks like node's fs.realpathSync is pretty slow for network drives and most of the time is spent internally calling lstatSync.

Will re-profile after upgrading to atom-shell 0.14

@kevinsawicki kevinsawicki self-assigned this Jul 21, 2014

This comment has been minimized.

Show comment Hide comment
@sincontri

sincontri Aug 7, 2014

Please this is a blocking issue for those needing to edit files over sftp.
I'm on Ubuntu 14.04, atom 116.
Many thanks!

Please this is a blocking issue for those needing to edit files over sftp.
I'm on Ubuntu 14.04, atom 116.
Many thanks!

This comment has been minimized.

Show comment Hide comment
@iRet

iRet Sep 26, 2014

Slow on afp shared folder. Intermittent becomes almost irresponsible.
Connection is about 50mb/s, another editors works well on same share.

iRet commented Sep 26, 2014

Slow on afp shared folder. Intermittent becomes almost irresponsible.
Connection is about 50mb/s, another editors works well on same share.

This comment has been minimized.

Show comment Hide comment
@philipgiuliani

philipgiuliani Sep 26, 2014

Contributor

Im also a OSX network user, so my working directory is on a server. Atom is running compared to at-home (where im working locally) really slow here, but i can work with it.

The most annoying thing is the toggle speed of the tree-view. When i toggle a folder it needs very long to show the files in there.

Contributor

philipgiuliani commented Sep 26, 2014

Im also a OSX network user, so my working directory is on a server. Atom is running compared to at-home (where im working locally) really slow here, but i can work with it.

The most annoying thing is the toggle speed of the tree-view. When i toggle a folder it needs very long to show the files in there.

This comment has been minimized.

Show comment Hide comment
@iRet

iRet Sep 26, 2014

@philipgiuliani sometimes it works for me as slow as usual, but another time words appears after a few seconds after typing, it's not usable. I thought something is happened with my connection but sublime fast as hell on same share, tried to restart atom nothing helped.

iRet commented Sep 26, 2014

@philipgiuliani sometimes it works for me as slow as usual, but another time words appears after a few seconds after typing, it's not usable. I thought something is happened with my connection but sublime fast as hell on same share, tried to restart atom nothing helped.

This comment has been minimized.

Show comment Hide comment
@lee-dohm

lee-dohm Sep 26, 2014

Owner

I use sshfs for network access on OS X and it is a little bit slower, but only noticeable on really large file trees. Maybe using a different protocol would help?

Owner

lee-dohm commented Sep 26, 2014

I use sshfs for network access on OS X and it is a little bit slower, but only noticeable on really large file trees. Maybe using a different protocol would help?

This comment has been minimized.

Show comment Hide comment
@sincontri

sincontri Oct 3, 2014

I noticed it is working much faster now, on sshfs network mount, both linux and mac os.
Thanks

I noticed it is working much faster now, on sshfs network mount, both linux and mac os.
Thanks

This comment has been minimized.

Show comment Hide comment
@Nikita240

Nikita240 Jan 23, 2015

+1, slow for me too.

Makes webdev really difficult.

+1, slow for me too.

Makes webdev really difficult.

This comment has been minimized.

Show comment Hide comment
@GM-Alex

GM-Alex Feb 4, 2015

Same for me with 0.177.0

GM-Alex commented Feb 4, 2015

Same for me with 0.177.0

This comment has been minimized.

Show comment Hide comment
@toddbu

toddbu Mar 24, 2015

Same for me on the last few releases including 0.187.0. I'm running SMB shares on a CentOS server running under VirtualBox on the same Mac that I'm running Atom on. It's really random but happens once or twice a day. When I close the process Atom stays running and I have to force quit the app.

toddbu commented Mar 24, 2015

Same for me on the last few releases including 0.187.0. I'm running SMB shares on a CentOS server running under VirtualBox on the same Mac that I'm running Atom on. It's really random but happens once or twice a day. When I close the process Atom stays running and I have to force quit the app.

This comment has been minimized.

Show comment Hide comment
@pheuter

pheuter Mar 28, 2015

Same on Atom 0.188.0, having a hard time opening an smb drive from virtualbox on OS X.

pheuter commented Mar 28, 2015

Same on Atom 0.188.0, having a hard time opening an smb drive from virtualbox on OS X.

This comment has been minimized.

Show comment Hide comment
@JohnRoach

JohnRoach May 14, 2015

I will like to inform you that this(loading mounted drive folders or files) also is slow in Windows 7.

I will like to inform you that this(loading mounted drive folders or files) also is slow in Windows 7.

This comment has been minimized.

Show comment Hide comment
@benjamindean

benjamindean Jun 4, 2015

In addition, fs.realpathSync is, actually, slow for almost everything. This happens only when opening file using context menu (Open With) on remote server (using nautilus in my case). Don't know the reasons, but that's was the issue for a long time. This is, probably, avoidable by not using something other than readFileSync and writeFileSync when path are not in the System context. Anyway, nothing as fast as a native implementation.
Nothing bad happens when opening the same file from Atom, thou.

In addition, fs.realpathSync is, actually, slow for almost everything. This happens only when opening file using context menu (Open With) on remote server (using nautilus in my case). Don't know the reasons, but that's was the issue for a long time. This is, probably, avoidable by not using something other than readFileSync and writeFileSync when path are not in the System context. Anyway, nothing as fast as a native implementation.
Nothing bad happens when opening the same file from Atom, thou.

This comment has been minimized.

Show comment Hide comment
@medfreeman

medfreeman Jun 19, 2015

Same on Ubuntu 14.04, accessing a SFTP server through Nautilus.

Same on Ubuntu 14.04, accessing a SFTP server through Nautilus.

This comment has been minimized.

Show comment Hide comment
@eikeco

eikeco Jul 2, 2015

I am accessing files via a Parallels Virtual Machine and having the same problem. I'm on a Mac.
Atom Version 1.0.0.

Even when a file is open it takes a moment to catch up with what you type.

eikeco commented Jul 2, 2015

I am accessing files via a Parallels Virtual Machine and having the same problem. I'm on a Mac.
Atom Version 1.0.0.

Even when a file is open it takes a moment to catch up with what you type.

This comment has been minimized.

Show comment Hide comment
@leonardfactory

leonardfactory Jul 6, 2015

Same here with 1.0.0. Parallels VM with Windows 8, I'm working on a Mac.

Same here with 1.0.0. Parallels VM with Windows 8, I'm working on a Mac.

@kevinsawicki kevinsawicki removed their assignment Aug 6, 2015

@izuzak izuzak added the network label Aug 20, 2015

This comment has been minimized.

Show comment Hide comment
@jryanad

jryanad Aug 25, 2015

Having an issue as well. Atom 1.0.7 with running Virtual Box and connecting via smb on Mac 10.10.5. Just continually loads most times and have to restart the process to get it to "kick in".

Edit
However, I just disabled the Recent Projects package and now it seems to load up fine. Will continue testing and report back.

jryanad commented Aug 25, 2015

Having an issue as well. Atom 1.0.7 with running Virtual Box and connecting via smb on Mac 10.10.5. Just continually loads most times and have to restart the process to get it to "kick in".

Edit
However, I just disabled the Recent Projects package and now it seems to load up fine. Will continue testing and report back.

This comment has been minimized.

Show comment Hide comment
@incognick

incognick Nov 4, 2015

Same issue Atom 1.1.0 Windows 7. Accessing a mapped network drive. I dropped a folder into Atom and the GUI locked up. Clicking on any folder locks up the GUI unless I wait 30+ seconds. I can browse the folders using windows explorer with only a slight delay (1-2 seconds).

Same issue Atom 1.1.0 Windows 7. Accessing a mapped network drive. I dropped a folder into Atom and the GUI locked up. Clicking on any folder locks up the GUI unless I wait 30+ seconds. I can browse the folders using windows explorer with only a slight delay (1-2 seconds).

This comment has been minimized.

Show comment Hide comment
@VasiliDarozhkin

VasiliDarozhkin Dec 13, 2015

I run Atom 1.2.4 on Windows 7.
Project files are on the VirtualBox Ubuntu 12.04, on the same laptop
smbd process takes about 50% of CPU (2-core virtual machine)
strace shows endless getcwd and lstat commands over the project files

I run Atom 1.2.4 on Windows 7.
Project files are on the VirtualBox Ubuntu 12.04, on the same laptop
smbd process takes about 50% of CPU (2-core virtual machine)
strace shows endless getcwd and lstat commands over the project files

This comment has been minimized.

Show comment Hide comment
@drj-io

drj-io Dec 16, 2015

Hi I was running Atom 1.3.1 on Mac. I'm mounting shares using SSHFS. It works great, but Atom becomes sluggishly slow. I've since switched back to Sublime for SSHFS projects.

Editing, saving, loading... everything. Hope this helps.

drj-io commented Dec 16, 2015

Hi I was running Atom 1.3.1 on Mac. I'm mounting shares using SSHFS. It works great, but Atom becomes sluggishly slow. I've since switched back to Sublime for SSHFS projects.

Editing, saving, loading... everything. Hope this helps.

This comment has been minimized.

Show comment Hide comment
@carlosperate

carlosperate Dec 17, 2015

Having the same issue in 1.3.2 on Windows with mapped drives from a samba network connection, locks up constantly, it's very disruptive to the workflow, and makes it almost unusable for this kind of environments.

Having the same issue in 1.3.2 on Windows with mapped drives from a samba network connection, locks up constantly, it's very disruptive to the workflow, and makes it almost unusable for this kind of environments.

This comment has been minimized.

Show comment Hide comment
@Marox44

Marox44 Feb 10, 2016

Same issure here. Atom 1.5.0 on Windows. Working on folder from windows network drive makes Atom slow and laggy. (works great on local files)

I switched from Brackets recently as it also had similar issues. Unlike Brackets however, Atom loads folder pretty quickly (~20sec), but then the entire workflow (autocomplete, moving lines, switching tabs, undo-ing, etc., and even the typing itself) starts freezing and lagging.
Brackets on the other hand had serious issue opening the folder itself (up to 15min), but then the interface was working decently, having almost no performance issues (slightly at switching tabs and saving but I think it's acceptable)

I assume it has something to do with file caching.

Marox44 commented Feb 10, 2016

Same issure here. Atom 1.5.0 on Windows. Working on folder from windows network drive makes Atom slow and laggy. (works great on local files)

I switched from Brackets recently as it also had similar issues. Unlike Brackets however, Atom loads folder pretty quickly (~20sec), but then the entire workflow (autocomplete, moving lines, switching tabs, undo-ing, etc., and even the typing itself) starts freezing and lagging.
Brackets on the other hand had serious issue opening the folder itself (up to 15min), but then the interface was working decently, having almost no performance issues (slightly at switching tabs and saving but I think it's acceptable)

I assume it has something to do with file caching.

This comment has been minimized.

Show comment Hide comment
@ThePooN

ThePooN Feb 17, 2016

I recently switched from ST3 to Atom. It works great on local folders but network folders are laggy as hell. Please fix that.

ThePooN commented Feb 17, 2016

I recently switched from ST3 to Atom. It works great on local folders but network folders are laggy as hell. Please fix that.

This comment has been minimized.

Show comment Hide comment
@bitswarming

bitswarming Feb 17, 2016

Switch back to ST3, like me))))

Switch back to ST3, like me))))

This comment has been minimized.

Show comment Hide comment
@NotChillcapped

NotChillcapped Mar 21, 2016

2 years later and this is still the case... huge bummer

2 years later and this is still the case... huge bummer

This comment has been minimized.

Show comment Hide comment
@ThePooN

ThePooN Mar 25, 2016

This issue is opened for more than a year. What's going on?

ThePooN commented Mar 25, 2016

This issue is opened for more than a year. What's going on?

This comment has been minimized.

Show comment Hide comment
@carlosperate

carlosperate Mar 25, 2016

Maybe the original title should be changed from "Accessing files on network drives is slow" to "Accessing files on network drives makes Atom unusably slow", since the first sounds like a minor inconvenience, and the second is pretty much its current state.

Maybe the original title should be changed from "Accessing files on network drives is slow" to "Accessing files on network drives makes Atom unusably slow", since the first sounds like a minor inconvenience, and the second is pretty much its current state.

This comment has been minimized.

Show comment Hide comment
@adamfranco

adamfranco Mar 25, 2016

I can confirm that this problem is both present and makes Atom completely unusable on OSX Yosemite connecting to a Samba share on a gigabit university LAN. My primary mode of working is on a Mac, using Samba to mount a home directory on a Linux host in the on-campus data center. I've been heavily using Atom as my primary editor since 1.0 and this problem seems to have been getting worse recently. While Atom has always been unusable over a slower residential-via-VPN-to-university connection, usage on a local gigabit link had been workable until recent releases. I've been running the Atom beta for a while and on March 8th I first noticed the 1.6 beta locking up. At this point both 1.6.0 and the 1.7 betas are totally unusable when opening projects on a Samba share.

I'll have to download a bunch of old versions and do testing to see when the additional slow-down was added in the 1.6 beta, but that may have been more of a straw breaking this camel's back rather than the root of the issue.

I can confirm that this problem is both present and makes Atom completely unusable on OSX Yosemite connecting to a Samba share on a gigabit university LAN. My primary mode of working is on a Mac, using Samba to mount a home directory on a Linux host in the on-campus data center. I've been heavily using Atom as my primary editor since 1.0 and this problem seems to have been getting worse recently. While Atom has always been unusable over a slower residential-via-VPN-to-university connection, usage on a local gigabit link had been workable until recent releases. I've been running the Atom beta for a while and on March 8th I first noticed the 1.6 beta locking up. At this point both 1.6.0 and the 1.7 betas are totally unusable when opening projects on a Samba share.

I'll have to download a bunch of old versions and do testing to see when the additional slow-down was added in the 1.6 beta, but that may have been more of a straw breaking this camel's back rather than the root of the issue.

This comment has been minimized.

Show comment Hide comment
@wot48

wot48 Mar 25, 2016

There is a random pause saving/open file. My Samba share is very fast read/write using other Windows programs. I've tried Sublime text 2/3, VS Code, Notepadd++.. none of them have this problem. It makes Atom useless over network drive, sadly I have to switch to Sublime Text for now.

Windows 10 Pro. Version 1511.
Samba 4.4

wot48 commented Mar 25, 2016

There is a random pause saving/open file. My Samba share is very fast read/write using other Windows programs. I've tried Sublime text 2/3, VS Code, Notepadd++.. none of them have this problem. It makes Atom useless over network drive, sadly I have to switch to Sublime Text for now.

Windows 10 Pro. Version 1511.
Samba 4.4

This comment has been minimized.

Show comment Hide comment
@adamfranco

adamfranco Mar 25, 2016

I've done a little bit of testing with Atom 1.7-beta0 running on OSX to a Samba share on a Linux host. My test project is tiny, with 3 project files and 1 subdirectory and a short (16-commit) git history -- a total of 73 files and directories.
screen shot 2016-03-25 at 10 55 55 am

After adding the project folder (another super slow process) and waiting for Atom to finish loading the project I ran strace on the Linux host and watching for access to files and directories for my samba process:

strace -f -p 22384 |& grep -P 'open|lstat'

While Atom was just sitting there was no disk activity, however after focusing to another application, just bringing Atom back to the foreground resulted in 388 lstat and open calls. While editing a file, every time typing pauses, there are usually about 27 lstat and open calls to check the state of the current buffer against the current Git HEAD.

In my testing on a larger project folder with 1,128 project files & directories, running the git status checks (over a relatively fast VPN this time) takes about 20 minutes to complete and results in 29,927 lstat and open calls.

Making the situation even worse, multiple runs of these git status checks seem to queue up if you focus to Atom and away a few times, potentially queuing up hours of I/O with just a few ALT + TAB/CMD + TAB keystrokes. For any medium-sized project, this could result in hours of waiting time.

My own hunch is that Atom's regular refresh of the Git status of the entire project tree every time the application is brought into focus is a large part of the problem. Even if you've patiently waited for the project to load, switching to another application and back starts the entire cycle over again.

One potential solution would be to split Git usage into several operational modes: "automatic", "manual", and "off" with the current behavior in "automatic" mode and the "manual" mode providing controls in the UI to trigger a status refresh of the project. The manual and off modes could potentially be enabled for only certain projects/paths, for example allowing automatic mode for local Project Folders and manual mode for anything under /mnt/ or /Volumes/.

I've done a little bit of testing with Atom 1.7-beta0 running on OSX to a Samba share on a Linux host. My test project is tiny, with 3 project files and 1 subdirectory and a short (16-commit) git history -- a total of 73 files and directories.
screen shot 2016-03-25 at 10 55 55 am

After adding the project folder (another super slow process) and waiting for Atom to finish loading the project I ran strace on the Linux host and watching for access to files and directories for my samba process:

strace -f -p 22384 |& grep -P 'open|lstat'

While Atom was just sitting there was no disk activity, however after focusing to another application, just bringing Atom back to the foreground resulted in 388 lstat and open calls. While editing a file, every time typing pauses, there are usually about 27 lstat and open calls to check the state of the current buffer against the current Git HEAD.

In my testing on a larger project folder with 1,128 project files & directories, running the git status checks (over a relatively fast VPN this time) takes about 20 minutes to complete and results in 29,927 lstat and open calls.

Making the situation even worse, multiple runs of these git status checks seem to queue up if you focus to Atom and away a few times, potentially queuing up hours of I/O with just a few ALT + TAB/CMD + TAB keystrokes. For any medium-sized project, this could result in hours of waiting time.

My own hunch is that Atom's regular refresh of the Git status of the entire project tree every time the application is brought into focus is a large part of the problem. Even if you've patiently waited for the project to load, switching to another application and back starts the entire cycle over again.

One potential solution would be to split Git usage into several operational modes: "automatic", "manual", and "off" with the current behavior in "automatic" mode and the "manual" mode providing controls in the UI to trigger a status refresh of the project. The manual and off modes could potentially be enabled for only certain projects/paths, for example allowing automatic mode for local Project Folders and manual mode for anything under /mnt/ or /Volumes/.

This comment has been minimized.

Show comment Hide comment
@Zireael07

Zireael07 Mar 25, 2016

@adamfranco: Good analysis. I'd tag relevant project members if I knew who they were...

@adamfranco: Good analysis. I'd tag relevant project members if I knew who they were...

This comment has been minimized.

Show comment Hide comment
@joshaber

joshaber Mar 25, 2016

Contributor

Thanks for the great writeup @adamfranco! That's more or less what I found while investigating #11214.

#11277 should improve the situation somewhat, but your suggestion for an off/on/manual setting would be a great addition too.

Contributor

joshaber commented Mar 25, 2016

Thanks for the great writeup @adamfranco! That's more or less what I found while investigating #11214.

#11277 should improve the situation somewhat, but your suggestion for an off/on/manual setting would be a great addition too.

This comment has been minimized.

Show comment Hide comment
@MichaelThessel

MichaelThessel Mar 25, 2016

Several people in #11214 pointed out that downgrading to 1.5.4 solved the issue for them. I just did the same on my install and the situation is much improved. I still see a very high amount of stat calls as soon as I ALT + tab to Atom, but the smbd CPU utilization went down from continuous 100% to continuous < 20%. I don't see the GUI freezing anymore either. If people looking for a workaround for the time being, downgrading to 1.5.4 and disabling automatic updates might be a solution.

Several people in #11214 pointed out that downgrading to 1.5.4 solved the issue for them. I just did the same on my install and the situation is much improved. I still see a very high amount of stat calls as soon as I ALT + tab to Atom, but the smbd CPU utilization went down from continuous 100% to continuous < 20%. I don't see the GUI freezing anymore either. If people looking for a workaround for the time being, downgrading to 1.5.4 and disabling automatic updates might be a solution.

This comment has been minimized.

Show comment Hide comment
@adamfranco

adamfranco Mar 25, 2016

For reference, this gist shows all 388 lstat and open calls made when bringing Atom into focus with my small test Project Folder (3 files).

Looking at this, I notice that the project directory, .git directory, and some .git subdirectories get scanned sometimes several times during the status check for each file. It may be that the Git implementation is searching the the .git sub-directories for objects and not caching objects or the directory listings.

For reference, this gist shows all 388 lstat and open calls made when bringing Atom into focus with my small test Project Folder (3 files).

Looking at this, I notice that the project directory, .git directory, and some .git subdirectories get scanned sometimes several times during the status check for each file. It may be that the Git implementation is searching the the .git sub-directories for objects and not caching objects or the directory listings.

This comment has been minimized.

Show comment Hide comment
@adamfranco

adamfranco Mar 29, 2016

I've re-run my tests (from above) against a new Atom 614eb81 source-build which includes @joshaber's improvements from #11280 and #11277.

I now see 292 lstat and open calls made when bringing Atom into focus with my small test Project Folder (3 files) on a Samba share (down from 388 for Atom 1.7-beta0), a 25% drop in the number of calls. On my larger test project with 1,122 Project files I saw an even more dramatic improvement, with the number of lstat and open calls dropping from 24,731 to 12,854 -- a 48% drop in the number of calls. 🎉 Great improvements, @joshaber, thanks!

The git-status checks are still demanding quite a bit of I/O, just not quite as much as before. 😄

(This note is also relevant to #11214)

I've re-run my tests (from above) against a new Atom 614eb81 source-build which includes @joshaber's improvements from #11280 and #11277.

I now see 292 lstat and open calls made when bringing Atom into focus with my small test Project Folder (3 files) on a Samba share (down from 388 for Atom 1.7-beta0), a 25% drop in the number of calls. On my larger test project with 1,122 Project files I saw an even more dramatic improvement, with the number of lstat and open calls dropping from 24,731 to 12,854 -- a 48% drop in the number of calls. 🎉 Great improvements, @joshaber, thanks!

The git-status checks are still demanding quite a bit of I/O, just not quite as much as before. 😄

(This note is also relevant to #11214)

This comment has been minimized.

Show comment Hide comment
@joshaber

joshaber Mar 29, 2016

Contributor

That's great news @adamfranco! Thanks for re-running the numbers.

Contributor

joshaber commented Mar 29, 2016

That's great news @adamfranco! Thanks for re-running the numbers.

This comment has been minimized.

Show comment Hide comment
@adamfranco

adamfranco Mar 31, 2016

I've been poking a bit at tree-view to see what effect disabling it's git-status checks can eliminate the intense I/O by commenting out a few lines in the tree-view module:

diff --git a/lib/directory.coffee b/lib/directory.coffee
index 279b64b..251141c 100644
--- a/lib/directory.coffee
+++ b/lib/directory.coffee
@@ -30,7 +30,7 @@ class Directory
     @status = null
     @entries = {}

-    @submodule = repoForPath(@path)?.isSubmodule(@path)
+    # @submodule = repoForPath(@path)?.isSubmodule(@path)

     @subscribeToRepo()
     @updateStatus()
diff --git a/lib/helpers.coffee b/lib/helpers.coffee
index 3135dfd..b059f9b 100644
--- a/lib/helpers.coffee
+++ b/lib/helpers.coffee
@@ -2,6 +2,7 @@ path = require "path"

 module.exports =
   repoForPath: (goalPath) ->
+    return null
     for projectPath, i in atom.project.getPaths()
       if goalPath is projectPath or goalPath.indexOf(projectPath + path.sep) is 0
         return atom.project.getRepositories()[i]

What I've found is that this hack does seem to eliminate almost all of the git-related I/O. That said, it seems that something else in Atom is still doing a full-depth scan of every file & directory in Project Folders on Atom-focus -- even with the tree-view module and almost all core and community modules are completely disabled. I've been able to trigger this full Project Folder scan-on-focus effect even when quitting and re-starting Atom with tree-view disabled.

Ideally, when a Project Folder is added to the editor and git-status checks are disabled, only the visible (expanded) directories & open files would be stat-ed on focus, not the full-depth contents of Project Folder. Maybe if we can figure out what else is scanning the Project Folder then a two-pronged approach to get Atom's I/O down to usable levels on networked file-systems:

  • stop tree-view git-status checks
  • stop the other Project Folder scan

I've been poking a bit at tree-view to see what effect disabling it's git-status checks can eliminate the intense I/O by commenting out a few lines in the tree-view module:

diff --git a/lib/directory.coffee b/lib/directory.coffee
index 279b64b..251141c 100644
--- a/lib/directory.coffee
+++ b/lib/directory.coffee
@@ -30,7 +30,7 @@ class Directory
     @status = null
     @entries = {}

-    @submodule = repoForPath(@path)?.isSubmodule(@path)
+    # @submodule = repoForPath(@path)?.isSubmodule(@path)

     @subscribeToRepo()
     @updateStatus()
diff --git a/lib/helpers.coffee b/lib/helpers.coffee
index 3135dfd..b059f9b 100644
--- a/lib/helpers.coffee
+++ b/lib/helpers.coffee
@@ -2,6 +2,7 @@ path = require "path"

 module.exports =
   repoForPath: (goalPath) ->
+    return null
     for projectPath, i in atom.project.getPaths()
       if goalPath is projectPath or goalPath.indexOf(projectPath + path.sep) is 0
         return atom.project.getRepositories()[i]

What I've found is that this hack does seem to eliminate almost all of the git-related I/O. That said, it seems that something else in Atom is still doing a full-depth scan of every file & directory in Project Folders on Atom-focus -- even with the tree-view module and almost all core and community modules are completely disabled. I've been able to trigger this full Project Folder scan-on-focus effect even when quitting and re-starting Atom with tree-view disabled.

Ideally, when a Project Folder is added to the editor and git-status checks are disabled, only the visible (expanded) directories & open files would be stat-ed on focus, not the full-depth contents of Project Folder. Maybe if we can figure out what else is scanning the Project Folder then a two-pronged approach to get Atom's I/O down to usable levels on networked file-systems:

  • stop tree-view git-status checks
  • stop the other Project Folder scan

This comment has been minimized.

Show comment Hide comment
@adamfranco

adamfranco Mar 31, 2016

I've done a little more debugging and found that git-repository-async.js's refreshStatus() method is being called for the full project, even without the tree-view module enabled. This is the "other Project Folder scan" that I noticed in the comment above.

It seems that project's addPath() method is registering a repositoryForDirectorySync promise that sets up the event that calls refreshStatus() every focus. This can be seen in project.coffee:

addPath: (projectPath, options) ->
  ...
  repo = null
  for provider in @repositoryProviders
    break if repo = provider.repositoryForDirectorySync?(directory)
  @repositories.push(repo ? null)
   ...

This in turn uses the GitRepositoryProvider to set up GitRepository which due to its default true value for refreshOnWindowFocus, attaches a refreshStatus() call to window.onfocus().

It looks like the GitRepositoryProvider is passing options that include project options and the config to the GitRepository, but the options are keyed as {config: {...}, project: {...} when the GitRepository is looking for the top-level refreshOnWindowFocus key, so (it at least seems) that adding a refreshOnWindowFocus value to one's .atom/config.cson wouldn't allow disabling of refreshOnWindowFocus. Editing the GitRepository to set refreshOnWindowFocus to false does successfully stop I/O on focus.

If there is a desire to keep git integration, but reduce its impact, there seem several places that are asking for full-project refreshStatus() promises when maybe they could be asking to refresh the status of a single file that was changed (on save) or just the visible files (as seen by tree-view).

To fully halt git integration it seems that putting a check in Project.addPath() to see if the current path is in an exclude-list might do the trick.

I've done a little more debugging and found that git-repository-async.js's refreshStatus() method is being called for the full project, even without the tree-view module enabled. This is the "other Project Folder scan" that I noticed in the comment above.

It seems that project's addPath() method is registering a repositoryForDirectorySync promise that sets up the event that calls refreshStatus() every focus. This can be seen in project.coffee:

addPath: (projectPath, options) ->
  ...
  repo = null
  for provider in @repositoryProviders
    break if repo = provider.repositoryForDirectorySync?(directory)
  @repositories.push(repo ? null)
   ...

This in turn uses the GitRepositoryProvider to set up GitRepository which due to its default true value for refreshOnWindowFocus, attaches a refreshStatus() call to window.onfocus().

It looks like the GitRepositoryProvider is passing options that include project options and the config to the GitRepository, but the options are keyed as {config: {...}, project: {...} when the GitRepository is looking for the top-level refreshOnWindowFocus key, so (it at least seems) that adding a refreshOnWindowFocus value to one's .atom/config.cson wouldn't allow disabling of refreshOnWindowFocus. Editing the GitRepository to set refreshOnWindowFocus to false does successfully stop I/O on focus.

If there is a desire to keep git integration, but reduce its impact, there seem several places that are asking for full-project refreshStatus() promises when maybe they could be asking to refresh the status of a single file that was changed (on save) or just the visible files (as seen by tree-view).

To fully halt git integration it seems that putting a check in Project.addPath() to see if the current path is in an exclude-list might do the trick.

adamfranco added a commit to adamfranco/atom that referenced this issue Apr 4, 2016

Learn configuration of FS paths for which to skip VCS integration.
This patch allows users to configure paths such as `/mnt/` or `/Volumes/`
which map to network shares or other targets with slow I/O performance.
By skipping VCS integration on these paths, I/O is significantly reduced at
the cost of not having VCS integration and allowing Atom to at least be usable
as an editor when pointed at mounted network shares.

This is a attempt to mitigate some of the effects of #1766.

This comment has been minimized.

Show comment Hide comment
@Rod-O

Rod-O Apr 5, 2016

If I point my fs and atom one level up, atom works fine, of course, git is not working, but I can work without problems.

Rod-O commented Apr 5, 2016

If I point my fs and atom one level up, atom works fine, of course, git is not working, but I can work without problems.

This comment has been minimized.

Show comment Hide comment
@sampotts

sampotts Apr 5, 2016

Cool, I'll give that a go. To be honest, I'm not bothered about the built in Git as I use tortoise on the windows box anyway.

sampotts commented Apr 5, 2016

Cool, I'll give that a go. To be honest, I'm not bothered about the built in Git as I use tortoise on the windows box anyway.

This comment has been minimized.

Show comment Hide comment
@danielvoicu

danielvoicu Jun 3, 2016

The issue - or a very similar one - is still there. I work on a network drive mapped through a SFTP connection. When I open a new project, atom shows the folder structure, but when trying to open a file, it doesn't load the content of the file, showing it as empty.

The server is mapped as a network drive on Windows 10.

The issue - or a very similar one - is still there. I work on a network drive mapped through a SFTP connection. When I open a new project, atom shows the folder structure, but when trying to open a file, it doesn't load the content of the file, showing it as empty.

The server is mapped as a network drive on Windows 10.

This comment has been minimized.

Show comment Hide comment
@rviertel2

rviertel2 Aug 26, 2016

I am also running into this issue. I worked on-site for about 3 months and connected to the network directly. I was able to edit network files/projects with Atom 1.9.8 and did not have any problems. Since then, I have moved out of state and now am connecting to the network over a vpn. I upgraded to Atom 1.9.9 and now editing files over the network drive is extremely slow. My machine is running Windows 10.

I am also running into this issue. I worked on-site for about 3 months and connected to the network directly. I was able to edit network files/projects with Atom 1.9.8 and did not have any problems. Since then, I have moved out of state and now am connecting to the network over a vpn. I upgraded to Atom 1.9.9 and now editing files over the network drive is extremely slow. My machine is running Windows 10.

This comment has been minimized.

Show comment Hide comment
@cy2k

cy2k Aug 26, 2016

@Rod-O

If I point my fs and atom one level up, atom works fine, of course, git is not working, but I can work without problems.

To clarify what I think you meant, what I did was removed my individual project folders from the view on the left, and instead, I added a "project" that is really just a directory that contains multiple project folders in it. That seems to break the git coloring in the tree view, meaning my alt-tabs are now WAY faster, and I've just lost the coloring (which is OK for me right now, the speed is more important to me in the short term).

Great idea, thanks man.

cy2k commented Aug 26, 2016

@Rod-O

If I point my fs and atom one level up, atom works fine, of course, git is not working, but I can work without problems.

To clarify what I think you meant, what I did was removed my individual project folders from the view on the left, and instead, I added a "project" that is really just a directory that contains multiple project folders in it. That seems to break the git coloring in the tree view, meaning my alt-tabs are now WAY faster, and I've just lost the coloring (which is OK for me right now, the speed is more important to me in the short term).

Great idea, thanks man.

This comment has been minimized.

Show comment Hide comment
@STLMikey

STLMikey Aug 26, 2016

@cy2k Im experiencing the same thing, opening the folder above my project the .git file(s) seems to drastically improve performance

@cy2k Im experiencing the same thing, opening the folder above my project the .git file(s) seems to drastically improve performance

This comment has been minimized.

Show comment Hide comment
@danielvoicu

danielvoicu Aug 29, 2016

I found a workaround that seems to fix the lagging issue. When I had this problem, I used Netdrive2 to mount the SFTP network drive. I started using Mountain Duck, and everything seems to be ok now, I can even see the git coloring and all, without any noticeable problems in performance.

I found a workaround that seems to fix the lagging issue. When I had this problem, I used Netdrive2 to mount the SFTP network drive. I started using Mountain Duck, and everything seems to be ok now, I can even see the git coloring and all, without any noticeable problems in performance.

This comment has been minimized.

Show comment Hide comment
@elchinillo

elchinillo Sep 21, 2016

Same thing for me. I have a network location on windows 8, which is a folder shared using Samba on an Ubuntu VM. I have to wait several seconds to open an small file.

Same thing for me. I have a network location on windows 8, which is a folder shared using Samba on an Ubuntu VM. I have to wait several seconds to open an small file.

This comment has been minimized.

Show comment Hide comment
@Celadora

Celadora Dec 29, 2016

Disabling the package "Git Diff" seemed to help, but I haven't confirmed that yet, it might be a fluke.

Disabling the package "Git Diff" seemed to help, but I haven't confirmed that yet, it might be a fluke.

This comment has been minimized.

Show comment Hide comment
@julenpardo

julenpardo Feb 28, 2017

Same here with Linux Mint 18 and:

Atom    : 1.14.2
Electron: 1.3.13
Chrome  : 52.0.2743.82
Node    : 6.5.0

Same here with Linux Mint 18 and:

Atom    : 1.14.2
Electron: 1.3.13
Chrome  : 52.0.2743.82
Node    : 6.5.0

This comment has been minimized.

Show comment Hide comment
@arashdb

arashdb May 3, 2017

I am observing this issue (both on windows(64bit) and on Ubuntu). If I have a few files open from my network drive and close Atom, next time I open up Atom, all those tabs (expectedly) open up, but it freezes and will not respond to any commands unless I open up a new window and close the frozen window. If all the open files are local (and not on a network drive) this issue will not happen.
I did turn off the "Git Diff" and saw no change in the behavior. Is there any chance that it might be an indexing issue?

arashdb commented May 3, 2017

I am observing this issue (both on windows(64bit) and on Ubuntu). If I have a few files open from my network drive and close Atom, next time I open up Atom, all those tabs (expectedly) open up, but it freezes and will not respond to any commands unless I open up a new window and close the frozen window. If all the open files are local (and not on a network drive) this issue will not happen.
I did turn off the "Git Diff" and saw no change in the behavior. Is there any chance that it might be an indexing issue?

This comment has been minimized.

Show comment Hide comment
@nkkollaw

nkkollaw May 5, 2017

The issue seems to be caused by the .git folder, or more precisely by the tree view trying to color files and folders according to their status.

If you open the directory above the .git folder it loads just fine, although you don't have the coloring.

It would be nice if tree view asked you whether you want to keep parsing the directory structure when it detects performance issues (ex., after 20 seconds or so).

nkkollaw commented May 5, 2017

The issue seems to be caused by the .git folder, or more precisely by the tree view trying to color files and folders according to their status.

If you open the directory above the .git folder it loads just fine, although you don't have the coloring.

It would be nice if tree view asked you whether you want to keep parsing the directory structure when it detects performance issues (ex., after 20 seconds or so).

This comment has been minimized.

Show comment Hide comment
@karneaud

karneaud Sep 24, 2017

+1 having the same issue with latest Atom on XOS Seirra with AFP network drive.

+1 having the same issue with latest Atom on XOS Seirra with AFP network drive.

This comment has been minimized.

Show comment Hide comment
@quickdraw6906

quickdraw6906 Sep 30, 2017

Same on Samba. Disabling the Atom git package and removing the project folder makes it lickety-speedy. What a bummer. Workaround:

  1. Don't use a mapped drive anymore. Clone the repo locally.
  2. Install remote-sync package.
  3. Follow instructions for package (great documentation!) to create a .remote-sync.json file in your directory to watch.
  4. Add that file to .gitignore. Super fast SCP sync whenever you save a file.
  5. Go into your .ssh dir. i.e., /home/sean/.ssh
  6. Create a new ssh key with ssh-keygen. Copy the contents of the .pub file to the clipboard.
  7. On the sync target, go into the .ssh directory of the user you put in the config file. 'data' is the user in my case:
    cd /home/data/.ssh
  8. Create a file there called authorized-keys
  9. Paste the contents of the key and save the file.

Didn't have to restart Atom or anything. Lightening fast!

quickdraw6906 commented Sep 30, 2017

Same on Samba. Disabling the Atom git package and removing the project folder makes it lickety-speedy. What a bummer. Workaround:

  1. Don't use a mapped drive anymore. Clone the repo locally.
  2. Install remote-sync package.
  3. Follow instructions for package (great documentation!) to create a .remote-sync.json file in your directory to watch.
  4. Add that file to .gitignore. Super fast SCP sync whenever you save a file.
  5. Go into your .ssh dir. i.e., /home/sean/.ssh
  6. Create a new ssh key with ssh-keygen. Copy the contents of the .pub file to the clipboard.
  7. On the sync target, go into the .ssh directory of the user you put in the config file. 'data' is the user in my case:
    cd /home/data/.ssh
  8. Create a file there called authorized-keys
  9. Paste the contents of the key and save the file.

Didn't have to restart Atom or anything. Lightening fast!

This comment has been minimized.

Show comment Hide comment
@btn-ninja

btn-ninja Oct 17, 2017

I tried Atom with Expandrive and it was unbearably slow. SublimeText3 with Expandrive was okay, but had random un-alerted failures to save, which cost me 1-2 hours per week; that solution also didn't work with the sass compiler. Does anyone recommend a solution, any editor/virtual drive combo which works reliably?

I tried Atom with Expandrive and it was unbearably slow. SublimeText3 with Expandrive was okay, but had random un-alerted failures to save, which cost me 1-2 hours per week; that solution also didn't work with the sass compiler. Does anyone recommend a solution, any editor/virtual drive combo which works reliably?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment