Periodic/automatic repo fetches

[onecrayon]

Problem description

One of the features I miss from Git Tower is that it automatically ran git fetch at intervals. This ensured that my tags, remote branches, and so forth were up to date, and reduced the likelihood of me wasting time trying to commit stuff to a branch that has changed upstream.

Preferred solution

I would love if Sublime Merge periodically fetched repos for open windows, without needing me to explicitly initiate a fetch.

Alternatives

There are a lot of ways to approach this. At a low level, there could be a mechanism for automatically running commands when a window is opened or focused (maybe with a minimum amount of time since the last time the command automatically ran, in case users are swapping between windows regularly). There could be a mechanism for automatically running commands on a strict time interval. Either mechanism could be specific to fetching, and controlled via preferences. Etc.

Basically, all I care about as a user is that my sidebar stays up to date without needing me to babysit it, and the software isn't constantly pinging the server with fetch requests. :-)

[Zooce]

Might I suggest making this configurable as well. Some workflows may not favor this approach.

[jasonycw]

Or maybe add back the dedicated button for fetch next to pull like SourceTree that let user to make sure they can fetch properly.
Ctrl+R seems no UI feedback to show it did its work and not sure it is working.
F5 seems is bind to nothing, maybe a good key for doing fetch?
Currently the background fetch is a bit too slow and inconsistent.

[omertuc]

F5 seems is bind to nothing, maybe a good key for doing fetch?

Sounds like a great idea

[srbs]

F5 doesn't quite make sense as it is analogous to Cmd/Ctrl+R which is already bound to refresh (reload) the repository. Users could get confused as to which refresh keyboard combo to use for what they want.

Given that SM automatically refreshes the repository, it would make sense to have that be a less common or well known keyboard shortcut (Cmd+Option+R / Ctrl+Alt+R maybe) and let fetch be Cmd/Ctrl+R and F5.

[omertuc]

Given that SM automatically refreshes the repository, it would make sense to have that be a less common or well known keyboard shortcut (Cmd+Option+R / Ctrl+Alt+R maybe) and let fetch be Cmd/Ctrl+R and F5.

Fine with me but I think it would be a bit unfair for people who got used to using Ctrl+R for refreshing (even if it's for no reason)

[srbs]

would be a bit unfair for people who got used to [it]

Agreed, however:

1. it's a new product which no official "1.0" version as it were, I think it is perfectly reasonable to make breaking changes if the original decisions don't make as much sense once a bit of user feedback comes into play (keep in mind, space on the commit tree has changed behaviors; see also Don't use space to open the Contents Tree #256)
2. I would bet that no one really uses as it appears to do nothing & as mentioned before:

   Ctrl+R seems no UI feedback to show it did its work and not sure it is working.

[KiboOst]

Same here. Previously using github desktop, SM is amazing but I miss this automatic feature or at least a button in top bar to know I'm behind before trying to push.

[adriantombu]

Is there any news on this ? I work on multiple computers and forget times and times again to fetch / pull, it's really annoying. I was using Tower before and that is the feature that I miss the most now.

[GigaGrunch]

Since SM refreshes instantly when you do something in the CLI, you can run a simple bash script for fetching every now and then. It's by far not as pretty as a setting in SM would be, but it does the job.
Here is it as a one liner that fetches every minute and prints the time and directory each time (just so you can see that it doesn't stop 🤷), just paste it in a terminal (or Git Bash on Windows) or put it into a script file to store it.

while :; do echo "$(date +'%H:%M') | fetching '$(basename $(pwd))'..."; git fetch; sleep 60; done

[vladdeSV]

I would very much appreciate a periodic fetch. I sometimes forget to fetch when I start working on something.

Most commonly I would only need to fetch the current branch, not all.

[ratijas]

As I was reading about the issue, I just realized something.

The proper fix to this problem would be NOT a periodic check by a client (which is what we are doing manually anyway).

The real solution would be to extend core git protocol. Instead of short polling for updates, git server should adopt a command like IMAP IDLE to be able to communicate real-time notifications.

I did a preliminary research in google / duckduckgo, but to my surprise it doesn't seem like any work is being done in that direction. There was, however, one thread of discussion found at Git Mailing List Archive: "IMAP IDLE"-like long-polling "git fetch".

[ratijas]

Interestingly, Gerrit (code review tool & infrastructure) has been having this feature for ages: gerrit stream-events.

For example, one can monitor which patches and reviews are published to The Qt Project:

$ ssh codereview.qt-project.org gerrit stream-events

{"submitter":{"name":"Alexandru Croitor","email":"...","username":"alexandru.croitor"},"refUpdate":{"oldRev":"0000000000000000000000000000000000000000","newRev":"d0a1f5fbd8be58ca73947e3de279599de03345ee","refName":"refs/changes/25/317825/1","project":"qt/qtdeclarative"},"type":"ref-updated","eventCreatedOn":1602858068}
{"submitter":{"name":"Alexandru Croitor","email":"...","username":"alexandru.croitor"},"refUpdate":{"oldRev":"0000000000000000000000000000000000000000","newRev":"e1629f2bd714feff8ad041fc7fb5e99f94f2f77a","refName":"refs/changes/25/317825/meta","project":"qt/qtdeclarative"},"type":"ref-updated","eventCreatedOn":1602858068}
{"uploader":{"name":"Alexandru Croitor","email":"...","username":"alexandru.croitor"},"patchSet":{"number":1,"revision":"d0a1f5fbd8be58ca73947e3de279599de03345ee","parents":["6b20e0e08ed48982e849dbbf9fc53113a491f5e4"],"ref":"refs/changes/25/317825/1","uploader":{"name":"Alexandru Croitor","email":"...","username":"alexandru.croitor"},"createdOn":1602858068,"author":{"name":"Alexandru Croitor","email":"...","username":"alexandru.croitor"},"kind":"REWORK","sizeInsertions":72,"sizeDeletions":-72},"change":{"project":"qt/qtdeclarative","branch":"dev","id":"Ie5bba408000211b24694aa0143bdf79c4a298f42","number":317825,"subject":"CMake: Regenerate examples to use qt_add_executable","owner":{"name":"Alexandru Croitor","email":"...","username":"alexandru.croitor"},"url":"https://codereview.qt-project.org/c/qt/qtdeclarative/+/317825","commitMessage":"CMake: Regenerate examples to use qt_add_executable\n\nTask-number: QTBUG-87661\nChange-Id: Ie5bba408000211b24694aa0143bdf79c4a298f42\n","createdOn":1602858068,"status":"NEW"},"project":"qt/qtdeclarative","refName":"refs/heads/dev","changeKey":{"id":"Ie5bba408000211b24694aa0143bdf79c4a298f42"},"type":"patchset-created","eventCreatedOn":1602858068}
{"submitter":{"name":"Qt Sanity Bot","email":"qt_sanitybot@qt-project.org","username":"qt_sanity_bot"},"refUpdate":{"oldRev":"e1629f2bd714feff8ad041fc7fb5e99f94f2f77a","newRev":"c95dae1b459281b967f0dd78173cda8a982ffc5b","refName":"refs/changes/25/317825/meta","project":"qt/qtdeclarative"},"type":"ref-updated","eventCreatedOn":1602858069}
...

[vladdeSV]

As I was reading about the issue, I just realized something.

The proper fix to this problem would be NOT a periodic check by a client (which is what we are doing manually anyway).

The real solution would be to extend core git protocol. Instead of short polling for updates, git server should adopt a command like IMAP IDLE to be able to communicate real-time notifications.

I did a preliminary research in google / duckduckgo, but to my surprise it doesn't seem like any work is being done in that direction. There was, however, one thread of discussion found at Git Mailing List Archive: "IMAP IDLE"-like long-polling "git fetch".

That is a neat idea, however I do not think that is related to this specific issue. Extending the git protocol should be taken up with git itself?

[ratijas]

@vladdeSV

Indeed, that's what I wrote. It would require changes to the git protocol, and then waiting for major git servers' providers to upgrade, before widely adopting it in clients.

For the time being, spamming git fetch in a background loop seems like the only viable solution. But if the project of your choice is hosted with Gerrit, you could stream events, filtering them down to those concerning your branches, and trigger git fetch much less often, thus reducing traffic overhead.

[mariobaldini]

+1

GitKraken has this feature of background refresh and displaying a notification in the top bar when the remote became ahead by some commit, avoiding wasted time with local commit and push conflict.

[vladdeSV]

I would like to add that Visual Studio Code also has this feature.

[ratijas]

Surprisingly, no one here ever talked about how fundamentally incompatible periodic fetches are with force-pushes.

There is an option git push --force-with-lease[=<refname>[:<expect>]] which exists to prevent accidental overrides of someone else' changes. And you should always use be using it instead of dangerously destructive --push. Just get used to it, make it habit.

But automatic background refreshes would actually defeat even that safety mechanism — unless you know better how to pass those optional <refname>:<expect> arguments, that is.

I encourage everyone in this thread to take a look at the official git documentation, concerns which it raises and workarounds it offers.

I'm not here to teach you how to work, but generally when multiple people work on a single feature, it's better to merge commits from their personal branches every so often (possibly in a form of pull requests), instead of concurrently pushing to a shared branch. Your workflow may vary though. Personally, I'd prefer having some sort of opt-in "push notifications" (with multiple levels of irony) from git servers.

[vladdeSV]

Hey @ratijas. I don't quite understand your viewpoint here.

This issue is about the client periodically running git fetch. Your entire reasoning is either about extending the core git functionality or about force pushing a branch (?). This is not related to the issue.

Personally, I'd prefer having some sort of opt-in "push notifications" (with multiple levels of irony) from git servers.

This is not something Sublime Merge will be able to do unless it is a feature in git, correct?

generally when multiple people work on a single feature, it's better to merge commits from their personal branches every so often

I cannot argue with that, and completely agree. This issue however, is more geared towards when I myself work on my own projects on one machine, and later forget to do a pull on my other machine. Not only that, when collaborating, if I merge my feature branch into the main branch, but forget to do a pull first, the merge commits will look ugly and be mostly annoying.

I would suggest the moderation team mark our back and fourths as off-topic, since I do not feel they provide anything to the issue.

[mariobaldini]

@ratijas we are talking mostly about an app notification, not related to the git commands, repo or workflow -- which are left untouched and up to the user preference.

Just like you can visit the repo webpage and check the last commit timestamp;
the topbar could notify something like "this repo is behind of origin by N commits", or update the tree view.

Again, this would be purely an app action, not any git change to the local repo.
And informative: just letting you know that origin has new content and you later applying your workflow actions

[4ekki]

Is it possible to at least auto-fetch on application start?

[srbs]

@4ekki, not on startup, but I did write a way to get manual fetch all to work: https://forum.sublimetext.com/t/manual-fetch-all-command/57556

[CrepeGoat]

So I can see why someone might not want an auto-fetch feature. Fetching technically changes the commits accessible within a repo, so e.g., in a(n uncommon) corner-case where your local repo has a reference to a recently deleted upstream branch, fetching will remove that commit.

An alternative could be this: currently the fetch command is coupled in the same button as the pull command, but they could be separated so that fetch gets its own button. Then on this separate button, there could be a color indication of whether fetchable items are available, allowing the user to click the fetch button when prompted by the color indicator. (It could probably also use a tooltip showing the last check timestamp.) The user could also flip a setting to turn that "fetchable item check" into a full-fetch operation.

There are definitely some details to figure out re: how it checks for fetchables (I don't think this is a function of vanilla git) and when it should check (if it checks too often it's consuming bandwidth, if it check too infrequently it can mislead the user). RE: the "when", IMO it makes sense to refresh:

• no more than every [X] minutes (this can be a user setting; default to 5min?)
• on every refocus on a given git repo tab. e.g.:
  • clicking from another app into Sublime Merge
  • clicking from one Sublime Merge tab into another
  • switching from one desktop into another

But anyway, I think that could address the issue. Thoughts?

[ratijas]

There are definitely some details to figure out re: how it checks for fetchables (I don't think this is a function of vanilla git) and when it should check (if it checks too often it's consuming bandwidth, if it check too infrequently it can mislead the user). RE: the "when", IMO it makes sense to refresh:

This is a function of vanilla git. See git remote show origin output.

  Local branch configured for 'git pull':
    master merges with remote master
  Local ref configured for 'git push':
    master pushes to master (local out of date)

Somehow it knows that local branch is out of date. There might be simpler ways to check it though. I'm not much into git tricks.

[onecrayon]

I can see how auto fetching might mess up a local. The main functionality that I’m missing is auto updating “ahead/behind” badges in the sidebar. If it’s possible to update those without a fetch, I’d be all for that instead.

[Zwyx]

Just wanted to add my support for this issue.

I activated this exact feature in VS Code, however it works only when the project is open in VS Code. If I'm just checking a repo in Sublime Merge without having it open in VS Code, I will think it's up to date, because I'm used to that now, but it might not be.

About ratijas's comment on push --force-with-lease being broken by this feature: it is not relevant in, at least, my team's way of working. We don't work on shared branch, we do PRs on a main, protected branch. Having auto fetch is useful to quickly checkout someone else's branch, or to know if our branch needs to be rebased before making the PR, all that without having to fetch first.

To me, the idea behaviour would be to have the side bar in Sublime always up to date with the remote repo. In the same way that files changes are always up to date with the file system.

The Fetch button, next to the Pull one, is a good idea too. See also #1313.

[themilkman]

Related: #1500 (comment)

[Zwyx]

Excellent point, themilkman, git fetch --dry-run would be very useful for those not willing to have unexpected overrides with git push --force-with-lease. Having this as a configurable option (either fetch periodically, or fetch --dry-run periodically and show the status) would be excellent.

[dpjohnst]

Hi all,

Thank you for sharing your feedback!

We've added this in Sublime Merge build 2080 (development), and we're excited for you to try it 🎉
Please leave any feedback you have here and we'll use this to tweak this functionality in future builds.

Kind regards,
- Dylan from Sublime HQ

[Zwyx]

Hi @dpjohnst,

Thank you for working on this feature and letting us know about it! It seems to be working well.

Maybe it could be useful to be able to activate this "per-repository".

I always have plenty of repos open, some of them shared with my team, some of them not. It makes sense to auto fetch often the shared repos, but not the others (for instance, my dot files repo: I only push to it).

A way to implement this could be to add a checkbox Automatically fetch this repository somewhere here:

when the main auto fetch setting is active.

This is nothing more than a "nice to have" though, I'm pretty happy autofetching all my open repositories.

Cheers!

[themilkman]

Hi @djohnston, extracted from a discord discussion a possible problem I see with the current implementation:
After a rebase or so if I want to force push (and I know its implications and dangers) a commit/branch. --force-with-lease won't let me do that if the remote refs have changes (e.g. someone else pushed a commit) which I would overwrite.
Now, when SublimeMerge performs that auto-fetch e.g. when I use the Command Palette to navigate to "Push... -> push --force-with-lease" and it fetches in that second while I do that, my local remote-refs are updated and git performs the force-push anyway - even I (as a human) didn't notice the remote changes as I didn't see them.
Also already described here: #1500 (comment)

[Zwyx]

Hi @themilkman,

I believe that automatic fetching is not recommended if multiple persons are susceptible to commit on the same branch.

However, in teams where no one ever commit on someone else's feature branch, automatique fetching is useful (to not have to fetch before checking out someone else's branch in order to review their PR, among others).

Having multiple people committing on the same branch might be common in teams following the git-flow branching model, but I believe is less common and not recommended when following the simpler GitHub-flow. The git-flow author himself recommends GitHub-flow for simple and continuously delivered software.

To conclude, I believe that turning ON or OFF this option really depends on the context you work in. And the fact that it is turned OFF by default is a good thing.

(Personal note: in my team, no one commits on someone else's branch. I've had automatic fetching active in VS Code for a long time, and it works well for me. Having it now in Sublime is even better: the repo doesn't need to be opened in VS Code to be kept up to date.)

[dpjohnst]

Hi all,

This has now been added to Sublime Merge 2083 (Stable) 🎉

Cheers,
- Dylan from Sublime HQ