gitfs clone failing silently

[jakehilton]

I have setup my gitfs to clone my repo as follows:

fileserver_backend:
  - git

gitfs_provider: gitpython
gitfs_base: master

gitfs_remotes:
    - ssh://git@some.gitlab.com:22/salt/myrepo.git

This seems to run fine as evidenced by the debug output when I run the sever in debug mode:

# salt-master -l debug
[DEBUG   ] Updating git fileserver cache
[DEBUG   ] Set lock for ssh://git@some.gitlab.com:22/salt/myrepo.git
[DEBUG   ] gitfs is fetching from 'ssh://git@some.gitlab.com:22/salt/myrepo.git'

The issue I am seeing is that it does create a git cache dir but the only thing in it is my .git directory. Inside that the config looks like this:

[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        url = ssh://git@some.gitlab.com:22/salt/myrepo.git
        fetch = +refs/heads/*:refs/remotes/origin/*
        fetch = +refs/tags/*:refs/tags/*
[http]
        sslVerify = False

When I run a 'git pull' from inside the cloned directory I get the following error:

# git pull
There is no tracking information for the current branch.
Please specify which branch you want to merge with.
See git-pull(1) for details

    git pull <remote> <branch>

If you wish to set tracking information for this branch you can do so with:

    git branch --set-upstream-to=origin/<branch> master

If I add the following to the .git/config it allows it to work:

[branch "master"]
        remote = origin
        merge = refs/heads/master

I am running on ubuntu 14.04 with python-git | 0.3.2~RC1-3.

My git version is 1.9.1

Thanks for any help.

[jakehilton]

I also just noticed that even when the logs show it as updating from the repo.. nothing is actually updated.

[DEBUG   ] Set lock for ssh://git@some.gitlab.com:22/salt/myrepo.git
[DEBUG   ] gitfs is fetching from 'ssh://git@some.gitlab.com:22/salt/myrepo.git'
[DEBUG   ] Removed lock for ssh://git@some.gitlab.com:22/salt/myrepo.git

The above is logged but the files are not updated.

[jakehilton]

Forgot to mention I'm running 2015.8.1

[jfindlay]

@jakehilton, thanks for the report.

@terminalmage

[terminalmage]

This is by design. When using gitfs, the repo is never cloned. It just doesn't make sense to do so, since we have to dynamically support all branches and tags as fileserver environments. So, we simply configure the "origin" remote to point at the desired git repo, and fetch it during the fileserver update. When a file is requested from a given tag/branch, the python interface to git assembles the file from the blob data.

You should never run a git pull from a gitfs cachedir. The fileserver cache will be updated roughly every 60 seconds, and can be triggered by using salt-run fileserver.update if an immediate refresh is desired.

As this seems to be a misunderstanding of how gitfs works, I am going to close this. If you are experiencing a specific problem, please respond and we can troubleshoot it.

[jakehilton]

Thank you for the reply.

That does clear up how it functions. Might I suggest documentation clarification so that users have a proper expectation?

Could you also clarify if the git_pillar is supposed to function similarly? I couldn't get that to work at all if I didn't clone the repo.

[terminalmage]

You really shouldn't ever need to interact with the directories managed by gitfs or git_pillar, I'm puzzled as to why you were doing so.

[jakehilton]

Because I can't get them to work properly without some sort of a push. Is it normal to have to tell each node to refresh pillar data when it's set to use ext_pillar? I would think/hope it would get the latest in the repo automatically.

I'm also puzzled as to why it's not more automatic.

[terminalmage]

git_pillar and gitfs remotes are refreshed in a "maintenance" process which does all its tasks and then sleeps for 60 seconds (this number is configurable via the loop_interval master config value). It's not really feasible to make it "automatic", as we'd be fetching every time you try to do something in Salt. It is possible via Salt's reactor system to use a git post-receive hook to trigger a gitfs update (see here).

If it is not possible to do a post-receive hook, then you can force a gitfs update by running salt-run fileserver.update -l debug (the -l debug is optional but I find it gives useful information). No such runner function exists to force a git_pillar update at this time.

Also, you may have incorrect assumptions about when pillar data is updated. It is only refreshed automatically when a minion starts, and when a state.highstate is run. Again, doing it every time you do something in Salt would just not be feasible. So, it is possible that your git_pillar repos are being updated, but your pillar data has not been refreshed, especially if you are using state.sls instead of state.highstate to execute Salt states. To force a refresh of pillar data, you can use saltutil.refresh_pillar.

[jakehilton]

Please pardon my misunderstanding.. I was under the impression that every 60 seconds salt would refresh the remote server backends. So that when a state.highstate was run on the minion the pillar data would be fresh.. aka I wouldn't have to manually tell each minion that they should refresh the pillar data from git. This somewhat defeats the purpose of an automated system. In order to get my data to the minions I do have to make a "salt '*' saltutil.refresh_pillar" call from the master. Is this the intended usage?

[terminalmage]

You only need to do so if you want the new pillar data in that interval of time between when you push to the repo and the next time the repo is updated.

[terminalmage]

Similarly, you only need to do a salt-run fileserver.update if you want access to changes pushed to a gitfs remote during the interval of time between the push and the next automatic fileserver update.

[jakehilton]

And therein lies the issue. I've waited upwards of 10 minutes and the minions never refresh the pillar data unless I explicitly signal them to do so. I'll revisit my configs and tests but what you described is what I expected.

[terminalmage]

OK, if it is not updating, that is not expected behavior. If you can produce a reproducible use case, I'd be happy to help troubleshoot.

One thing you might want to do is look at the master log and see if there are errors with updating your gitfs/git_pillar remotes.

[jakehilton]

I have been testing this weekend and it seems that when I push changes that eventually they do get to the minions and I've verified this by running a highstate on the box and seeing the updates.

What is now troubling is I've setup a schedule to run every hour to run highstate on the minions.. but it doesn't seem to be working. I've followed this guide in the highstates section: https://docs.saltstack.com/en/latest/topics/jobs/index.html#highstates

I've put my schedule in my top.sls file for my pillar:

schedule:
  highstate:
    function: state.highstate
    minutes: 60

I'm now hijacking this thread.. so if you'd rather me stop this madness here and continue elsewhere I'm more than happy to do that.. but it does seem related to the git_pillar remote files syncing.

[jakehilton]

I think I got it working.. will do some more testing to confirm.