Add implementation of libgit2 (Follow configuration from source-controller)

[Nordes]

Now on source-controller we can create a gitrepository resource for azure using implementation libgit2. Related bug (closed) is fluxcd/source-controller#104

Example:

flux create source git flux-system \
  --git-implementation=libgit2 \
  --secret-ref=git-ssh-key \
  --url=ssh://git@ssh.dev.azure.com/v3/org/collection/my_flux_repository \
  --branch=main \
  --interval=1m

When the image-automation-controller try to get from git, it fails since it's not following the git-implementation directive.

It would be very nice to be able to have the full flow working. (Adding flux -to- auto-update)

Right now the error is as below:

{"level":"error","ts":"2020-12-16T18:12:00.877Z","logger":"controller","msg":"Reconciler error","reconcilerGroup":"image.toolkit.fluxcd.io","reconcilerKind":"ImageUpdateAutomation","controller":"imageupdateautomation","name":"flux-system","namespace":"flux-system","error":"unable to clone 'ssh://git@ssh.dev.azure.com/v3/org/collection/my_flux_repository', error: empty git-upload-pack given"}

Images:
source-controller:v0.5.4
image-automation-controller:v0.1.0
image-reflector-controller:v0.1.0

[phillebaba]

@Nordes this will probably be done as image automation controller reaches an official release. It was a lot of work getting libgit2 in place for source-controller as it still has to work with the original library. Adding it now might probably make the development slower as it reaches the finish line.

I know that you are on vacation @squaremo so whenever you get back and see this can you comment if you have any thoughts on the matter :)

[Nordes]

Thanks a lot for the update and all your work :). Once it's done, I will be able to upgrade from V1 to V2 ;).

[squaremo]

@phillebaba Is there a possibility that only clones need to go through the libgit2 path, while commits and pushes can still be done by go-git? (My guess is that push involves the same bits of protocol that clone does, that is the bits that don't work in go-git+Azure ...)

[Nordes]

As the the error is the same I had when the clone was failing (before it was fixed to use go-git). I guess the same protocol is used for push. Which is a bit a pity (Azure Devops).

[phillebaba]

So I have spent some time debugging different git clients with Azure Devops and my experience has been that Azure Devops will reject any gitwire protocol request (https and ssh) that does not advertise the correct capabilities in the first handshaking part. So basically I do not think it is possible to commit with go-git.

[Nordes]

@squaremo: I know this have been closed already. I was only able now to go back and test it.

I was wondering why the checkout was not working? I can see the PR and the code change (image-automation-controller/controllers/imageupdateautomation_controller.go). I can see the libgit2 implementation for the push and commit. However, the checkout seems to fails for some reason. From within the code I see the reference to the CheckoutStrategyForRef which should take the proper implementation, but it seems to do something wrong after.

{"level":"error","ts":"2021-02-04T15:59:24.072Z","logger":"controller-runtime.manager.controller.imageupdateautomation","msg":"Reconciler error","reconciler group":"image.toolkit.fluxcd.io","reconciler kind":"ImageUpdateAutomation","name":"flux-system","namespace":"flux-system","error":"unable to clone 'ssh://git@ssh.dev.azure.com/v3/MyOrg/Proj/My.Flux.Configuration', error: user cancelled hostkey check"}

My SSH key and git repository are properly configured since the source-controller is able to fetch the content and deploy updates through the libgit2 gitImplementation. So, the hosts file must also be ok and that error should not happen.

Any ideas?

[containerpope]

@squaremo @Nordes
I am currently having the exact same issue with Azure DevOps. All my GitOps process works, but the automation controller is unable to execute the host key check. When there is an issue with the hostkey, the regular kustomizations should not be able to run or am I getting something wrong here.

│   Conditions:                                                                                                                                                                                                                                                                                                                                                  │
│     Last Transition Time:  2021-02-05T17:27:30Z                                                                                                                                                                                                                                                                                                                │
│     Message:               unable to clone 'ssh://XXX@vs-ssh.visualstudio.com/v3/XXX/XXX/XXX', error: user cancelled hostkey check                                                                                                                                                                                                │
│     Reason:                ReconciliationFailed                                                                                                                                                                                                                                                                                                                │
│     Status:                False                                                                                                                                                                                                                                                                                                                               │
│     Type:                  Ready                                                                                                                                                                                                                                                                                                                               │
│   Observed Generation:     1                                                                                                                                                                                                                                                                                                                                   │
│ Events:                                                                                                                                                                                                                                                                                                                                                        │
│   Type    Reason  Age                      From                         Message                                                                                                                                                                                                                                                                                │
│   ----    ------  ----                     ----                         -------                                                                                                                                                                                                                                                                                │
│   Normal  error   4m21s (x334 over 3d15h)  image-automation-controller  unable to clone 'ssh://XXX@vs-ssh.visualstudio.com/v3/XXX/XXX/XXX', error: user cancelled hostkey check

Is there any information I can extract, that might help you with this?

[stefanprodan]

@containerpope please open a new issue

[containerpope]

@stefanprodan no problem, please see: #106