feat(images): gcr pubsub push notification for image updates

[sjk07]

Add ability to listen to Google Pubsub push notifications from the google container registry.

This is still a WIP; fighting with my own cluster to setup things end-to-end and also add authentication token validation.
closes #18

[squaremo]

@sjk07 Do you need some help pushing this further? No worries if you are just busy doing other stuff :-)

[sjk07]

@squaremo I got everything working with Google auth stuff but got stuck when trying to write tests. Could use some help with that.

the Google docs tell you to pass the bearer token via API and Google will authenitcate for you and Slsince the handler doesn't have a client param im not able to mock out the call / return.

I also wanted to add some options for enabling authentication, expected audience, etc.
Happy to push up the code I have now and discuss

[squaremo]

Yes please, push any more WIP -- we can tidy up later :-)

So far I see you have a handler, and a test which goes as far as the other tests; is there extra work that must be done for it to operate in a real environment?

[sjk07]

The current code works as is in an environment but without authentication. Authentication changes a few things. I'll push up what I have with the auth implementation and we can work though the issues.

[sjk07]

@squaremo sorry for the delay, was on vacation 🍻 ; I have pushed up my code for the authentication piece, If you can provide some guidance with extending the config and testing would be much appreciated.

[squaremo]

Thanks Scott, I'll have a look.

was on vacation 🍻

Sounds like a good vacation :-P

[squaremo]

OK I think I have got my head around the authentication bit: each hook POST has an Authorization header with a token which can be checked against their OAuth2 service (there's some mention in the docs of caching responses, but: optimisation, for later, perhaps).

So the things to resolve are:

• Can authorisation be switched on in config, and how does that get wired through to the hander?

The change you made to the config struct is reasonable, I reckon, even if it only gets used for GCR. The endpoint could be given to the sourceHandler so it gets to see whether it should do authentication -- or some similar formulation.

• how to test that the authentication works

The simplest thing might be to swap out the URL for OAuth2 verification, and start an httptest.Server to do it for you. It's a bit of a juggling act! But even if you reassign a var just for testing, that's not too bad.

[sjk07]

@squaremo added the configuration stuff you talked about, let me know if this is what you envisioned. I didn't quite follow the testing of the auth, I just tested the authentication function directly and mocked the response back to show that it works as intended.

This is currently running on our development clusters with auth enabled. Ill have to go back and double check if no-auth works in a real env again but this should be pretty much good to go!

[tun0]

Hi @sjk07, what does it take to test this? Specifically stuff like configuration, on both the side of flux-recv and GCP/GCR.

[chainlink]

It might be a little easier to use this library https://github.com/golang/oauth2/blob/master/google/default.go

Specifically, you can pass it a service account json blob, and it handles translating that into a bearer token

[dmccaffery]

I think this all looks good, personally.

A short write up / example of setting up a GCR pubsub push subscription would be helpful for those who are not familiar.

Re: testing with a real token -- I don't think that's necessary.

@chainlink are you suggesting using a real token purely for testing purposes? The token is contained within the webhook post and is issued by GCP so you can prove the request came from them. There is no need to supply one to flux-recv in normal operation.

[dmccaffery]

@squaremo any other requirements on this PR before it can be merged? :)

[squaremo]

This is 🔝, thank you! I made a couple of suggestions; the main thing to do is rebase on master and update the new handler (Harbor) with the endpoint argument, I think.

[squaremo]

I did a double-take at .ToUpper vs. insert here. Perhaps either make the const value "insert" and compare it to .ToLower, or make the const name INSERT (though go lint might take offence at that, not sure).

[squaremo]

Can this be more explicit than %v? A minor thing.

[chainlink]

@chainlink are you suggesting using a real token purely for testing purposes? The token is contained within the webhook post and is issued by GCP so you can prove the request came from them. There is no need to supply one to flux-recv in normal operation.

Apologies, I thought this was reading the topic directly and thus would be need a service account.

[hiddeco]

@sjk07 I am happy to pick this up and address Michael's comments for you if this PR no longer has your attention.

[squaremo]

There's a stray file, I think, but looks good -- thanks @sjk07, @hiddeco and commenters 🍍

[sjk07]

@squaremo sorry for going MIA on this; Thanks @hiddeco for seeing this through!