Bootstrap Keycloak using Admin CLI / REST API

[alexeykazakov]

Docs: http://www.keycloak.org/docs/3.3/server_admin/topics/admin-cli.html

Moved from fabric8-services/keycloak#42

[jstrachan]

I'm thinking its maybe easier to just use the REST API directly from golang.

We do something like that already in fabric8-tenant when using kubernetes so that we can add the redirect URI of a tenants jenkins into the fabric8-online-platform client so that folks can login via SSO to the tenants new jenkins (as wildcards are not allowed in redirect URIs).. e.g. this code GETs some JSON from KeyCloak, modifies it and PUTs it back
https://github.com/fabric8-services/fabric8-tenant/blob/master/openshift/kube_keycloak.go#L48-L77

we're about to do something similar in https://github.com/fabric8io/exposecontroller/ so that we can enable/disable https dynamically as part of the exposecontroller initialisation/installation (can't find the issue right now).

I figure we could maybe have a little library of golang code for working with the KeyCloak REST API then use it from fabric8-auth / fabric8-tenant / exposecontroller; maybe over time we can refactor the code to just leave all KeyCloak code in fabric8-auth and add simple REST APIs that fabric8-tenant or exposecontroller can use (to add a redirect URI or enable/disable https respectively)?

[alexeykazakov]

Sounds good too me. We can use REST API for that. I've changed the title.
And also agree that over time we should leave that code in Auth and other services will use it via Auth REST API. So, KC can remain an implementation detail.

[jstrachan]

Agreed.

Plus it might be nice when running upstream fabric8 on, say, GKE - to just delegate to Google's OAuth stuff for example

[sbose78]

Hi @jstrachan , we started working on this -
At the moment, @kishansagathiya is looking at how to configure keycloak over the REST APIs by passing over a JSON file , and calling specific APIs directly ( a combo of json file and direct API calls for specific tasks. )

Do you have any design ideas about this, that would help fabric8 upstream ?

[rawlingsj]

Not sure if this helps but I can describe how we currently auto configure KC in fabric8.

Currently we have a configmap for the keycloak realm.json https://github.com/fabric8-apps/keycloak-app/blob/master/keycloak/src/main/fabric8/cm.yml

We then use exposecontroller / configmapcontoller to replace placeholders such as redirect URLs once they're created at runtime https://github.com/fabric8-apps/keycloak-app/blob/master/keycloak/src/main/fabric8/cm.yml#L49-L54. We then use an init container to do the parameter substitution and mount the json onto a shared volume in the KC pod https://github.com/fabric8-apps/keycloak-app/blob/master/keycloak/src/main/fabric8/deploymentConfig.yml#L44-L142.

Right now our biggest issue is when the KC pod is restarted we use the -Dkeycloak.migration.strategy=IGNORE_EXISTING migration flag which means any manual KC config updates are lost. We cant use the other strategy either as that means we can't rollout updated config changes in subsequent releases.

[sbose78]

Thanks @rawlingsj !

This makes sense. So effectively it would be nice if something like Auth or a Go binary can call keycloak and 'check' if it has everything needed, else - call the REST API to 'initialize' keycloak with all the config ?

[jstrachan]

so not being able to ever do migrations is fairly major ;). It doesn't seem the KC can survive a reboot of your laptop either btw.

also we couldn't figure out a way to configure the IDP as the JSON seemed to require primary keys; so if there was some magic way to define logical entities like clients + IDPs in a way that can handle migrations (e.g. so we could later on change a property of a particular client - etc) that'd be awesome.

Basically we want an upsert kinda behaviour really - for any IDP / client we want to create it if its not there or update the values we give if they are provided - otherwise leave all the clients / users as they are in the DB

[jstrachan]

@sbose78 agreed - we should move away from this -Dkeycloak.migration.strategy=IGNORE_EXISTING approach which doesn't really seem fit for purpose and instead have some kind of init container that can read a blob of JSON from a ConfigMap and do upsert operations on the parts we need to insert/update. If that init-container could also do env var substitutation as well - that'd simplify our lives too ;) e.g. replacing the real URLs for things which are often not known until runtime (redirect URLs etc)

[kishansagathiya]

@sbose78 @alexeykazakov You can configure Keycloak using REST APIs by passing over a JSON file. I have put together a small shell script , a sample JSON testrealm.json

[kishansagathiya]

@sbose78 @alexeykazakov Keycloak can be configured by passing a json file. I have put together a small shell script for that, testrealm.json here

The script contains two api requests, first one GETs the access token and second one POSTs the json content to create the realm

[jstrachan]

@kishansagathiya a good test case will be, can you configure a primary identity provider via this new JSON mechanism

[sbose78]

@jstrachan we just had a discussion in the auth chat to do something like this , and avoid calling APIs with a json file..

if ( fabric8 realm is not present )
                - call API to create new realm
                - call APIs to accomplish the 'diff' between the stuff in default realm and what we need to have.

What do you think? :)

That'll help us handle changes in a nice version'd manner.

[jstrachan]

yeah - though we need upsert though - create the realm/client/identity provider - or if it already exists update the fields that are in the JSON we configure by hand.

its a bit like we want kubectl apply - f foo.json - create it if its not there and update it if it is - but only update the bits of JSON supplied; ignore the rest

[sbose78]

Yup, that's the idea. We are on the same page. 👍

[alexeykazakov]

BTW maybe instead of making workaround / own implementation we should contribute some better and more flexible configuration mechanism to Keycloak. Something which will help us a lot to use on our side.

[jstrachan-testing]

its worth a discussion for sure. e.g. maybe there could just be some 'configure' REST endpoint that we post all the resources we need to be inserted/updated as a big blob of JSON and KeyCloak does the right thing? Am sure other KeyCloak users will want something similar

[kishansagathiya]

As per the REST API doc, we can

• Update the top-level information of the realm Any user, roles or client information in the representation will be ignored.
• Partial import from a JSON file to an existing realm.

@jstrachan It is also possible to create a new identity provider( as well as update) using REST API by passing JSON, http://www.keycloak.org/docs-api/3.1/rest-api/index.html#_identity_providers_resource

[xcoulon]

@sbose78 is this issue still relevant?

[sbryzak]

No longer relevant.