Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

User sync (cache refresh) in a containerised environment #1096

Open
skybert opened this issue Aug 14, 2018 · 5 comments

Comments

@skybert
Copy link
Contributor

commented Aug 14, 2018

When running oxTrust in a container, it's no longer predictable which IP it will have. In a docker-compose stack, it is instead referred to a name, like oxtrust and normally this is not a problem, configuration files just refer to its Docker/container stack local host name oxtrust and Docker makes sure that the containers within a compose stack can resolve it correctly.

However, this change to oxTrust back in 2014 introduced IP checking when performing the user sync and it thus makes it hard to set up AD/LDAP user sync in a containerised environment as this IP might change every time a docker-compose stack is brought down and up again: c2b808a

Perhaps a way to solve this, is to allow for oxTrustCacheRefreshServerIpAddress to be a host name and check this if the IP test fails. It's in any case a problem I expect will trouble anyone wanting to run AD/LDAP sync in a Gluu system running in containers.

Some useful pieces of code:

services:
  oxtrust:
    image: gluufederation/oxtrust:3.1.2_01
    # This assigns oxtrust as the hostname to the container, 
    # which otherhwise would have gotten the Docker ID as 
    # its hostname
    hostname: oxtrust

Getting a hold of the the host name is then returned from the eth0 interface of the container:

import java.util.*;
import java.net.*;

public class n {
  public static void main(String[] args) throws Exception {
    for (NetworkInterface ni :Collections.list(NetworkInterface.getNetworkInterfaces())) {
      System.out.println(ni);

      for (InetAddress ia : Collections.list(ni.getInetAddresses())) {
        System.out.println(
            "  ia=" + ia
            + " hostname=" + ia.getHostName()
            + " canonical=" + ia.getCanonicalHostName());
      }
    }
  }
}

Output running inside the oxtrust container defined above:

/tmp # java n
name:eth0 (eth0)
  ia=/172.18.0.12 hostname=oxtrust canonical=oxtrust
name:lo (lo)
  ia=/127.0.0.1 hostname=localhost canonical=localhost
/tmp # 

@willow9886 willow9886 added this to the 3.1.4 milestone Aug 14, 2018

@earezki

This comment has been minimized.

Copy link
Contributor

commented Aug 15, 2018

Hello Skybert,

Thank you for reporting this issue, and providing all those details helped me to better understand it.

I've been looking into this, and indeed this would require constant configuration if your ip changes per deployement.
Unfortunately, using the host won't solve the issue either, because we can't guarantee the infrastructure's behaviour (Changing the OS, using Rkt ...).

Preferably, I would use an application specific handling mechanism, maybe a nodeID that is unique in a clustring environment (provide a way to disable it when using a single node).
But changing this now would be risky, as it would break for users who want to migrate into this newer version.

Have you tried to use a static ip in your container as a temporary solution ?

Kind regards,
El Mehdi

@earezki earezki added the enhancement label Aug 18, 2018

@earezki earezki modified the milestones: 3.1.4, 4.0 Aug 19, 2018

@yurem

This comment has been minimized.

Copy link
Contributor

commented Aug 20, 2018

I think we should allow to write conditions for this. I offer add new method to Cache Refresh script boolean isReady()
Also we can move code which check IP address to NetworkService. After that if:

    def getApiVersion(self):
        return 2

CacheRefreshTimer should call isReady() method to determine if we need to run CR now and on this node.

In this flow admin can write own logic for this to conform his environment.

@skybert

This comment has been minimized.

Copy link
Contributor Author

commented Aug 20, 2018

Hi @earezki ,

thanks for your reply. I haven't had the chance to check out assigning an IP to the container, yet. By the look of it, that would indeed be a workaround in our environment. Will keep you posted.

because we can't guarantee the infrastructure's behaviour (Changing the OS, using Rkt ...).

I don't understand what you mean with "changing the OS" would make querying a hostname not an option. As for rkt, AFAICT from the rkt documentation, setting the hostname is supported, but you may know this better than me.

I'm looking forward to seeing a solution to this which works in in all your supported container environments.

Keep up the good work.

Cheers,

-Torstein

@yurem

This comment has been minimized.

Copy link
Contributor

commented Aug 20, 2018

Here is sample method implementation:

    def isAllowedToStart(self, configurationAttributes):
        serverInterface = configurationAttributes.get("primary_server_interface").getValue2()
        networkService = CdiUtil.bean(NetworkService)
        result = networkService.hasInterface(serverInterface)

        return result

    def getApiVersion(self):
        return 2
@skybert

This comment has been minimized.

Copy link
Contributor Author

commented Aug 22, 2018

For your reference, setting a static IP with docker-compose syntax version 3, involves:

networks:
  um-net:
    ipam:
      config:
        - subnet: 172.18.0.0/16

The oxtrust container gets its static IP like this:

services:
  oxtrust:
    networks:
      um-net:
        ipv4_address: 172.18.0.7 

And then adding every container to this network. You don't have to assign an IP to all the other containers, but you need to tell docker-compose that they should be included in it. Otherwise, Docker will not set up sufficient routing between the containers:

services:
  oxauth:
      networks:
        um-net:
  nginx:
    networks:
      um-net
[..]

Tested with:

Docker version 18.06.0-ce, build 0ffa825
docker-compose version 1.21.2, build a133471

@yurem yurem assigned syntrydy and unassigned earezki Jan 15, 2019

@yurem yurem modified the milestones: 4.0, 4.1 Mar 5, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.