Cosmic Explorer IT Infrastructure

This repository contains the resources and documentation necessary to set up the collaboration cyberinfrastructure used by Cosmic Explorer. We deploy four primary services:

• An instance of COmanage Registry to allow people to sign up to Cosmic Explorer and for the provisioning of associated services.
• An instance of the DocDB document server based on the fork used by the LIGO Document Control Center to manage the storage and retrieval of documents.
• A GNU Mailman instance for mailing lists.
• Integration with the cosmic-explorer organization on GitHub for collaborative repository management.
• A Shibboleth Identity Provider that acts as an ORCiD to SAML gateway for providing user identities.

This repository contains instructions for:

• Installing COmanage and setting up the registry.
• Installing and running an instance of the DCC.
• Installing and configuring Mailman.
• Configuring the Shibboleth IdP.

The repository provides two tools used by the services:

• An implementation of a RESTful API to the DCC DocDB Database that allows COmanage to provision authors and groups in the DCC.
• A helper container to determine when a port is open from inside a Docker container network.

The infrastructure also relies on the following repositories hosted by the cosmic-explorer GitHub organization:

• A fork of COmanage Regustry that contains the source for the fixed GitHub provisioner and the DCC provisioner until these are merged into the main COmanage repository.
• A fork of the Hydra Login and Consent Node used by the DCC to perform OAuth2 delegation of credentials to COmanage. This is essentially the same as the original version, but removes the foo@bar.com login as authentication is managed by Apache Shibboleth which reverse proxies to the consent node.
• A fork of the COmanage registry docker containers to allow us to make any CE specific changes. Currently this is even with the upstream repository as no patches are needed.

The infrastructure also relies on the following externally provided infrastructure:

• The Linux Server implementation of Let's Encrypt to obtain host certificates run from a Docker container.
• The Ory Hydra OAuth2 Server used to secure the RESTful interface to the DCC.
• Docker containers for Postgress and MariaDB for database support.

Server Setup

To create and deploy these services, we use a single VMware host machine ce-services.phy.syr.edu that contains three additional virtual NICs for the services that we will deploy:

• roster.cosmicexplorer.org, an alias to 128.230.146.12, internally known as ce-roster.phy.syr.edu.
• dcc.cosmicexplorer.org, an alias to 128.230.146.13, internally known as ce-dcc.phy.syr.edu.
• mail.cosmicexplorer.org, an alias to 128.230.146.15, internally known as ce-mail.phy.syr.edu.

The services themselves are run inside Docker containers on the machines listed above.

First set up the host networking to configure the multiple NICs to route to the 128.230.146.0 subnet correctly.

Install OpenLDAP so that the host can use the slappasswd tool:

yum -y install epel-release
yum config-manager --set-enabled PowerTools
yum install openldap-servers

Shibboleth Setup

These services should be federated as Shibboleth Service Providers with InCommon Research and Scholarship and have appropriate host certificates and Shibboleth metadata prior to configuring them.

The sugwg/apache-shibd Docker container can be used to create the Shibboleth metadata for federation to incommon. To do this, first obtain InCommon host certificates for each interface.

To create the Shibboleth metadata, run the commands below for each interface. First, make a directory for each interface

mkdir -p ce-mail ce-roster ce-dcc

and perform in the apache shibd configuration step in each directory.

COmanage

The default Shibboleth attribute map maps the user's given name and surname to two the variables givenName and sn. COmanage wants these to be stored in variables with a common string (we use name) with the suffixes _GIVEN and _FAMILY. This set up adds the <Attribute Resolver> elements needed to create these.

git clone https://github.com/sugwg/apache-shibd.git
cd apache-shibd/certificates
./keygen.sh
cd ..
cp /path/to/certs/ce-roster_phy_syr_edu_cert.cer certificates/hostcert.pem
cp /path/to/certs/ce-roster.phy.syr.edu.key certificates/hostkey.pem
cat >> assertion-consumer-service.xml <<EOF
	       <EndpointBase>https://roster.cosmicexplorer.org/Shibboleth.sso</EndpointBase>
           <EndpointBase>https://ce-roster.phy.syr.edu/Shibboleth.sso</EndpointBase>
EOF
cat >> provider-metadata.xml <<EOF
	<MetadataProvider type="XML" url="https://sugwg-ds.phy.syr.edu/sugwg-orcid-metadata.xml"
        backingFilePath="/var/log/shibboleth/sugwg-orcid-metadata.xml" reloadInterval="82800" legacyOrgNames="true"/>

        <AttributeResolver type="Template" sources="givenName" dest="name_GIVEN">
            <Template>\$givenName</Template>
        </AttributeResolver>
        <AttributeResolver type="Template" sources="sn" dest="name_FAMILY">
            <Template>\$sn</Template>
        </AttributeResolver>
EOF
docker build \
    --build-arg SHIBBOLETH_SP_ENTITY_ID=http://ce-roster.phy.syr.edu/shibboleth-sp \
    --build-arg SHIBBOLETH_SP_SAMLDS_URL=https://dcc.cosmicexplorer.org/shibboleth-ds/index.html \
    --build-arg SP_MD_SERVICENAME="Syracuse University Gravitational Wave Group - CE COmanage" \
    --build-arg SP_MD_SERVICEDESCRIPTION="Cosmic Explorer COmanage Roster" \
    --build-arg SP_MDUI_DISPLAYNAME="Syracuse University Gravitational Wave Group - CE COmanage" \
    --build-arg SP_MDUI_DESCRIPTION="Cosmic Explorer COmanage Roster" \
    --build-arg SP_MDUI_INFORMATIONURL="https://cosmicexplorer.org" \
    --rm -t cosmicexplorer/apache-shibd-roster .
    
docker network create --attachable \
    --opt 'com.docker.network.bridge.name=bridge-roster' \
    --opt 'com.docker.network.bridge.host_binding_ipv4'='128.230.146.12' \
    --driver=bridge \
    --subnet=192.168.100.0/24 \
    --ip-range=192.168.100.0/24 \
    --gateway=192.168.100.1 \
    bridge-roster

docker run --name=apache-shibd-roster --rm -d \
    --network=bridge-roster \
    --ip=192.168.100.2 \
    --hostname ce-roster.phy.syr.edu \
    --domainname phy.syr.edu \
    -v `pwd`/shibboleth:/mnt \
    -p 128.230.146.12:443:443 \
    cosmicexplorer/apache-shibd-roster:latest

DCC

git clone https://github.com/sugwg/apache-shibd.git
cd apache-shibd/certificates
./keygen.sh
cd ..
cp /path/to/certs/ce-roster_phy_syr_edu_cert.cer certificates/hostcert.pem
cp /path/to/certs/ce-roster.phy.syr.edu.key certificates/hostkey.pem
cat >> assertion-consumer-service.xml <<EOF
	       <EndpointBase>https://dcc.cosmicexplorer.org/Shibboleth.sso</EndpointBase>
           <EndpointBase>https://ce-dcc.phy.syr.edu/Shibboleth.sso</EndpointBase>
EOF
cat >> provider-metadata.xml <<EOF
	<MetadataProvider type="XML" url="https://sugwg-ds.phy.syr.edu/sugwg-orcid-metadata.xml"
        backingFilePath="/var/log/shibboleth/sugwg-orcid-metadata.xml" reloadInterval="82800" legacyOrgNames="true"/>
EOF
docker build \
    --build-arg SHIBBOLETH_SP_ENTITY_ID=http://ce-dcc.phy.syr.edu/shibboleth-sp \
    --build-arg SHIBBOLETH_SP_SAMLDS_URL=https://dcc.cosmicexplorer.org/shibboleth-ds/index.html \
    --build-arg SP_MD_SERVICENAME="Syracuse University Gravitational Wave Group - CE DCC" \
    --build-arg SP_MD_SERVICEDESCRIPTION="Cosmic Explorer DCC" \
    --build-arg SP_MDUI_DISPLAYNAME="Syracuse University Gravitational Wave Group - CE DCC" \
    --build-arg SP_MDUI_DESCRIPTION="Cosmic Explorer DCC" \
    --build-arg SP_MDUI_INFORMATIONURL="https://cosmicexplorer.org" \
    --rm -t cosmicexplorer/apache-shibd-dcc .
    
docker network create --attachable \
    --opt 'com.docker.network.bridge.name=bridge-dcc' \
    --opt 'com.docker.network.bridge.host_binding_ipv4'='128.230.146.13' \
    --driver=bridge \
    --subnet=192.168.101.0/24 \
    --ip-range=192.168.101.0/24 \
    --gateway=192.168.101.1 \
    bridge-dcc
        
docker run --name=apache-shibd-dcc --rm -d \
    --network=bridge-dcc \
    --ip=192.168.101.2 \
    --hostname ce-dcc.phy.syr.edu \
    --domainname phy.syr.edu \
    -v `pwd`/shibboleth:/mnt \
    -p 128.230.146.13:443:443 \
    cosmicexplorer/apache-shibd-dcc:latest

Mailman

git clone https://github.com/sugwg/apache-shibd.git
cd apache-shibd/certificates
./keygen.sh
cd ..
cp /path/to/certs/ce-roster_phy_syr_edu_cert.cer certificates/hostcert.pem
cp /path/to/certs/ce-roster.phy.syr.edu.key certificates/hostkey.pem
cat >> assertion-consumer-service.xml <<EOF
           <EndpointBase>https://mail.cosmicexplorer.org/Shibboleth.sso</EndpointBase>
           <EndpointBase>https://ce-mail.phy.syr.edu/Shibboleth.sso</EndpointBase>
EOF
cat >> provider-metadata.xml <<EOF
	<MetadataProvider type="XML" url="https://sugwg-ds.phy.syr.edu/sugwg-orcid-metadata.xml"
        backingFilePath="/var/log/shibboleth/sugwg-orcid-metadata.xml" reloadInterval="82800" legacyOrgNames="true"/>
EOF
docker build \
    --build-arg SHIBBOLETH_SP_ENTITY_ID=http://ce-mailman.phy.syr.edu/shibboleth-sp \
    --build-arg SHIBBOLETH_SP_SAMLDS_URL=https://dcc.cosmicexplorer.org/shibboleth-ds/index.html \
    --build-arg SP_MD_SERVICENAME="Syracuse University Gravitational Wave Group - CE Mailman" \
    --build-arg SP_MD_SERVICEDESCRIPTION="Cosmic Explorer Mailman Server" \
    --build-arg SP_MDUI_DISPLAYNAME="Syracuse University Gravitational Wave Group - CE Mailman" \
    --build-arg SP_MDUI_DESCRIPTION="Cosmic Explorer Mailman Server" \
    --build-arg SP_MDUI_INFORMATIONURL="https://cosmicexplorer.org" \
    --rm -t cosmicexplorer/apache-shibd-mail .
    
docker network create --attachable \
    --opt 'com.docker.network.bridge.name=bridge-mail' \
    --opt 'com.docker.network.bridge.host_binding_ipv4'='128.230.146.15' \
    --driver=bridge \
    --subnet=192.168.102.0/24 \
    --ip-range=192.168.102.0/24 \
    --gateway=192.168.102.1 \
    bridge-mail

docker run --name=apache-shibd-mail --rm -d \
    --network=bridge-mail \
    --ip=192.168.102.2 \
    --hostname ce-mail.phy.syr.edu \
    --domainname phy.syr.edu \
    -v `pwd`/shibboleth:/mnt \
    -p 128.230.146.15:443:443 \
    cosmicexplorer/apache-shibd-mail:latest

Download Metadata

Once the containers are running, the metadata can be obtained from the Shibboleth.sso/Metadata endpoint. Send the SP metdata to InCommon for federation.

Preserve the data that this container generates by copying the files attribute-map.xml, inc-md-cert.pem, shibboleth2.xml, sp-encrypt-cert.pem, and sp-encrypt-key.pem from the shibboleth/ to /etc/shibboleth on the host by running the commands

mkdir -p /etc/shibboleth
cp shibboleth/* /etc/shibboleth

Stop Apache Container

Finally, shut down the Apache container with

docker stop apache-shibd-roster apache-shibd-dcc apache-shibd-mail