Django SAML MDQ

A lightweight SAML2 MDQ server that implements draft-young-md-query implementation:

1. Runs on top of the metadata downloaded and validated by pyFF (batch pipeline)
2. Have signing features (on top of xmlsec)
3. Have ValidUntil definitions feature
4. Supports the following Entities Identifiers: urlencoded, {sha1} and {base64}

Remember that pyFF is needed for metadata downloading, it can run as daemon or as a scheduled process (batch)

Table of contents

• Installation

  • Requirements
  • Configure pyFF
  • Test the pipeline
  • Configure Django MDQ

• Client configurations

  • Shibboleth IdP Configuration
  • PySAML2

• Performances

  • Performances
• Authors
• Credits

Requirements

apt install build-essential xmlsec1 python3-dev libxml2-dev libxslt1-dev libyaml-dev python3-pip
pip3 install --upgrade pip
pip3 install virtualenv django lxml xmlsec

Install pyFF

virtualenv -p python3 python-pyff
source python-pyff/bin/activate
pip3 install git+https://github.com/IdentityPython/pyFF.git

Configure pyFF

# Create a folder for the configuration
mkdir pyff-configuration
cd pyff-configuration

# create folder for the certificates
mkdir certificates

# create certificates
openssl req -nodes -new -x509 -days 3650 -keyout certificates/private.key -out certificates/public.cert -subj '/CN=your.own.fqdn.com'

# create a pipeline directory
mkdir pipelines

Create a pipelines to fetch and handle all the Idem + eduGAIN metadata, this would be similar to the following. Name it pipelines/garr_batch.fd:

# Metadata download and validation
- load xrd garr-loaded.xrd:
  - ./pipelines/garr.xrd
# select can even filter entity by IDPSSO or SPSSO Description and things ...
# - select: "!//md:EntityDescriptor[md:SPSSODescriptor]"
- select
- store:
     directory: ./garr
- publish:
     output: ./garr/garr-loaded.xml
- stats

Now create the XRD file where to configure the URLs where the Metadata can be downloaded. Name it pipelines/garr.xrd

<?xml version="1.0" encoding="UTF-8"?>
<XRDS xmlns="http://docs.oasis-open.org/ns/xri/xrd-1.0">
  <XRD>
    <Link rel="urn:oasis:names:tc:SAML:2.0:metadata" href="http://md.idem.garr.it/metadata/idem-test-metadata-sha256.xml"/>
  </XRD>
  <XRD>
    <Subject>http://md.idem.garr.it/metadata/edugain2idem-metadata-sha256.xml</Subject>
    <Link rel="urn:oasis:names:tc:SAML:2.0:metadata" href="http://md.idem.garr.it/metadata/edugain2idem-metadata-sha256.xml">
        <Title>IDEM+eduGAIN</Title>
        <ds:KeyInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
            <ds:X509Data>
                <ds:X509Certificate>
                MIIDWzCCAkOgAwIBAgIJALo/EGIq8rgNMA0GCSqGSIb3DQEBCwUAMEQxCzAJBgNV
                BAYTAklUMRYwFAYDVQQKDA1JREVNIEdBUlIgQUFJMR0wGwYDVQQDDBRJREVNIE1l
                dGFkYXRhIFNpZ25lcjAeFw0xOTAxMjIxNjA5MjBaFw0yMjAxMjExNjA5MjBaMEQx
                CzAJBgNVBAYTAklUMRYwFAYDVQQKDA1JREVNIEdBUlIgQUFJMR0wGwYDVQQDDBRJ
                REVNIE1ldGFkYXRhIFNpZ25lcjCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoC
                ggEBAMay3N21fswu3AE6hqCPUVjvCyol5OKTHs9CXDIFyAoigP+YSdloLSGwx6n6
                ks9aBbJqlzRBIEd3CpByvX7GmBuITl3ElhxMY40Cv/ULok1GbDmQMhPScU6J1f9b
                526R9Ks+BbYZYmBRX9gqmpX1R867IES4z+JhXnXr5K8HTPjfaDGh2xORL6msXjww
                DJgaJCOpBCctLvCWcmUp0ucpl8VHGjFAAI5Eb6pwQEEPj1yqW52ggM+AHNFY6bAC
                9RX7Qv8MonQZwXpNNBNL+UcnGLVBXtBftd4zq7XxPNN9F/Ele3YJGaOVk8cCEJt5
                SfTeguzUaAyh8f/BfEs6CwucCSsCAwEAAaNQME4wHQYDVR0OBBYEFCZQVW7g6mc9
                3zaJP/p0lGbVQ4O6MB8GA1UdIwQYMBaAFCZQVW7g6mc93zaJP/p0lGbVQ4O6MAwG
                A1UdEwQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAF6OKKdWyeI385ZS5i29mSMA
                4BoPCVAhyXDMLMdqTQqvZp3PAL/zjLYRYKgGH53d4uN/EztWM8YBdyzBzdbpFWpd
                wRGzwyfXzt6l2luElWb59PacNqHbBkyFO2YZmgqLzgrVX1gA3/3ij9zrLqd1lHVH
                MHPUpqv98KYXnttyzhacdYaRGDO/2A28U9QeRq2/HgVScklhJvoySeNyXNspYfte
                ePRxeHBj21DgiQb+X1+ovKASM+RULa6cA1TJBCop+VqZMZiRJ3Rj6RML63ckEO8H
                Md/XFvxlr+P2JcVKzHaZEEUGGINUCCuDABqKBZOqykGWXDastVw6/I0OIdLmWNI=
                </ds:X509Certificate>
            </ds:X509Data>
      </ds:KeyInfo>
    </Link>
  </XRD>
</XRDS>

Test the pipeline

pyff pipelines/garr.fd

You should have an output of this kind

total size:     6003
selected:       6003
          idps: 3257
           sps: 2744

Configure Django MDQ

If you need to integrate saml2_mdq in a preexisting django project you can install it as an app:

pip install saml2_mdq

Then add saml2_mdq into your settings.INSTALLED_APPS, it doesn't need migrations. Add also django.middleware.http.ConditionalGetMiddleware in settings.MIDDLEWARE to enable ETag in the HttpResponse headers.

If you instead just need a fully working MDQ server you can copy the entire project this way

git clone https://github.com/UniversitaDellaCalabria/Django-MDQ.git
cd Django-MDQ
pip install -r requirements

Then

1. Copy django_mdq/settingslocal.py.example to django_mdq/settingslocal.py and edit it
2. in django_mdq/settingslocal.py configure:
   • PYFF_METADATA_FOLDER must point to the folder where the pyFF downloads periodically the metadata xml files
   • PYFF_METADATA_LOADED must point to the full metadata xml published by pyff, containing all the entities
   • METADATA_SIGNER_KEY and METADATA_SIGNER_CERT to enable Metadata signing features (optional, not required)
   • METADATA_CACHE_CONTROL to set the Http-Header Cache control max-age
   • METADATA_VALID_UNTIL to set the freshness of the metadata
3. This projects doesn't need of any database configuration
4. Run it in development mode ./manage.py runserver 0.0.0.0:8001 or in production one (see gunicorn or uwsgi examples to do that)

To create your Metadata RSA keys you can even use this example command:

openssl req -nodes -new -x509 -days 3650 -keyout certificates/private.key -out certificates/public.cert -subj '/CN=your.own.fqdn.com'

Shibboleth IdP Configuration

This is a metadata-provider definition file that can be included in/opt/shibboleth-idp/conf/services.xml. In the MetadataProviders resources, called <util:list id="shibboleth.MetadataResolverResources"> we can put it as the child element <value>%{idp.home}/conf/metadata-providers-mdq.xml</value>. Just change django_mdq.url in your production url.

<?xml version="1.0" encoding="UTF-8"?>
<!-- This file is an EXAMPLE metadata configuration file. -->
<MetadataProvider id="ShibbolethMetadataMdq" xsi:type="ChainingMetadataProvider"
    xmlns="urn:mace:shibboleth:2.0:metadata"
    xmlns:resource="urn:mace:shibboleth:2.0:resource"
    xmlns:security="urn:mace:shibboleth:2.0:security"
    xmlns:md="urn:oasis:names:tc:SAML:2.0:metadata"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="urn:mace:shibboleth:2.0:metadata http://shibboleth.net/schema/idp/shibboleth-metadata.xsd
                        urn:mace:shibboleth:2.0:resource http://shibboleth.net/schema/idp/shibboleth-resource.xsd
                        urn:mace:shibboleth:2.0:security http://shibboleth.net/schema/idp/shibboleth-security.xsd
                        urn:oasis:names:tc:SAML:2.0:metadata http://docs.oasis-open.org/security/saml/v2.0/saml-schema-metadata-2.0.xsd">

    <MetadataProvider id="DynamicEntityMetadata" xsi:type="DynamicHTTPMetadataProvider"
                  connectionRequestTimeout="PT5S"
                  connectionTimeout="PT5S"
                  socketTimeout="PT3S">

    <!-- Enable this if have configured METADATA_SIGNER_KEY and METADATA_SIGNER_CERT in Django-MDQ settingslocal.py
    <MetadataFilter xsi:type="SignatureValidation" requireSignedRoot="true"
                    certificateFile="%{idp.home}/credentials/mdq-cert.pem"/>
    -->

    <!-- Enable this if have configured METADATA_VALID_UNTIL in Django-MDQ settingslocal.py
    <MetadataFilter xsi:type="RequiredValidUntil" maxValidityInterval="P14D"/>
    -->

    <MetadataQueryProtocol>https://django_mdq.url/</MetadataQueryProtocol>
    </MetadataProvider>

</MetadataProvider>

Test the configuration

# reload ShibbolethIdP or Metadata Service
touch /opt/jetty/webapps/idp.xml

# do a mdquery
/opt/shibboleth-idp/bin/mdquery.sh -e https://coco.release-check.edugain.org/shibboleth --saml2 -u http://localhost:8080/idp

A test with PySAML2

import io
import json
import urllib.request

from saml2.mdstore import MetaDataMDX

def b64(entity_name):
    return '{base64}'+base64.b64encode(entity_name.encode()).decode()

# when available
mdq_url = "http://localhost:8001"
mdq_cert = "certificates/public.cert"

entity2check = 'https://idp.unical.it/idp/shibboleth'

cert = open(mdq_cert)

# omit cert if unavailable
mdx = MetaDataMDX(mdq_url, cert=cert)
# base64 entity name trasformation
# mdx = MetaDataMDX(mdq_url, cert=cert, entity_transform=b64)

# certificati
mdx.certs(entity2check, "idpsso", use="encryption")

# get certs from idp
mdx.service(entity2check, 'idpsso_descriptor', 'single_sign_on_service')
mdx.certs(entity2check, "idpsso", use="signing")

Authors

Giuseppe De Marco giuseppe.demarco@unical.it

Credits

IdentityPython