pycsw example

[florianm]

Is there a working example for these two use cases:

• expose datacats CKAN as CSW using pycsw
• expose a remote WMS as CSW using pycsw

Could a pycsw server be run as a container like celery?

Assuming I've installed pycsw like any other extension by cloning it into the datacats environment, and have installed the requirements:

sudo apt-get install -y python-dev libxml2-dev libxslt-dev libgeos-dev
pip install sqlalchemy

The vanilla way python pycsw/bin/pycsw-admin.py -c post_xml -u http://localhost:PORT/pycsw/csw.py -x pycsw/post.xml indicates that binary pycsw/bin/pycsw-admin.py and config pycsw/post.xml live inside pycsw in my datacats environment. What would be recommended settings for:

• database - sqlite or postgres (do I need to use the credentials of the postgres container or spin up another db container)
• log file location
• pycsw server home

Is there a "datacats" way or should I create supervisor jobs running my pycsw servers?

[JackMc]

You can submit a pull request adding libgeos-dev to the Dockerfile's list of installs, and then running it with datacats shell.

[deniszgonjanin]

This example is interesting because pycsw is a standalone piece of software, with it's own processes, database connections, etc... The approach above might work fine, but now we're trying to shoehorn another application into the datacats way of doing things, which is sometimes very CKAN-centric.

This is something we have talked about, but we haven't reached any agreement on what the best strategy is regarding supporting this. Anything except supporting CKAN only leads us down a path of trying to build a general-purpose web application devops tool. Which would be really cool, but sounds like a lot of work.

A PR for libgeos-dev to the Dockerfile is probably a good idea anyway.

[florianm]

Sorry, I wasn't fully aware that pycsw is definitely a distinct package.
I've managed to harvest the pycsw example WMS service into a dockerised pycsw server from GeographicaGS.

I had to modify the pycsw config file to allow transactions from all IPs (still working on some firewall/procy issues). The write-up is here.

Based on the pycsw Dockerfile, could datacats provide a similar image, only with transactions enabled in the config, and logging enabled with a mapped out log dir?

A separate image for pycsw along the lines of your CKAN images would mean we could run one pycsw container per harvested source, such as separate WMS endpoints.

Would the benefits of having a full WMS > CSW > CKAN harvesting setup under your complete control outweigh the costs of maintaining another image?

edit: PR for clarity.

[florianm]

nearly working...
after I ran datacats tweak --add-pycsw pycsw got registered the datacats/pycsw container in envmane/.datacats-environment. So far so good!

datacats reload
...
APIError: 500 Server Error: Internal Server Error ("Cannot start container a6d77ebc871dfcbf836aa97e632c196290f90d7ea6b98bc17ef4ba9203282b14: Cannot link to a non running container: /datacats_pycsw_landgate_primary AS /datacats_web_landgate_primary/pycsw")

@JackMc and co: Where is the name /datacats_web_landgate_primary/pycsw constructed?

FYI currently the working example is:

cd datacats/docker/pycsw
docker build -t datacats/pycsw .
docker run -d -p 9000:8000 -it datacats/pycsw python /var/www/pycsw/csw.wsgi