Minutes 2011 10 24..25

Minutes 2011-10-24..25 SARA-GWDG

Participants:

• SARA
  • Pieter van Beek
  • Markus van Dijk
  • Evert Lammerts
• GWDG
  • Tibor Kalman

Part I (Day 1)

Introduction, motivation

PvB: Setting of the project. CATCHPlus, collaboration with GWDG.

TK: New director at GWDG since October. He came from Dortmund, took his IT research department with him. Plans are not known yet.
GWDG: Currently more focus is on long term data preservation. In DARIAH-DE, GWDG would have started with PID stuff, then LTA, but currently first LTA, then PID's. (Among others the Göttingen University has to show how they do LTA at the beginning of next year.) The current LTA infrastructure at the GWDG is not enough for that.
In short, it's not sure at the moment how GWDG will continue with PID.

TK: Officially, Tibor has time to spend on version 2 of the API. Most important users are:

• SUB (Staats und UniBibliotek Göttingen). They use GRID storage. They scan pages and give them PIDs. At the moment, they also use textgrid IDs (some internal product), but the global uniqueness is not guaranteed and there's no global resolver. Internally they will keep using textgrid ids, externally they'll start using PIDs.
• Some CLARIN partners
  • MPI-PL, etc
• Some GWDG partners
  • MPDL, etc

TK: There's an existing v1.
Further development plans:

• Original plan: contribute on version 2 (with SARA).
• Secondary (fallback) plan: just add a few necessary features to v1 alone.

TK: There's money from DARIAH-DE and CLARIN-D and some more small parties. DARIAH (the largest party) wants to have new features. Idea of GWDG synergy: GWDG wanted to join all requirements and money, and use it to work.

TK: Policy for v1 software: Code is provided, but recipients are requested not to distribute it any further.

Required outcomes of this meeting

• All the information needed by GWDG to decide if we can collaborate or not.
• Collaboration environment.
• Milestones and deliverables

Agenda

• Discuss COBII.
• Environment at SARA.
  • handle system, versions etc.
  • web service (PHP)
  • organisation
• Environment at GWDG.
  • handle system, versions etc.
  • web service API
  • code?
  • Tibor's experiences with the handle system
• Discuss the features that need to be implemented in V2 for CLARIN-D and DARIAH-D.
• V2 document
• Create a roadmap
• Strategy
  • collaboration with other institutes?
  • openness to other developers?

Hands on:

• Source code of v1. MD: software age, dependencies? TK: It's about 2-3 years old.

Tasks

• TK: Check mirroring, versions of HS at GWDG and SARA.
• PvB: Arrange diner.
• PvB: Arrange car tuesday

PID @ GWDG

Organizational: PID people @ GWDG:

Ulrich Schwardmann: Vice group leader. Strategical planning.
Kai Germershausen: is the sysadmin for the Handle System @GWDG
Tibor:

• Primary (official) task: DARIAH Workpackage involving scientific data management.
• Secondary: other task in DARIAH, involving PIDs.

Technical: PID machines @ GWDG:

4 virtual machines:

• Mirror of MPI-PL (v7)
• Global Handle Registry
• Production
  • Handle Server
  • PostgreSQL
  • Tomcat
  • PID service
• hdl-test
  • Handle Server
  • PostgreSQL
  • Tomcat
  • PID service
  • runs a test Handle Server, with the GWDG prefix, but not globally resolvable and not publicly available (auth)

Currently everything runs on SuSE Linux Enterprise Server (SLES) 10.x, The Tomcat install is old (5.x). There are problems with the JAVA version on SuSE 10.x.

Service

PID service root: http://handle.gwdg.de:8080/pidservice/
Policy: checksum cannot be modified (you should create a new PID).
Currently one prefix is used for all the communities.

Future plans

• Move to High Availability MySQL service.
• Separate DB, HS, and PID services v1 and v2
• PID-Service upgrade: for all community (just 1 service runs)
• SLES upgrade to 11.x

PID @ SARA

Organizational: PID people @ SARA:

CATCHPlus: Hennie Brugman
SARA: Pieter van Beek, Markus van Dijk

Technical: PID machines @ SARA:

everything runs on 1 server (with the name: ws2.grid.sara.nl)

ws2.grid.sara.nl

• handle
  • handle v6.2.5_02
  • 3 instances (2 primary and 1 is the gwdg-mirror)
  • each listens on an own ports
  • idea(?): 1 server process for all primaries and 1 for the secondaries (mirrors)
  • DB for handle: MySQL
    • 1 MySQL server with 3 DBs (for the prefixes)
• web service on the top of this

Service

Service URL: https://ws2.grid.sara.nl/catchplus/
docu referenced on the webpage: https://docs.google.com/Doc?docid=0AV1_-ddi0ofuZGNtcnZueDNfNjdyODU2NXFjeg&hl=en

• PHP
• (!)REST client in Chrome
• response: text/plain, XHTML, JSON
• POST: template creation is possible
• search by regexp is possible (Accept: text/plain for a text-based list)
• DB statements are created directly by the PHP webapp
• handle system is in read-only mode
• prefixes: one for every customer
• update strategy: it is possible to upgrade per customer

Flipovers

• Handle System Architecture @ GWDG

Part II (Day 2)

Morning

• Atomic Batch Operations (ABOs):
  Discussion about replication and atomic batch operations. If modifications are done behind the back of the HS server, no replication takes place, unless you manually update the transaction log. Atomic batch operations cannot be done without collaboration of the HS server.
• Hacking/Security:
  Markus would like to present his first ideas on how to deploy the service in a secure way.
• LDAP:
  GWDG uses LDAP as authentication source. The PID service has a special service account to authenticate the users. PB imagined that the HS itself would be used to store AAA information, but MD & TK think it's not a good idea to enforce SPs into a certain method of AAA. TK doesn't see the advantage of putting all AAA information into the HS. Maybe we could have a pluggable authentication API?

Agenda

• TK will show source code from v1.
• MoSCoW tasks:
  • What have we done this month.
  • What will we do next month.
• 16:00 Meeting @Meertens.

Markus' planning ==

• Have a handle service running in a play garden.
• Have a JAVA web service perform read operations on the HS and/or database.
• Devise a security plan for the system in production.

Risk Log

• Performance issue at GWDG with write operations through the HS server. Creating a PID takes time in the order of seconds.

Flipovers

• Requirements
• Roadmap

Part III (CatchPlus meeting)

Introduction

GWDG, EPIC, CatchPlus

MoSCoW tasks

Re-order of priorities to be compatible with the GWDG-Roadmap/DARIAH-Milestones