Skip to content

Commit

Permalink
docs: stage protection
Browse files Browse the repository at this point in the history 
      Issue: #4477
  • Loading branch information
@DmitryLitvintsev
DmitryLitvintsev committed Dec 5, 2018
1 parent 0375c56 commit a02c60a
Show file tree
Hide file tree
Showing 2 changed files with 132 additions and 60 deletions.
184 changes: 128 additions & 56 deletions docs/TheBook/src/main/markdown/config-stage-protection.md
View file
Expand Up @@ -7,83 +7,155 @@ Table of Contents
+ [Configuration of Stage Protection](#configuration-of-stage-protection)
+ [Definition of the White List](#definition-of-the-white-list)


A dCache system administrator may specify a list of DNs/FQANs which are allowed to trigger tape restores for files not being available on disk. Users, requesting tape-only files, and not being on that *white list, will* receive a permission error and no tape operation is launched. Stage protection can be enhanced to allow authorization specific to a dCache storage group. The additional configuration parameter is optional allowing the stage protection to be backwards compatible when stage authorization is not specific to a storage group.
Access to tape is expensive. To avoid inefficient use of tape resources, or "stage mayhem" by random, chaotic user
activity a mechanism exists in dCahe called "stage protection" that allows to control access to data on tape based
on user identity (DN), VO group membership and VO role (defined in FQAN),
storage group and protocol. Attempts to stage data that does not satisfy criteria of
stage permission configuration will result in permission denied errors

CONFIGURATION OF STAGE PROTECTION
=================================

Stage protection can optionally be configured in the poolmanager rather than on the doors and the pinmanager. Thus the white list needs to be present on a single node only. To enable this, define the following parameter in **/etc/dcache/dcache.conf**:


dcache.authz.staging.pep=PoolManager

The file name of the white list must be configured by setting the **dcache.authz.staging** parameter in **/etc/dcache/dcache.conf**:

dcache.authz.staging=PATH-ODC-ED/StageConfiguration.conf

The parameter needs to be defined on all nodes which enforce the stage protection, i.e., either on the doors and the `pinmanager`, or in the `poolmanager` depending on the stage policy enforcement point.

DEFINITION OF THE WHITE LIST
============================

The Stage Configuration File will contain a white list. Each line of the white list may contain up to three regular expressions enclosed in double quotes. The regular expressions match the DN, FQAN, and the Storage Group written in the following format:

"<DN>" ["<FQAN>" ["<StorageGroup>"] ]

Lines starting with a hash symbol `#` are discarded as comments.

The regular expression syntax follows the syntax defined for the [Java Pattern class](http://docs.oracle.com/javase/6/docs/api/java/util/regex/Pattern.html).

Example:
Here are some examples of the White List Records:

".*" "/atlas/Role=production"
"/C=DE/O=DESY/CN=Kermit the frog"
"/C=DE/O=DESY/CN=Beaker" "/desy"
"/O=GermanGrid/.*" "/desy/Role=.*"
The stage protection rules are captured in stage configuration file which
is pointed to by the variable :
```
dcache.authz.staging = <path>/StageConfiguration.conf
```
The stage protection policy enforcement point (PEP) can be
`PoolManager` or `doors`. Configurable using the following variable:
```
dcache.authz.staging.pep
```
If set to `PoolManager` like so:
```
dcache.authz.staging.pep = PoolManager
```
then the stage protection applies to all transfers in the system and stage protection
configuration file naturally has to be available on the host running `PoolManager`, if
set for `doors` like:
```
dcache.authz.staging.pep = doors
```
then the stage protection applies only to transfers performed by doors on hosts that have
`dcache.authz.staging` defined and the file present. The host running `PinManager` also has to have
`dcache.authz.staging` defined and the file present.

The default settings are `dcache.authz.staging = ` (not set) and
`dcache.authz.staging.pep = doors`.

DEFINING STAGE PROTECTION
=========================

Stage protection can be setup as a white or a black list. Blacklisting is achieved by
using `!` in front of an expression.

Each line of the list may contain up to four regular expressions
enclosed in double quotes. The regular expressions match the DN, FQAN,
the Storage Group and protocol specified in the following format:

"<DN>" ["<FQAN>" ["<StorageGroup>" ["<protocol>"]] ]

Lines starting with a hash symbol `#` are discarded as comments. The regular expression syntax follows the syntax defined for
the [Java Pattern class](http://docs.oracle.com/javase/6/docs/api/java/util/regex/Pattern.html).


Some examples of the White List Records:
```
".*" "/atlas/Role=production"
"/C=DE/O=DESY/CN=Kermit the frog"
"/C=DE/O=DESY/CN=Beaker" "/desy"
"/O=GermanGrid/.*" "/desy/Role=.*"
```

This example authorizes a number of different groups of users:

- Any user with the FQAN tlas/Role=production.
- The user with the DN /C=DE/O=DESY/CN=Kermit the frog, irrespective of which VOMS groups he belongs to.
- The user with the DN /C=DE/O=DESY/CN=Beaker but only if he is also identified as a member of VO desy (FQAN /desy)
- Any user with DN and FQAN that match /O=GermanGrid/.\* and /desy/Role=.\* respectively.
- Any user with the FQAN `atlas/Role=production`.
- The user with the DN `/C=DE/O=DESY/CN=Kermit the frog`, and any VOMS groups he belongs to.
- The user with the DN `/C=DE/O=DESY/CN=Beaker` but only if he is also identified as a member of VO desy (FQAN `/desy`)
- Any user with DN and FQAN that match `/O=GermanGrid/.\*` and `/desy/Role=.\*` respectively.

If a storage group is specified all three parameters must be provided. The regular expression `".*"` may be used to authorize any DN or any FQAN. Consider the following example:
If a storage group is specified, three parameters must be provided. The regular expression `".*"` may be used to authorize any DN or any FQAN. Consider the following example:

Example:
```
".*" "/atlas/Role=production" "h1:raw@osm"
"/C=DE/O=DESY/CN=Scooter" ".*" "sql:chimera@osm"
```

In the example above:

- Any user with
FQAN
/atlas/Role=production
`/atlas/Role=production`
is allowed to stage files located in the storage group
h1:raw@osm
.
`h1:raw@osm`.
- The user
/C=DE/O=DESY/CN=Scooter
, irrespective of which VOMS groups he belongs to, is allowed to stage files located in the storage group
sql:chimera@osm
.

With the plain `dCap` protocol the DN and FQAN are not known for any users.

Example:
In order to allow all `dCap` users to stage files the white list should contain the following record:

"" ""
`/C=DE/O=DESY/CN=Scooter`, irrespective of which VOMS groups he belongs to, is allowed to stage files located in the storage group
`sql:chimera@osm`.

In case this line is commented or not present in the white list, all DCAP users will be disallowed to stage files.
If protocol is specified all four parameters must be provided.

It is possible to allow all `dCap` users to stage files located in a certain storage group.
Example:

Example:
In this example, all `dCap` users are allowed to stage files located in the storage group `h1:raw@osm`:
```
".*" "/atlas/Role=production" "h1:raw@osm" "Htt.*"
"/C=DE/O=DESY/CN=Scooter" ".*" "sql:chimera@osm" "GFtp.*"
```

"" "" "h1:raw@osm"
In the example above:

[Java Pattern class]: http://java.sun.com/javase/6/docs/api/java/util/regex/Pattern.html
- Any user with
FQAN
`/atlas/Role=production`
is allowed to stage files located in the storage group
`h1:raw@osm` using HTTP protocol.
- The user
`/C=DE/O=DESY/CN=Scooter`, any VOMS groups he belongs to,
is allowed to stage files located in the storage group
`sql:chimera@osm` using GFTP protocol.


Exact protocol names are `DCap-3.0`, `GFtp-1.0`, `GFtp-2.0`, `Http-1.1`, `NFS4-4.1` and ` Xrootd-2.7`.
The version suffix is subject to change therefore `.*` comes in handy.

Non-authenticated protocols like plain `dCap` protocol or `NFS` protocol (if so setup)
carry null for DN and FQAN. A `"""` expression will match nulls.

In order to allow all users using `dCap` protocol to stage data for any storage
group the list configuration would look like:
```
"" "" ".*" "dCap.*"
```
NB: Once stage protection configuration exists, the PEP will process it for match and if
no match found staging will be denied. Therefore an empty stage configuration file will
effectively deny staging for all.

As was mentioned above, black lists can be formed my adding `!<expr>` in front of matching
regular expression translating into "staging is allowed if not matching `<expr>`".

In this example:
```
".+"
"" "" "!nova.*"
"" "" "nova.*" "!NFS4.*"
```
- Any authenticated user (non empty DN) can stage any files using any protocol (not a black list per se but is used here from a real life setup).
- All non-authenticated users can stage files not belonging to storage groups matching
`nova*` using any protocol.
- All non-authenticated users cannot stage files belonging to storage groups matching
`nova*` using `NFS` protocol.

NB: A root user is special. All authorization checks are by-passed for root user. Therefore
in the example above as root user still will be able to stage `nova` data using `NFS` protocol.

The `!` notation is a convenience feature, the same setup can be expressed in using proper
Java regular expression for negation:

```
".+"
"" "" "^(?:(?!nova).)*$"
"" "" "nova.*" "^(?:(?!NFS4).)*$"
```

The stage protection configuration file can be edited on the running system at anytime and
the policies will take effect once file is saved to disk.
8 changes: 4 additions & 4 deletions docs/TheBook/src/main/markdown/intro.md
View file
Expand Up @@ -77,14 +77,14 @@ Components of dCache System
| [poolmanager](config-PoolManager.md) | When a file reading or writing a transfer request is sent to the dCache system, the poolmanager then decides how to handle this request. |
| [resilience](config-resilience.md) |Controls the number of replicas of a file on the pools |
| [hoppingmanager](config-hopping.md) | ? |
| cleaner | ? |
| cleaner | Service that orchestrates file replica distribution across the pools triggered by variety of conditions. |
| info | ? |
| missing-files | + |
| [missing-files](config-missing-files.md) | A component designed to react to requests to retrieve missing files. |
|[pool](cookbook-pool.md) | Data storage ? (cookbook) |
| [pnfsmanager](config-PnfsManager.md) | Managing the pnfs file system (hierarchy), pnfs database, meta-data |
| spacemanager | + |
| [srm](config-SRM.md) | Provides dynamic space allocation and file management on shared storage components on the Grid |
| srmmanager | ? |
| [srm](config-SRM.md) | Front-end to Storage Resource Manager (SRM). Client entry point ot SRM sub-system (see below) |
| [srmmanager](config-SRM.md) | SRM Back-end. Provides dynamic space allocation and file management on shared storage components on the Grid |
| [zookeeper](config-zookeeper.md) | A distributed directory and coordination service on which dCache relies on|
| [gplazma](config-gplazma.md) | Authentication and authorization interface to limit access to data |
| [dCap](cookbook-dCap.md) | Supports all necessary file metadata and name space manipulation operations |
Expand Down

0 comments on commit a02c60a

Please sign in to comment.