Dear dCache devs,
We have a project with genetics data. To prevent this data from being copied elsewhere, we have set up net units like psu create unit -net 145.100.32.40/32 to create a network based whitelist in the PoolManager. So, only clients from allowed addresses, such as the uploader's host and a fully locked down compute cluster, have access to the data. This works very well for reading and writing. But staging through the API is a problem. This is the result of a bulk stage request:
[root@dcache-aio git]# /root/git/SpiderScripts/ada/ada --stat-request 34972583-9228-441e-8f18-1660b0f033ce
{
"nextId" : -1,
"uid" : "34972583-9228-441e-8f18-1660b0f033ce",
"arrivedAt" : 1776689039163,
"startedAt" : 1776689039952,
"lastModified" : 1776689039970,
"status" : "COMPLETED",
"targetPrefix" : "/",
"targets" : [ {
"target" : "/pnfs/grid.sara.nl/data/agh/tape/testfile",
"state" : "FAILED",
"submittedAt" : 1776689039165,
"startedAt" : 1776689039165,
"finishedAt" : 1776689039966,
"errorType" : "java.lang.Throwable",
"errorMessage" : "No read links configured for [net=0:0:0:0:0:0:0:1,protocol=Http/1,store=agh:rwtape@osm,cache=,linkgroup=]",
"id" : 33930844
} ]
}
Note the net=0:0:0:0:0:0:0:1. It looks like the client IP address is not propagated from API to the Bulk service, or from the Bulk service to the PinManager. Instead, the IPv6 equivalent of "localhost" is passed on. But since it is not on the whitelist, it is refused.
As a workaround I have added 0:0:0:0:0:0:0:1 (or ::1/128 to be exact) to the whitelist. Reading and writing through WebDAV is still protected, but staging is allowed, but from any address, which is not ideal.
I would expect that the client address would be propagated to PinManager and PoolManager, so the PoolManager can fully do its pool selection magic, also for bulk staging. Should it indeed work like this, or are there reasons why the API client adress is not propagated?
Kind regards,
Onno
Dear dCache devs,
We have a project with genetics data. To prevent this data from being copied elsewhere, we have set up net units like
psu create unit -net 145.100.32.40/32to create a network based whitelist in the PoolManager. So, only clients from allowed addresses, such as the uploader's host and a fully locked down compute cluster, have access to the data. This works very well for reading and writing. But staging through the API is a problem. This is the result of a bulk stage request:Note the
net=0:0:0:0:0:0:0:1. It looks like the client IP address is not propagated from API to the Bulk service, or from the Bulk service to the PinManager. Instead, the IPv6 equivalent of "localhost" is passed on. But since it is not on the whitelist, it is refused.As a workaround I have added
0:0:0:0:0:0:0:1(or::1/128to be exact) to the whitelist. Reading and writing through WebDAV is still protected, but staging is allowed, but from any address, which is not ideal.I would expect that the client address would be propagated to PinManager and PoolManager, so the PoolManager can fully do its pool selection magic, also for bulk staging. Should it indeed work like this, or are there reasons why the API client adress is not propagated?
Kind regards,
Onno