Skip to content

Commit

Permalink
pool: throw exception with meaningful error message
Browse files Browse the repository at this point in the history
Motivation:

We have had a report of a stacktrace like:

    09 Apr 2019 15:19:38 (dc111_1) [door:WebDAV-DTEAM-dccore01-pps@webdav-dteam-https-dccore01-ppsDomain:AAWGGM4PiRg RemoteTransferManager PoolDeliverFile 0000D86B06173CE44C73BEDB8378F0B2B082] Please report the following stack-trace to <support@dcache.org>
    java.lang.RuntimeException: CacheException(rc=27;msg=Could not create mover for RemoteHttpsDataTransfer-1.1:https://prometheus.desy.de:2443/VOs/dteam/smoke-test-push-ui01-test.pic.es-20774)
        at org.dcache.pool.classic.MoverRequestScheduler.lambda$getOrCreateMover$0(MoverRequestScheduler.java:207) ~[dcache-core-3.2.47.jar:3.2.47]
        at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660) ~[na:1.8.0_191]
        at org.dcache.pool.classic.MoverRequestScheduler.getOrCreateMover(MoverRequestScheduler.java:202) ~[dcache-core-3.2.47.jar:3.2.47]
        at org.dcache.pool.classic.IoQueueManager.getOrCreateMover(IoQueueManager.java:162) ~[dcache-core-3.2.47.jar:3.2.47]
        at org.dcache.pool.classic.PoolV4.queueIoRequest(PoolV4.java:815) [dcache-core-3.2.47.jar:3.2.47]
        at org.dcache.pool.classic.PoolV4.ioFile(PoolV4.java:822) [dcache-core-3.2.47.jar:3.2.47]
        at org.dcache.pool.classic.PoolV4.messageArrived(PoolV4.java:1083) [dcache-core-3.2.47.jar:3.2.47]
        at sun.reflect.GeneratedMethodAccessor41.invoke(Unknown Source) ~[na:na]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_191]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_191]
        at org.dcache.cells.CellMessageDispatcher$LongReceiver.deliver(CellMessageDispatcher.java:305) [dcache-core-3.2.47.jar:3.2.47]
        at org.dcache.cells.CellMessageDispatcher.call(CellMessageDispatcher.java:202) [dcache-core-3.2.47.jar:3.2.47]
        at org.dcache.cells.AbstractCell.messageArrived(AbstractCell.java:331) [dcache-core-3.2.47.jar:3.2.47]
        at dmg.cells.nucleus.CellAdapter.messageArrived(CellAdapter.java:890) [cells-3.2.47.jar:3.2.47]
        at dmg.cells.nucleus.CellNucleus$DeliverMessageTask.run(CellNucleus.java:1218) [cells-3.2.47.jar:3.2.47]
        at org.dcache.util.BoundedExecutor$Worker.run(BoundedExecutor.java:251) [dcache-common-3.2.47.jar:3.2.47]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_191]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_191]
        at dmg.cells.nucleus.CellNucleus.lambda$wrapLoggingContext$4(CellNucleus.java:754) [cells-3.2.47.jar:3.2.47]
        at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_191]
    Caused by: diskCacheV111.util.CacheException: Could not create mover for RemoteHttpsDataTransfer-1.1:https://prometheus.desy.de:2443/VOs/dteam/smoke-test-push-ui01-test.pic.es-20774
        at org.dcache.pool.classic.AbstractMoverProtocolTransferService.createMover(AbstractMoverProtocolTransferService.java:87) ~[dcache-core-3.2.47.jar:3.2.47]
        at org.dcache.pool.classic.PoolV4.createMover(PoolV4.java:796) [dcache-core-3.2.47.jar:3.2.47]
        at org.dcache.pool.classic.PoolV4.lambda$queueIoRequest$3(PoolV4.java:815) [dcache-core-3.2.47.jar:3.2.47]
        at org.dcache.pool.classic.MoverRequestScheduler.createRequest(MoverRequestScheduler.java:238) ~[dcache-core-3.2.47.jar:3.2.47]
        at org.dcache.pool.classic.MoverRequestScheduler.lambda$getOrCreateMover$0(MoverRequestScheduler.java:205) ~[dcache-core-3.2.47.jar:3.2.47]
        ... 19 common frames omitted
    Caused by: java.lang.IllegalArgumentException: Parameter 'directory' is not a directory
        at org.apache.commons.io.FileUtils.validateListFilesParameters(FileUtils.java:545) ~[commons-io-2.4.jar:2.4]
        at org.apache.commons.io.FileUtils.listFiles(FileUtils.java:521) ~[commons-io-2.4.jar:2.4]
        at eu.emi.security.authn.x509.helpers.pkipath.PlainStoreUtils.establishWildcardLocations(PlainStoreUtils.java:92) ~[canl-2.1.2.jar:2.1.2]
        at eu.emi.security.authn.x509.helpers.pkipath.PlainStoreUtils.establishWildcardsLocations(PlainStoreUtils.java:112) ~[canl-2.1.2.jar:2.1.2]
        at eu.emi.security.authn.x509.helpers.trust.DirectoryTrustAnchorStore.update(DirectoryTrustAnchorStore.java:189) ~[canl-2.1.2.jar:2.1.2]
        at eu.emi.security.authn.x509.helpers.trust.OpensslTrustAnchorStoreImpl.<init>(OpensslTrustAnchorStoreImpl.java:61) ~[canl-2.1.2.jar:2.1.2]
        at eu.emi.security.authn.x509.impl.OpensslCertChainValidator.<init>(OpensslCertChainValidator.java:113) ~[canl-2.1.2.jar:2.1.2]
        at org.dcache.pool.classic.RemoteHttpTransferService.getValidator(RemoteHttpTransferService.java:153) ~[dcache-core-3.2.47.jar:3.2.47]
        at org.dcache.pool.classic.RemoteHttpTransferService.createMoverProtocol(RemoteHttpTransferService.java:126) ~[dcache-core-3.2.47.jar:3.2.47]
        at org.dcache.pool.classic.AbstractMoverProtocolTransferService.createMover(AbstractMoverProtocolTransferService.java:79) ~[dcache-core-3.2.47.jar:3.2.47]
        ... 23 common frames omitted

The cause was the lack of deployed IGTF trust store; therefore the
directory /etc/grid-security/certificates did not exist.

There are two problems here.  First, a RuntimeException is thrown, yet
this is not a bug.  Second, the error message ("Parameter 'directory' is
not a directory") provides almost no indication what is the problem.

Modification:

Add our own test to check the directory exist and is reasonable.

Result:

An unhelpful error message "Parameter 'directory' is not a directory" is
replaced with one that provides information on which directory is
missing.

Target: master
Request: 5.1
Request: 5.0
Request: 4.2
Request: 4.1
Request: 4.0
Request: 3.2
Requires-notes: yes
Requires-book: no
Patch: https://rb.dcache.org/r/11685/
Acked-by: Tigran Mkrtchyan
  • Loading branch information
paulmillar committed Apr 11, 2019
1 parent 668b305 commit 354f955
Showing 1 changed file with 5 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@
import eu.emi.security.authn.x509.impl.ValidatorParams;
import org.springframework.beans.factory.annotation.Required;

import java.io.IOException;
import java.security.SecureRandom;
import java.util.concurrent.TimeUnit;

Expand All @@ -40,6 +41,8 @@
import org.dcache.pool.movers.RemoteHttpDataTransferProtocol;
import org.dcache.pool.movers.RemoteHttpsDataTransferProtocol;

import static org.dcache.util.Files.checkDirectory;

public class RemoteHttpTransferService extends AbstractMoverProtocolTransferService
{
private String caPath;
Expand Down Expand Up @@ -143,9 +146,10 @@ public void shutdown()
}
}

private synchronized X509CertChainValidator getValidator()
private synchronized X509CertChainValidator getValidator() throws IOException
{
if (validator == null) {
checkDirectory(caPath);
OCSPParametes ocspParameters = new OCSPParametes(ocspCheckingMode);
ValidatorParams validatorParams =
new ValidatorParams(new RevocationParameters(crlCheckingMode, ocspParameters), ProxySupport.ALLOW);
Expand Down

0 comments on commit 354f955

Please sign in to comment.