Skip to content

Questions & Answers

naare edited this page Aug 16, 2019 · 48 revisions

Questions & Answers

Unable to load the LOTL: content is empty error

EU is in process of changing the location of member states trust providers List of Trusted Lists (LOTL). This list is essential for creating and validating signatures with Digidoc4j. The reason for this error may be that your current Digidoc4j integration uses the old location (https://ec.europa.eu/information_society/policy/esignature/trusted-list/tl-mp.xml) for the before mentioned LOTL and it is not available anymore. To solve the problem Digidoc4j needs to be pointed to the correct location: https://ec.europa.eu/tools/lotl/eu-lotl.xml.

There are different options to set the new location:

  • Set the new location through digidoc4j.yaml file. The general instructions can be found here.

Add TSL_LOCATION: "https://ec.europa.eu/tools/lotl/eu-lotl.xml" to the digidoc4j.yaml to override the LOTL location.

  • Set the new location programmatically through Configuration object. The general instructions can be found here.

Add configuration.setTslLocation("https://ec.europa.eu/tools/lotl/eu-lotl.xml"); to your code.

Integrating with Web browsers

See a working example: DigiDoc4j with hwcrypto.js example demo Web application

It is possible to use DigiDoc4j when signing documents using a Web Browser (Chrome, Firefox, IE, Safari). A JavaScript framework called hwcrypto.js can be used to integrate digital signing functionality on a Web page. Take a look at hwcrypto.js API description. Note that the old idCard.js cannot be used anymore by Chrome and the new hwcrypto.js is asynchronous demanding a dynamic page with Ajax. Note also that hwcrypto.js works properly only over HTTPS.

In the external signing example, the getSignerCertSomewhere method should return the certificate fetched by hwcrypto.js getCertificate call. This returns a certificate object. This is a hex string that can be turned into a byte array by using DatatypeConverter.parseHexBinary(certificateInHex); for example. You could then rename this method to getSignerCertFromBrowser instead.

In the same external signing example, the signDigestSomewhereRemotely method should call the hwcrypto.js sign method that returns a signature object (in hex) that must be turned into a byte array. This can also be done by using DatatypeConverter.parseHexBinary(signatureInHex); for example. You could then rename the signDigestSomewhereRemotely method to signDigestInBrowser instead.

The Web signing front-end (JavaScript) functionality can be tested on the hwcrypto.js demo page. Take a look at an example of signing with JavaScript.

If OCSP request has failed

If you get an error "Signature does not contain OCSP response" (OCSPRequestFailedException), then most likely your OCSP requests are not configured properly.

  • If you are using DigiDoc4j in production mode (configuration is in PROD mode), then either
  • You must have an IP based access to the OCSP service (http://ocsp.sk.ee/)
  • Or your OCSP requests must be signed by an OCSP Access Certificate (juurdepääsutõend)
  • The Configuration class has the following methods for configuring OCSP: setOCSPAccessCertificateFileName, setOCSPAccessCertificatePassword, setSignOCSPRequests, setOcspSource
  • digidoc4j.yaml can also be configured with the following parameters: DIGIDOC_PKCS12_CONTAINER, DIGIDOC_PKCS12_PASSWD and SIGN_OCSP_REQUESTS.
  • It is not necessary to sign OCSP requests in the test mode (if Configuration is in TEST mode)
  • If you are using test mode, then it is not possible to sign with production ID cards. Only test ID cards can be used in test mode.
  • You have to upload your test ID card signing certificate to the test OCSP service at https://demo.sk.ee/upload_cert/. This would allow you to sign with your test ID card and the OCSP service would return a valid response.
  • The test TSL does not contain production certificates so it is not possible to create a valid signature with production ID card in test mode

How to test with ID card?

  • If you want to test with your production ID card then you have to test in the production (PROD) mode.
  • If you want to test in the TEST mode, then you have to use test ID cards.
  • It is not possible to test in the test mode using production ID cards.
  • The test TSL does not contain production certificates so it is not possible to create a signature with production ID card in the test mode.
  • Here's an example of accessing smart card on your device to sign with a physical ID card.

Why is the library that slow?

DigiDoc4j is slow only in the first run, when it downloads the EU TSL (EU Trusted Lists of Certificates). TSL is refreshed once a day (by default) and it takes time to download (between 5-15 seconds).

  • Only the first operation is slow when the TSL is being loaded.
  • Additional operations are faster when the TSL is already in memory.
  • TSL is loaded lazily - only when necessary. Looking at container and signature details does not trigger TSL download.
  • It is possible to load TSL separately (e.g. in application startup) by calling configuration.getTSL().refresh();. This triggers TSL download and later operations (validations, signature creations) would not need to download TSL.
  • TSL is downloaded once a day by default. It takes about 5-15 seconds to load.
  • Make sure to use only one instance of the Configuration object. TSL is stored within the Configuration object memory.
  • See the performance tests results of the library's operation times.

How to add trust for new EU TSL signing certificates?

Trusted EU TSL signing certificates are used for validating the signature of the root TSL published by the European Commission.

DigiDoc4j library stores the trusted TSL signing certificates in a KeyStore, e.g. keystore.jks.

In order to update the TSL keystore, do as follows:

  • Copy the trusted EU TSL signing certificates from EU Trusted List of Trust Service Providers. The latest trusted EU TSL signing certificate is published here between Signature->KeyInfo->X509Data->X509Certificate tag. Certificate
  • Use KeystoreGenerator.java to generate new KeyStore with the new certificates.
  • Use the newly generated KeyStore with DigiDoc4j. The default KeyStore location keystore/keystore.jks and password can be modified:

Possible errors in log when LOTL is signed with new certificate digidoc4j is not trusting yet: The certificate used to sign this token is not found or not valid! INDETERMINATE NO_CERTIFICATE_CHAIN_FOUND The certificate chain for signature is not trusted, there is no trusted anchor.

How to sign with Estonian Mobile ID?

It is possible to sign BDOC and ASIC-E containers with Mobile ID DigiDoc Service by using two step external signing process.

  • Here's an example of doing two step external signing
  • Here's an example project of using Mobile ID DigiDoc Service
  • Signer's certificate can be retrieved by using DigiDocService GetMobileCertificate request
  • Digest can be signed using MobileIdService MobileSignHash and GetMobileSignHashStatusRequest requests
  • DigiDocService can be used with the following steps:
  1. Get the Mobile ID user's signer certificate (GetMobileCertificate request in DigiDocService)
  2. Create a container to be signed (using ContainerBuilder)
  3. Calculate digest to be signed (dataToSign.getDigestToSign())
  4. Sign the digest with Mobile ID (using MobileSignHash and GetMobileSignHashStatusRequest requests in MobileIdService)
  5. Finalize the signature (dataToSign.finalize(signatureValue))
  6. Add the signature to the container (container.addSignature(signature))
  • See the full DigiDocService specification, DigiDocService WSDL and MobileIdService WSDL
  • You need to request separate permissions from SK (Sertifitseerimiskeskus) to access the GetMobileCertificate MobileSignHash and GetMobileSignHashStatusRequest services. These two calls need special permissions in addition to the rest of the Mobile ID access.

Using DDoc containers in multi-threaded environment

Since DigiDoc4j uses jDigidoc library for handling DDoc containers, then it has the same problems that using jDigidoc has in multi-threaded environment. DigiDoc4j has solved most of those issues internally, but it is highly recommended to use a different temporary directory for each DDoc container, especially when opening existing containers.

  • Use ContainerBuilder.usingTempDirectory(String path) to specify a temporary directory for opening existing DDoc containers. The path should be unique for each container.
  • It is possible to initialize jDigidoc's ConfigManager manually (if you really need to) by calling ConfigManagerInitializer.forceInitConfigManager. Note that this is not a thread-safe operation, i.e. if multiple threads are handling DDoc containers at the same time, then calling this operation can cause weird side-effects. Use it wisely. Also note that it is not necessary to call this operation in normal circumstances, because DigiDoc4j already handles it by calling it only once and during initialization of the first DDoc container.

Does it really download all the TSL files every time?

  • Quick answer: No
  • Slightly longer answer: it uses caching so all the TSL files are downloaded only once and refreshed when necessary (when they expire).
  • If you see log messages like Downloading TrustStatusList for 'AT' from url='https://www.signatur.rtr.at/currenttl.xml' then it doesn't mean that the files are actually being downloaded. They could be taken from the cache.
  • The reason why the log messages are misleading is that DigiDoc4j uses underlying DSS (Digital Document Service) library and that's how this information is logged there.
  • If you are interested in seeing if cached files are used or not, then this information is logged with the Debug level in the DSS library under eu.europa.ec.markt.dss.validation102853.https.FileCacheDataLoader class. It is possible to enable logs by adding the following line <logger name="eu.europa.ec.markt.dss.validation102853.https.FileCacheDataLoader" level="DEBUG"/> to the logback.xml file.
  • It is possible to switch off those confusing download logs by adding those lines to the logback.xml file:
<logger name="eu.europa.ec.markt.dss.validation102853.tsl.TrustedListsCertificateSource" level="WARN"/>
<logger name="eu.europa.ec.markt.dss.validation102853.SignedDocumentValidator" level="WARN"/>
<logger name="eu.europa.ec.markt.dss.signature.validation.xades.XAdESCertificateSource" level="WARN"/>

How to clear TSL cache?

  • If you discover that the trusted certificates (like root CA, Timestamp etc) are not updated, then it might help to clear TSL cache.
  • TSL is cached in two locations in the temp directory set with the java.io.tmpdir property:
  • java.io.tmpdir/dss-cache-tsl
  • java.io.tmpdir/digidoc4jTSLCache
  • Depending on your operating system, the default java.io.tmpdir property may point to
  • /tmp on Linux (/tmp/dss-cache-tsl and /tmp/digidoc4jTSLCache)
  • C:\Users\_admin_\AppData\Local\Temp\ i.e. %TEMP% on Windows (%TEMP%\dss-cache-tsl and %TEMP%\digidoc4jTSLCache)
  • Somewhere obscure like /var/folders/_4/8979h_s11kvbylb3d5p_fydm0000gn/T/ on OSX.
  • Just delete all the files stored in the two directories

Getting an error while trying to test

  • If you get an error like this java.io.FileNotFoundException: test-tsl/trusted-test-mp.sha2 (or trusted-test-mp.xml)
  • Then take a look at this
  • We do not include test certificates in DigiDoc4j jar file so it must be configured separately.
  • Great Harm may happen if someone uses test certificates in production.

Using a YAML file for configuration

  • It is possible to configure DigiDoc4j using digidoc4j.yaml file (in addition to setting parameters in the Configuration class)
  • If you are using only BDoc/ASIC containers (creation and validation), then there are no mandatory parameters
  • If you are creating and validating old legacy DDoc containers, then the certificates must be specified. Use the default digidoc4.yaml file which contains the minimum required configuration for opening DDoc containers. Note that the default configuration contains both production and test certificates.
  • This digidoc4j.yaml example contains all the possible configuration parameters.

How to configure logging?

  • Following artifacts are used by Digidoc4J library for producing a log:
    • The Simple Logging Facade for Java (SLF4J)
    • logback-classic as the underlying logging framework
  • The configuration file logback.xml must be defined; read more about configuring logback from here.
  • In order to drive logging of validation results, the configuration context is supplemented with a configuration parameter called "PrintValidationReport". This flag controls whether to print validation report result into dedicated log stream. By default this is inactive for "PROD" mode and active for "TEST" mode. Use configuration.setPrintValidationReport method or "PRINT_VALIDATION_REPORT" parameter in "yaml" file to change this behaviour.

Which file extension to use?

  • If you are creating BDoc-TS (new TimeStamp i.e. ASIC-E) containers (with SignatureProfile.LT), then use .asice file extension.
  • If you are creating BDoc-TM (old TimeMark) containers (with SignatureProfile.LT_TM), then use .bdoc file extension.

Is BDoc-TS (ASIC-E) format compatible with the old jDigidoc library?

  • No, jDigidoc is not capable of opening BDoc-TS (new TimeStamp i.e. ASIC-E) containers.
  • If the containers must be compatible with other systems still using the old jDigidoc library, then it is recommended to use the BDoc-TM (old TimeMark) format (with SignatureProfile.LT_TM) for compatibility.

The application which is using DigiDoc4J fails on a clean shutdown?

A new shutdown hook was introduced in release 2.1.0 for shutting down executor services and TSL validation job to provide cleaner application. Please read the documentation of ShutdownHook for a suggestion on how to act accordingly depending on the circumstances.

Getting an IllegalArgumentException (from release 2.1.1 onward)

  • If you get an exception like this java.lang.IllegalArgumentException: Not supported: http://javax.xml.XMLConstants/property/accessExternalDTD)
  • Be sure that your integration doesn't use Xalan or XercesImpl dependencies and uses a patched Java version (JDK7u40+, JDK8 or higher).
  • Xalan and XercesImpl were used to patch XML vulnerabilities in older java versions. They should be discarded with higher versions because they override default Java XML security.
  • If it is not possible to remove Xalan, then you can set your system property to override TransformerFactory : System.setProperty("javax.xml.transform.TransformerFactory","com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl")

Usage of AIA OCSP for TimeStamp based ASIC-E containers (from release 3.1.0)

By default Digidoc4J uses OCSP url given in configuration (default is http://ocsp.sk.ee/). It is now possible to use automatic AIA OCSP selection for signatures made with LT or LTA profile. AIA OCSP usage is not possible for LT_TM profile.

To turn on automatic AIA OCSP usage following parameter must be set: configuration.setPreferAiaOcsp(true); (it is not currently possible to turn the feature on from digidoc4j.yaml)

AIA OCSP responders for Estonia return the certiface status "GOOD" for certificates that have expired. Digidoc4J checks for certificate expiration prior making the OCSP request, therefor no AIA OCSP requests are made for expired certificates.

When the feature is turned on then:

  • AIA OCSP url will be searched from certificate
  • If certificate do not have given AIA OCSP url (older certificates may not have this), then list from digidoc4j.yaml is used to select correct AIA OCSP url. NB! please use latest digidoc4j.yaml together with your implementation, otherwise the OCSP fetching will fail in this step.
  • If match is not found in digicod4j.yaml default is used (http://ocsp.sk.ee/)
You can’t perform that action at this time.