Error sending with AS4: java.security.InvalidAlgorithmParameterException: Expected AttachmentTransformParameterSpec

[runholen]

Just realised I probably first posted my issue first in the wrong repository, difi/oxalis instead of difi/Oxalis-As4, sorry about that. As the deadline for as4 is today I post it here too.
We did the AS4 testing earlier this month, but due to issue #452 in difi/oxalis, we have avoided upgrading our production server until the end of month, as we wanted AS2 to still work against older servers.
However, when we upgraded yesterday, we started getting error messages when sending as4:
no.difi.oxalis.as4.lang.OxalisAs4TransmissionException: Failed to send message
at no.difi.oxalis.as4.outbound.As4MessageSender.send(As4MessageSender.java:91)
Caused by: javax.xml.ws.soap.SOAPFaultException: Cannot setup signature data structure
at org.apache.cxf.jaxws.DispatchImpl.mapException(DispatchImpl.java:285)
Caused by: org.apache.cxf.interceptor.Fault: Cannot setup signature data structure
at org.apache.cxf.ws.security.wss4j.policyhandlers.AsymmetricBindingHandler.doSignBeforeEncrypt(AsymmetricBindingHandler.java:237)
Caused by: org.apache.wss4j.common.ext.WSSecurityException: Cannot setup signature data structure
Original Exception was org.apache.wss4j.common.ext.WSSecurityException: Expected AttachmentTransformParameterSpec
Original Exception was java.security.InvalidAlgorithmParameterException: Expected AttachmentTransformParameterSpec
at org.apache.wss4j.dom.message.WSSecSignatureBase.addReferencesToSign(WSSecSignatureBase.java:221)
Caused by: org.apache.wss4j.common.ext.WSSecurityException: Expected AttachmentTransformParameterSpec
Original Exception was java.security.InvalidAlgorithmParameterException: Expected AttachmentTransformParameterSpec
at org.apache.wss4j.dom.message.WSSecSignatureBase.addAttachmentReferences(WSSecSignatureBase.java:304)
Caused by: java.security.InvalidAlgorithmParameterException: Expected AttachmentTransformParameterSpec
at org.apache.wss4j.dom.transform.AttachmentContentSignatureTransform.init(AttachmentContentSignatureTransform.java:66)

I am really at a loss to what is the problem. Anyone who experiences the same, and have found a solultion to this?
Btw, it seems that the first time I send after a tomcat-reboot, I also get these errors:

Caused by: javax.xml.ws.soap.SOAPFaultException: A security error was encountered when verifying the message
at org.apache.cxf.jaxws.DispatchImpl.mapException(DispatchImpl.java:285)
Caused by: org.apache.cxf.binding.soap.SoapFault: A security error was encountered when verifying the message
at org.apache.cxf.binding.soap.interceptor.Soap12FaultInInterceptor.unmarshalFault(Soap12FaultInInterceptor.java:156)
org.apache.cxf.binding.soap.SoapFault: A security error was encountered when verifying the message
at org.apache.cxf.ws.security.wss4j.WSS4JUtils.createSoapFault(WSS4JUtils.java:236)
Caused by: org.apache.wss4j.common.ext.WSSecurityException: org.apache.jcp.xml.dsig.internal.dom.ApacheOctetStreamData cannot be cast to org.apache.jcp.xml.dsig.internal.dom.ApacheOctetStreamData
at org.apache.wss4j.dom.processor.SignatureProcessor.verifyXMLSignature(SignatureProcessor.java:399)
Caused by: java.lang.ClassCastException: org.apache.jcp.xml.dsig.internal.dom.ApacheOctetStreamData cannot be cast to org.apache.jcp.xml.dsig.internal.dom.ApacheOctetStreamData
at org.apache.wss4j.dom.transform.AttachmentContentSignatureTransform.transform(AttachmentContentSignatureTransform.java:104)

[FrodeBjerkholt]

Which versions of oxalis and oxalis-as4 do you use?

[runholen]

We upgraded to oxalis 4.1.0. We first tried with oxalis-as4 4.1.0 as that was what we had used earlier this month when testing, but after that failed we tried upgrading to oxalis-as4 4.1.1 but with no luck.

[FrodeBjerkholt]

This seem a bit strange: org.apache.jcp.xml.dsig.internal.dom.ApacheOctetStreamData cannot be cast to org.apache.jcp.xml.dsig.internal.dom.ApacheOctetStreamData

Which JDK are you using?

[runholen]

java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

[FrodeBjerkholt]

Could you try upgrading to:
java version "1.8.0_241"
Java(TM) SE Runtime Environment (build 1.8.0_241-b07)
Java HotSpot(TM) 64-Bit Server VM (build 25.241-b07, mixed mode)

[runholen]

We have now just switched to openjdk version "1.8.0_242"
We still cannot send. But now we seem to get another error message.

no.difi.oxalis.as4.lang.OxalisAs4TransmissionException: Failed to send message
Caused by: javax.xml.ws.soap.SOAPFaultException: These policy alternatives can not be satisfied:
{http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702}X509Token: The received token does not match the token inclusion requirement
{http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702}SignedParts: Soap Body is not SIGNED
at org.apache.cxf.jaxws.DispatchImpl.mapException(DispatchImpl.java:285)

Above this, we also get the following error messages:
org.apache.cxf.binding.soap.SoapFault: A security error was encountered when verifying the message
Caused by: org.apache.wss4j.common.ext.WSSecurityException: javax.xml.crypto.dsig.TransformException: javax.security.auth.callback.UnsupportedCallbackException: Unsupported callback
at org.apache.wss4j.dom.processor.SignatureProcessor.verifyXMLSignature(SignatureProcessor.java:399)
Caused by: javax.xml.crypto.dsig.XMLSignatureException: javax.xml.crypto.dsig.TransformException: javax.security.auth.callback.UnsupportedCallbackException: Unsupported callback
at org.apache.jcp.xml.dsig.internal.dom.DOMReference.transform(DOMReference.java:541)
Caused by: javax.xml.crypto.dsig.TransformException: javax.security.auth.callback.UnsupportedCallbackException: Unsupported callback
at org.apache.wss4j.dom.transform.AttachmentContentSignatureTransform.attachmentRequestCallback(AttachmentContentSignatureTransform.java:137)
Caused by: javax.security.auth.callback.UnsupportedCallbackException: Unsupported callback
at org.apache.cxf.ws.security.wss4j.AttachmentCallbackHandler.handle(AttachmentCallbackHandler.java:111)

[FrodeBjerkholt]

Can you show me the oxalis.xml of the inbound server and the oxalis.xml of the outbound sender?
If you are using the oxalis-standalone sender - What command line are you using?

[runholen]

We use the oxalis-standalone sender in a servlet like this:
TransmissionRequestBuilder builder = ooc.getTransmissionRequestBuilder();
builder.payLoad(new FileInputStream(f));
ParticipantIdentifier pi1 = ParticipantIdentifier.of(request.getReceiver());
ParticipantIdentifier pi2 = ParticipantIdentifier.of(request.getSender());
builder.receiver(pi1);
builder.sender(pi2);
TransmissionRequest transmissionRequest = builder.build();
Transmitter transmitter = ooc.getTransmitter();
TransmissionResponse response = transmitter.transmit(transmissionRequest);

[runholen]

here is the conf file too:

[OysteinLq]

EDIT: Running Inbound and Outbound in separate containers fixed this issue in Wildfly as well.

FWIW, we get the exact same Exceptions using Oxalis 4.1.0 and Oxalis AS4 4.1.1. It's flaky on our end though -- restarting the Outbound module always lets us transmit a single file before it starts failing again.

If you're running other applications in the same Tomcat, you might want to look into whether they're using different versions of cxf or wss4j (and possibly santurio xmlsec?) There's a lot of "magic" in discovering the security protocols, so these things are liable to give surprising results when they're unable to discover implementations of various interfaces etc, or discover an unexpected version.

We've not reported our problems, as we're running Oxalis in Wildfly 12, so we're fairly sure it's dependency hell biting us. (The Oxalis documentation is quite clear that we're on our own when we don't use Tomcat)

[FrodeBjerkholt]

There might be something to what @OysteinLq sais, beacuse of the following:

The stacktrace show that an UnsupportedCallbackException is thrown in AttachmentCallbackHandler.
When inspecting that code, we see that the exception is thrown when the callback argument is not an instanceof AttachmentRequestCallback, AttachmentResultCallback or AttachmentRemovalCallback.

In the calling class we see this:

        AttachmentRequestCallback attachmentRequestCallback = new AttachmentRequestCallback();
        attachmentRequestCallback.setAttachmentId(attachmentId);
        try {
            attachmentCallbackHandler.handle(new Callback[]{attachmentRequestCallback});
        } catch (Exception e) {
            throw new TransformException(e);
        }

This tells us that the argument is indeed an instance of AttachmentRequestCallback, so the execption shouldn't have been thrown.

It sounds very much like a classpath issue of some sort.

[runholen]

Hi again. I have now testet for a classpath-issue in my sender component by doing the following:
Instead of sending from tomcat, I have now made a standalone-class that looks like this:

public class As4TestSender {
public static void main(String[] args) throws Exception{
if (args.length == 0 || args[0].equals("sendnow") == false) throw new Exception("Aborting");
File f = new File("/home/rho/xml/avio_proline/9000432.xml");
System.out.println("Trying to send file "+f.getName());
OxalisOutboundComponent ooc = new OxalisOutboundComponent();
TransmissionRequestBuilder builder = ooc.getTransmissionRequestBuilder();
builder.payLoad(new FileInputStream(f));
ParticipantIdentifier pi1 = ParticipantIdentifier.of("0192:979990537");
ParticipantIdentifier pi2 = ParticipantIdentifier.of("0192:983777805");
builder.receiver(pi1);
builder.sender(pi2);
TransmissionRequest transmissionRequest = builder.build();
Transmitter transmitter = ooc.getTransmitter();
TransmissionResponse response = transmitter.transmit(transmissionRequest);
System.out.println("Ref: "+response.getTransmissionIdentifier().getIdentifier());
}
}

This class is run from a sh-file that is defined as follows:
java -classpath .:animal-sniffer-annotations-1.18.jar:cxf-rt-security-3.3.4.jar:jakarta.activation-api-1.2.1.jar:opensaml-saml-impl-3.3.0.jar:txw2-2.3.2.jar:asm-7.1.jar:cxf-rt-security-saml-3.3.4.jar:jakarta.xml.bind-api-2
.3.2.jar:opensaml-security-api-3.3.0.jar:woodstox-core-5.0.3.jar:bcprov-jdk15on-1.57.jar:cxf-rt-transports-http-3.3.4.jar:jasypt-1.9.3.jar:opensaml-security-impl-3.3.0.jar:wsdl4j-1.6.3.jar:checker-qual-2.8.1.jar:cxf-rt-ws-
addr-3.3.4.jar:java-support-7.3.0.jar:opensaml-soap-api-3.3.0.jar:wss4j-bindings-2.2.4.jar:commons-codec-1.13.jar:cxf-rt-wsdl-3.3.4.jar:jaxb-runtime-2.3.2.jar:opensaml-xacml-api-3.3.0.jar:wss4j-policy-2.2.4.jar:commons-io-
2.6.jar:cxf-rt-ws-policy-3.3.4.jar:joda-time-2.10.3.jar:opensaml-xacml-impl-3.3.0.jar:wss4j-ws-security-common-2.2.4.jar:cryptacular-1.1.1.jar:cxf-rt-ws-security-3.3.4.jar:jsr305-3.0.2.jar:opensaml-xacml-saml-api-3.3.0.jar
:wss4j-ws-security-dom-2.2.4.jar:cxf-core-3.3.4.jar:ehcache-2.10.6.jar:listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:opensaml-xacml-saml-impl-3.3.0.jar:wss4j-ws-security-policy-stax-2.2.4.jar:cxf-rt-bindin
gs-soap-3.3.4.jar:error_prone_annotations-2.3.2.jar:mail-1.4.7.jar:opensaml-xmlsec-api-3.3.0.jar:wss4j-ws-security-stax-2.2.4.jar:cxf-rt-bindings-xml-3.3.4.jar:failureaccess-1.0.1.jar:metrics-core-3.1.2.jar:opensaml-xmlsec
-impl-3.3.0.jar:xml-resolver-1.2.jar:cxf-rt-databinding-jaxb-3.3.4.jar:FastInfoset-1.2.16.jar:neethi-3.1.1.jar:oxalis-as4-4.1.1.jar:xmlschema-core-2.2.4.jar:cxf-rt-features-logging-3.3.4.jar:guava-28.1-jre.jar:opensaml-cor
e-3.3.0.jar:oxalis-standalone.jar:xmlsec-2.1.4.jar:cxf-rt-frontend-jaxws-3.3.4.jar:istack-commons-runtime-3.0.8.jar:opensaml-profile-api-3.3.0.jar:stax2-api-3.1.4.jar:cxf-rt-frontend-simple-3.3.4.jar:j2objc-annotations-1.3
.jar:opensaml-saml-api-3.3.0.jar:stax-ex-1.8.1.jar As4TestSender $1

It is run in a directory that only have these jar-files, the sh-file, and the class file, and nothing else.
The jar-files are the oxalis-standalone.jar file (13337314 bytes), and the 67 jar-files in oxalis-as4-4.1.1-dist. no other jar-file is included.

Running this file then gives these error messages:
org.apache.cxf.binding.soap.SoapFault: A security error was encountered when verifying the message
Caused by: org.apache.wss4j.common.ext.WSSecurityException: org.apache.jcp.xml.dsig.internal.dom.ApacheOctetStreamData cannot be cast to org.apache.jcp.xml.dsig.internal.dom.ApacheOctetStreamData
at org.apache.wss4j.dom.processor.SignatureProcessor.verifyXMLSignature(SignatureProcessor.java:399)
Caused by: java.lang.ClassCastException: org.apache.jcp.xml.dsig.internal.dom.ApacheOctetStreamData cannot be cast to org.apache.jcp.xml.dsig.internal.dom.ApacheOctetStreamData
at org.apache.wss4j.dom.transform.AttachmentContentSignatureTransform.transform(AttachmentContentSignatureTransform.java:106)
org.apache.cxf.binding.soap.SoapFault: A security error was encountered when verifying the message
at org.apache.cxf.ws.security.wss4j.WSS4JUtils.createSoapFault(WSS4JUtils.java:236)
Caused by: org.apache.wss4j.common.ext.WSSecurityException: org.apache.jcp.xml.dsig.internal.dom.ApacheOctetStreamData cannot be cast to org.apache.jcp.xml.dsig.internal.dom.ApacheOctetStreamData
at org.apache.wss4j.dom.processor.SignatureProcessor.verifyXMLSignature(SignatureProcessor.java:399)
Caused by: java.lang.ClassCastException: org.apache.jcp.xml.dsig.internal.dom.ApacheOctetStreamData cannot be cast to org.apache.jcp.xml.dsig.internal.dom.ApacheOctetStreamData
at org.apache.wss4j.dom.transform.AttachmentContentSignatureTransform.transform(AttachmentContentSignatureTransform.java:106)

no.difi.oxalis.as4.lang.OxalisAs4TransmissionException: Failed to send message
at no.difi.oxalis.as4.outbound.As4MessageSender.send(As4MessageSender.java:91)
Caused by: javax.xml.ws.soap.SOAPFaultException: These policy alternatives can not be satisfied:
{http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702}X509Token: The received token does not match the token inclusion requirement
{http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702}SignedParts: Soap Body is not SIGNED

[FrodeBjerkholt]

What about the the oxalis-standalone. Does that work as expected?

[runholen]

Have now tried running it as standalone with the following:
java -classpath .:standalone/*:as4/* eu.sendregning.oxalis.Main -f /home/rho/xml/avio_proline/9000432.xml -s 0192:983777805 -r 0192:979990537 -u https://ap.aviocloud.com/oxalis/as4 --cert /home/amj/ap_prod.cer --protocol peppol-transport-as4-v2_0

And I get this error:
java.util.concurrent.ExecutionException: no.difi.oxalis.as4.lang.OxalisAs4TransmissionException: Failed to send message
Caused by: no.difi.oxalis.as4.lang.OxalisAs4TransmissionException: Failed to send message
Caused by: org.apache.cxf.ws.policy.PolicyException: These policy alternatives can not be satisfied:
{http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702}X509Token: The received token does not match the token inclusion requirement
{http://docs.oasis-open.org/ws-sx/ws-securitypolicy/200702}SignedParts: Soap Body is not SIGNED
at org.apache.cxf.ws.policy.AssertionInfoMap.checkEffectivePolicy(AssertionInfoMap.java:179)

[FrodeBjerkholt]

I use the following commandline when sending to my local test AP:

java -DOXALIS_HOME=/c/dev/peppoltest/.oxalis \
 -classpath "standalone/*;as4/*" eu.sendregning.oxalis.Main \
-cert ap.cer \
-f 9000432.xml \
--protocol peppol-transport-as4-v2_0 \
 -u "http://localhost:8080/oxalis/as4"

You do not need to specify sender and receiver, because those are automatically extracted from the SBDH in the file you are sending,

I am not sure if you can use the -u parameter when communicating with a production endpoint. You should test against an AP using a TEST certficate.

[jonassss]

Hi, we are experiencing a similar issue in production where we are experiencing an exception caused by org.apache.wss4j.common.ext.WSSecurityException: javax.xml.crypto.dsig.TransformException: javax.security.auth.callback.UnsupportedCallbackException: Unsupported callback.

On further inspection, this is happening during the processing of the SignedInfo where we have a reference such as:

<ds:Reference URI="cid:64eaaf80-6f47-4170-9904-4f6df39a176d@wf01-prod.fdc.c.bitbit.net">
      <ds:Transforms>
         <ds:Transform Algorithm="http://docs.oasis-open.org/wss/oasis-wss-SwAProfile-1.1#Attachment-Content-Signature-Transform" />
      </ds:Transforms>
      <ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256" />
      <ds:DigestValue>d5jroYwpGf9boMqeZi1nN878ADE0T482mYQKRYg884s=</ds:DigestValue></ds:Reference>

We checked the libraries for the correct handlers, and all of the above algorithms should be supported.
The uri should be supported by org.apache.wss4j.dom.resolvers.ResolverAttachment.
The transform should be supported by org.apache.wss4j.dom.transform.AttachmentCiphertextTransform.

We also see a similar processing that leads to success (altough not in the SignedInfo).
The following example passes with success and originates from the testbed application https://testbed.peppol.eu/secure/suite/view.

<ns6:Reference URI="cid:f99488d4-1f4b-459b-bc56-e1d30acfd499@ip-10-241-1-75">
                        <ns6:Transforms>
                           <ns6:Transform Algorithm="http://docs.oasis-open.org/wss/oasis-wss-SwAProfile-1.1#Attachment-Content-Signature-Transform" />
                        </ns6:Transforms>
                        <ns6:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256" />
                        <ns6:DigestValue>Y7H21ALFEoR/gGipT3LOC+D/aiKcj8lXw7eHhyvHhh8=</ns6:DigestValue>
</ns6:Reference>

We are on the latest commit on both Oxalis (1126a75) and Oxalis AS4 (b3f5cfd).
Altough we have included the AS4 functionality using git submodule.
We are using CFX 3.3.5, wss4j 2.2.4 and xmlsec 2.1.4.

Edit: It seems to process correctly after we restart, but then stops processing correctly after a while (~2 hours).
We are running Oxalis as an application on Tomcat.
Outbound messages are processing correctly.

[FrodeBjerkholt]

Do you have an outbound servlet or similar on the same http server instance as the oxalis-inbound?
If so - can you try to run them on two separate instances? Like two tomcat servers on different ports?

[FrodeBjerkholt]

When conformance testing we had problems running two instances of the same war on one tomcat server. The problems disappeared when we ran two tomcat servers instead.

[runholen]

Yes we will try to set up two tomcat-instances as a workaround. After quite a lot of testing, we finally got a few of the EHFs through when using the standalone-approach. But once the Sender-Servlet was used once, all as4 -receivings from then on erred due to some conflict, possibly because the sender servlet has it's own copy of the as4-files.

[phax]

This is a bug in WSS4J: https://issues.apache.org/jira/projects/WSS/issues/WSS-660

[FrodeBjerkholt]

I am closing the issue. Will reopen if I get new reports.

[dladlk]

Suggested workaround for Oxalis-AS4 NemHandel eDelivery, which should work also on Oxalis-AS4 with the same CXF version: https://github.com/dladlk/oxalis-nemhandel-tomcat-multiple-instances

[phax]

Nice summary :)