Bank refuses CDB messages with ruby 3.1

[sriedel]

We've run into an odd issue while upgrading our application to ruby 3.1: CDB messages are refused by the bank with a '090004' Return code (EBICS_INVALID_ORDER_DATA_FORMAT). A CDB message with the same gem versions and same order data is accepted when issued from ruby 3.0.

Trying to debug the issue with the IT of the bank in question did not yield very precise results, unfortunately, as the server side cause of the rejection wasn't covered by their logging. The best esitmate of the bank is, that the value of the OrderData attribute cannot be decompressed due to it being invalid.

My suspicion on the other hand is, that this may have something to do with the OpenSSL versions used by ruby 3.0 (OpenSSL 1.1) and ruby 3.1 (OpenSSL 3.0).

Before I go poking things with a stick to see what squeaks, are you aware of any issues regarding ruby 3.1 that may cause OrderData to become unreadable to upstream? Or maybe some pointers where it would be sensible to start more detailed debugging? I have logs of the ebics messages in the relevant communications, in case they would help.

[tobischo]

Unfortunately, no immediate experience.

What we have is

3.2.0 :001 > RUBY_VERSION
 => "3.2.2"
3.2.0 :002 > OpenSSL::VERSION
 => "3.1.0"

and it is working nicely.

However we are certainly not yet on the latest version to verify that everything is ok and I do not have any test accounts/accesses available right now.

The assumption does sound reasonable though.

Which version of the Gem are you using exactly?

I assume CAMT data download is working nicely?
If that is the case, the place to start poking around would be https://github.com/railslove/epics/blob/master/lib/epics/generic_upload_request.rb

[sriedel]

Thanks for the quick response.

We're using epics 2.1.1 (revision 4a261c5) pulled from github, and rubyzip 2.3.2 in both ruby versions.

I'm not very familiar with EBICS; I'm just working on the source tree :)
I assumed CAMT was a command, however searching for it in the epics gem turns up nothing. What should I do to check if CAMT works?

[tobischo]

CAMT files contain the bank statements and for different specific purposes there are slightly different standards, e.g. C53 is for downloading the bank statements with confirmed transfers until end of day of the previous day (CAMT.053) for the accounts attached to you EBICS access. In epics it is implemented in a way that you can pass a start and end date.

[sriedel]

Ok, I've tried retrieving the transactions of the past week with the STA command, and the results are identical in ruby 3.1 and 3.0.

From that I'd deduct that we have no issues reading the banks data, it's just the bank that has issues with reading our encrypted data (authentication works though, maybe that's handled by a different system upstream).

I've tried running the steps used for creating the OrderData and OrderSignature tag values in parallel in a rails console, while trying to remove as much randomness as possible to see if there are any obvious deviations. However up until the sign_pss call, everything seems to be equal. Since the PSS algorithm returns a different value for each call I can't really proceed beyond that. I did however do a cross-verification of the PSS result in the other ruby version, and both verify_pss calls returned true. So no obvious bug/breakdown there.

For now I'm out of ideas as to what I can try next.

As a reference as to what I did in irb:

def cipher
  @cipher ||= OpenSSL::Cipher.new("aes-128-cbc")
end

def pad(d)
  len = cipher.block_size*((d.size / cipher.block_size)+1)

  d.ljust(len, [0].pack("C*")).tap do |padded|
    padded[-1] = [len - d.size].pack("C*")
  end
end

def encrypt(d)
  cipher.reset
  cipher.encrypt
  cipher.padding = 0
  cipher.key = '0123456789abcdef'
  (cipher.update(pad(d)) + cipher.final)
end

data = File.read('txn.xml')
epics = ... # setup of our epics client

# processing for OrderData

deflated = Zlib::Deflate.deflate(data)
Digest::SHA1.hexdigest(deflated)
# ruby 3.1 => 38038da221e34ef66a97458ff618b8942632482d
# ruby 3.0 => 38038da221e34ef66a97458ff618b8942632482d

encrypted = encrypt(deflated)
Digest::SHA1.hexdigest(encrypted)
# ruby 3.1 => eabda2f2353db4de2e6110f5d8adb2173c9fa653 
# ruby 3.0 => eabda2f2353db4de2e6110f5d8adb2173c9fa653

encoded = Base64.encode64(encrypted).gsub(/\n/,'')
Digest::SHA1.hexdigest(encoded)
# ruby 3.1 => 0e5b446eceafeac1d048c511a704e5a9c2794e54
# ruby 3.0 => 0e5b446eceafeac1d048c511a704e5a9c2794e54

# processing for OrderSignature
def digester
  @digester ||= OpenSSL::Digest::SHA256.new
end

raw_digest = digester.digest(data.gsub(/\n|\r/, ''))
Digest::SHA1.hexdigest(raw_digest)
# ruby 3.1 => 601ebcbabcc3d35e74ab1e74fcbe5f1601b217f5
# ruby 3.0 => 601ebcbabcc3d35e74ab1e74fcbe5f1601b217f5

raw_signature = epics.a.key.sign_pss('SHA256', raw_digest, salt_length: :digest, mgf1_hash:   'SHA256',)

# and verify with the following in the respective other ruby version:
epics.a.key.verify_pss('SHA256', ruby_3_1_pss, raw_digest, salt_length: :digest, mgf1_hash: 'SHA256')

# ruby 3.1 verifying ruby 3.0 => true
# ruby 3.0 verifying ruby 3.1 => true

[tobischo]

The last time I had a similarly sounding issue was actually when we switched to using openssl pss sign and verify in #103

The issue back then was that the custom signature/verify method failed to work correctly with keys larger than 2048 bit.

In order to debug that, we (my employer) bought a 1 year license from subsembly for their ebics test server (which costs like 1k EUR), however I do not have that available for lack of need, and because they were really fast in setting things up should the need arise again.

Unfortunately I haven't found any bank or service in germany that provides a test API for free, however there is https://iso20022test.credit-suisse.com/ which should be helpful with this.

Do you happen to have Dockerfiles or base images with which you can recreate the environment and steps that show the issue?
If e.g. you could recreate it and share how I can also take a closer look as well beyond guiding you through the debugging.

As for creating example files for switzerland: https://github.com/salesking/sepa_king supports pain.001 for switzerland and https://github.com/viafintech/sps_king supports it for pain.001 and pain.008 (debits). If this is an issue about the signature, it should be recreatable there.

Another question: How large are the files that you try to send via CDB? Like, how many entries do you have in there?
I am pretty sure that epics still has an issue with large files which is probably not as easily solvable.

[sriedel]

[just saw that the comment hadn't been submitted yet. oops]

I just checked the key sizes, and that should not be the issue. The CDBs are as small as they can be, I think. My test case has only one entry and that corresponds to what is generated in the production code.

No docker images available, unfortunately.

Checking for a test api isn't something I had considered, I'll give that a go. It may take a few days for me to actually get around to setting that up.

[marcusnasarek]

I ran into a similar issue with CCT and the GLS Bank (DZ Bank EBICS server) returns "EBICS_INVALID_ORDER_DATA_FORMAT - The transferred order data does not correspond with the specified format" when switching to ruby 3.2 (from 2.7). C53 and C52 Order Types work.

[tobischo]

Could you maybe provide some additional details?

Key size, ruby version, openssl version, ...

[marcusnasarek]

OpenSSL 3.0.9 30 May 2023 (Library: OpenSSL 3.0.9 30 May 2023)
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [x86_64-linux]
epics (2.1.1)
key length 2048

[tobischo]

In the hopes that it is still applicable, DZBank seems to provide (or did provide) a testing environment for ebics:
https://www.dzbank.de/content/dam/dzbank_de/de/home/produkte_services/Firmenkunden/dateien/downloads/EBICS/EBICS-Testzugang%20Kundeninformation_V1.1.pdf

Since I do not have a production EBICS ID with DZBank I cannot use that for testing

[marcusnasarek]

I tried but the GLS seems use their stack but to not sync the customer IDs with their test system. I then tried manually to check the crypto methods from the libs (encrypt, sign, digest, ..) if ruby 2.7.4/openssl 2.1.2 are doing the same as ruby 3.2.2/ openssl 3.1.0. encryption, signing, padding, base64, hashing with SHA256, ... all the same back and forth. I cannot find a difference. Also the XML documents sent via CCT are both the same, obviously the respective signature and encryption values are different because of the salt and random session key. I cannot find a reason why it's rejected. Looks like it's caused by the encrypted_order_signature value in the SignatureData field, but don't see why.

[tobischo]

Were both ruby versions compiled with the same openssl version underneath?

[marcusnasarek]

I manage the versions with rbenv and the ruby-build script compiles v2.7.4 against openssl1.1 and v3.2.2 against openssl 3.0.

…

On Wed, 14 Jun 2023 at 19:07, Tobias Schoknecht ***@***.***> wrote: Were both ruby versions compiled with the same openssl version underneath? — Reply to this email directly, view it on GitHub <#142 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ANKLRJT2NS4GQ5T4LIAQKJ3XLHVVJANCNFSM6AAAAAAX4RFDZ4> . You are receiving this because you commented.Message ID: ***@***.***>

-- *Marcus Nasarek* Managing Director *etonec GmbH* m: +49 152 2161 9714 e: ***@***.*** w: www.etonec.com

-- etonec GmbH | Kopernikusstrasse 35, 10243 Berlin | AG Berlin-Charlottenburg HRB 217087 | Geschäftsführer: Marcus Nasarek | Gesellschaftssitz Berlin | VAT DE323610849

[marcusnasarek]

I linked ruby 3.2.2 with openssl 1.1.1t and it worked for openssl gem version 3.1.0 and 2.2.3. If build ruby 3.2.2 with openssl 3.1.0 it throws the error "EBICS_INVALID_ORDER_DATA_FORMAT - The transferred order data does not correspond with the specified format"

[tobischo]

I wonder if it might be related to this:

#138 (comment)

So it is not the first time I came across an issue with openssl 3 and some things seem to still be missing there (oddly enough).

As of right now, the pragmatic solution would be to stick with openssl 1.1.1t then if this is a possibility for you.

Eventually of course we will need to look into this more deeply. I didn't have the time to dig into this though without being able to easily recreate the issue

[tobischo]

Looking at this recent discussion #129 (comment) and how it played out, I have also seen some weird behaviour with openssl and how it is used in different versions of epics and different versions of openssl, that ruby was linked against during compile.

I assume loading the keys works nicely between both version and between different openssl versions as that would have been mentioned here then. Therefore not directly related.

One thing I could imagine right now in order to debug would be to try to overwrite the bit that is providing the random part via monkey patching (haven't checked if possible yet) to get the randomness out of the way and have something a bit more comparable.

If it is then completely identical, then the bug would be in openssl 3 or the openssl bindings for ruby 😅

[tobischo]

Released a new version (v2.1.2) with the fix provided by @sos4nt

Please let me know if this solves it for everyone or if we need to dig into this further.
If no issues show up, I will probably close this issue in a week or two

[sriedel]

Initial tests look positive. We'll probably get to do a full test sometime towards the end of the month.

[sriedel]

We've been running with the new version for over a week now and have not encountered an error. I think we can consider this issue fixed. :)