OverflowError "VR of 'DS' must be <= 16 characters long" triggered when element is 16 characters long

[fedorov]

Describe the bug

OverflowError triggered while accessing PixelData, which the values compliant with the standard. In the sample referenced in the example below, we have this, which satisfies DS VR:

(0028,0030) DS [.002006091181818\.002006091181818]      #  34, 2 PixelSpacing

But nevertheless the error is triggered while trying to access PixelData:

OverflowError: Values for elements with a VR of 'DS' must be <= 16 characters long, 
but the float provided requires > 16 characters to be accurately represented. Use a 
smaller string, set 'config.settings.reading_validation_mode' to 'WARN' to override 
the length check, or explicitly construct a DS object with 'auto_format' set to True

Expected behavior

OverflowError does not get triggered.

Steps To Reproduce

Follow the steps of this Colab notebook: https://colab.research.google.com/drive/1FcSgjBKazh0YN-jlJYdID0YUTh90CAvZ?usp=sharing

Your environment

module       | version
------       | -------
platform     | Linux-5.4.144+-x86_64-with-Ubuntu-18.04-bionic
Python       | 3.7.13 (default, Mar 16 2022, 17:37:17)  [GCC 7.5.0]
pydicom      | 2.3.0
gdcm         | _module not found_
jpeg_ls      | _module not found_
numpy        | 1.21.5
PIL          | 9.1.0
pylibjpeg    | _module not found_
openjpeg     | _module not found_
libjpeg      | _module not found_

Related issue: imi-bigpicture/wsidicom#49

cc: @DanielaSchacherer @dclunie @hackermd

[pieper]

For reference, a possibly similar issue came up in dcmjs: dcmjs-org/dcmjs#175

[mrbean-bremen]

I had a quick look, and the problem seems to be that the length is not taken from the original string, but from the string representation, which in this case adds a leading zero... This check has been introduced in pydicom 2.2.0.

[mrbean-bremen]

For reference, a possibly similar issue came up in dcmjs: dcmjs-org/dcmjs#175

This is indeed somewhat similar as it also converted the original value to a normalized representation which is too long for DS. Note that pydicom generally preserves the original string, in this case it just did not do the validation on that string.

[pieper]

The pydicom approach of keeping the original values for DS and other potentially lossy conversions makes good sense and we should see how something similar could be done in javascript.

[mrbean-bremen]

Yes, this is a simple solution to a non-trivial problem, as it avoids the problems of floating point precision vs notation (or rather sidesteps them...).

[fedorov]

Thanks for addressing this so quickly!