Implement ICtCp color space per BT.2100 #69
Comments
|
Patches welcome. Is there an enumerated value for this system (matrix_coefficients)? |
|
It's a multi-step process. Convert RGB to LMS then do PQ. Then matrix. It's spelled out in BT.2100.
Regards,
Dzung Hoang
… On Mar 15, 2017, at 9:18 PM, sekrit-twc ***@***.***> wrote:
Patches welcome. Is there an enumerated value for this system (matrix_coefficients)?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
I mean for the standard colorspace triplet (matrix-transfer-primaries). Has a code value been specified by ITU/MPEG? As for the details of ICtCp, I see that the transfer characteristics can be either PG or HLG. |
|
The JCT-VC is preparing draft text to add ICtCp support in HEVC.
http://phenix.int-evry.fr/jct/doc_end_user/documents/25_Chengdu/wg11/JCTVC-Y1003-v1.zip
There is no assigned triplet for ICtCp. The matrix_coeffs syntax element value is 14 for ICtCp color space which is specified by equations (E-62) to (E-64). The transfer function can be PQ (transfer_characteristics = 16) or HLG (transfer_characteristics = 18), but the HEVC spec does not limit ICtCp to these two transfer functions.
|
|
If a matrix_coeffs value has been reserved, that is sufficient. The constants described in BT.2100 (or E-62 to E-64) appear to be specific to the Rec.2020 primaries. Is there a generalization? |
|
ICtCp was designed with Rec.2020 colorimetry in mind. It is possible, but unlikely, that ICtCp will be used with another colorimetry. |
|
Thank you for answering. I will review your patch if/when you submit it. |
|
I finished coding and am testing. |
|
I just checked in my code changes. Please review and post comments here. |
|
Can you upload a ICtCp test image with reference RGB image? I have left my remarks on your commit. |
|
I updated the code per your comments. I am working on generating ICtCp test image using HDRConvert used by JCT-VC/JVET. |
|
unit test pushed |
|
Once you upload the ICtCp test image, I can begin verification and merging. PLEASE NOTE: You must agree to either assign copyright of code to the project owner or license contributions under WTFPL (see COPYING). |
|
I agree to WTFPL. Regarding the ICtCp test image, I have a question on the image file format. I used testapp for my own testing and came across a limitation in the file format. This line of code in colorspaceapp.cpp colorspace_main() is a concern for me.
|
|
The colorspaceapp is only for testing the colorspace module, which works in float. The depthapp can generate floating-point images for use with colorspaceapp. You can upload the test image in any format you like, e.g. 10/16-bit yuv dump, BMP, etc. EDIT: Also, I have forgotten some details of this code, but I believe the second argument to imageframe::read is a default. The path specifier (e.g. 'bmp@') has higher precedence, and the FLOAT argument specifiers the output of imageframe::read. |
|
Any updates? |
|
I have been busy at work and am traveling. The difficulty I'm having is that I cannot find any coded content with ICtCp colorspace.
I thought I could use HDRtools from
https://gitlab.com/standards/HDRTools
but I don't trust it and need further study to verify that it works with ICtCp.
Regards,
Dzung Hoang
… On Mar 30, 2017, at 6:55 PM, sekrit-twc ***@***.***> wrote:
Any updates?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
I finished testing on an SDR image and an HDR image. The files are attached. |
|
I tried to verify the code with your EXR images, but when I convert 'cosmos_2048x858_ictcp' to RGB as such:
The output image is all white pixels. I inspected the contents of the ICtCp EXR, and it appears all Y pixel values are near 1.0, which would be unusual for ST.2084. Can you explain what colorspaces the images you provided are in? EDIT: The conversion also appears to suffer from poor accuracy. When I converted 'color_320x320_bt2020' to ICtCp and back, the recovered image had many artifacts on the right-hand (blue) side.
|
|
BT.2100 assumes that the R/G/B value of 1.0 corresponds to 10000 nits. Also, the floating point representation assumes full-range samples. Therefore you need to specify the following options: "--fullrange-in --fullrange-out --peak-luminance=10000" The cosmos image is an image taken from Cosmos Laundromat and remastered in HDR by Netflix. https://gooseberry.blender.org/ The color square is a sRGB image, and you need to set "--peak-luminance=100." I am attaching the script I used for testing. |
|
Thank you for the update. I was able to verify that the conversion from ICtCp to RGB matched the reference image with '--peak-luminance=10000'. Can you comment on the numerical issues I mentioned in my edit? I reproduced the steps used in the pipeline as a unit test and observed severe error in the green channel for certain test vectors. The attached code can be used to inspect the issue in a debugger. You can also observe these issues if you attempt to round-trip the color square, storing the intermediate result in half-precision (EXR). EDIT: Looking at the error growth, it appears that the LMS-RGB (inverse RGB-LMS) matrix is not stable. The relative error increases dramatically from 1E-5 to over 100%. |
|
I am in the process of implementing a DoubleToHalf conversion in Excel VBA to try to find a stable matrix. |
|
I have not done this investigation yet, but it may be necessary to use gaussian solvers (e.g. sgesv) instead of an inverse matrix. From what I can tell, an LU solution requires no more FLOPs (9 muls) than a matrix-vector product in the 3x3 case. |
|
I am looking into the effect of rounding modes when converting from float to half-float. Hopefully this can give a simpler solution. |
|
Do the standards or reference software provide any guidance on decoding ICtCp? On my end, I tried several approaches, but none of them were able to eliminate the problem. In fact, I noticed the issue can also be reproduced with the test vector (R=0, G=1.0, B=0). Even without any quantization of the intermediate result, this vector comes out as (R=0.09, G=1.0, B=0.014). Part of the issue appears to be the dynamic range of the ST.2084 function, as even 1E-5 absolute error near zero in the linear domain maps to 0.1 in the transformed domain. |
|
I just realized the aforementioned error would be invisible in PQ domain, since 1E-5 linear is 1 nit and 1.0 linear is 10000 nits. Perhaps there is no issue. |
|
I cannot reproduce your result for RGB=[0,1,0]. See the attached spreadsheet. One round trip with 10-bit quantization of ICtCp gives me the reconstructed RGB=[-0.0039, 0.997691, -5.93973E-05]. |
|
The discrepancy appears to be that you are using linear RGB values, but I am using R'G'B'. Additionally, for the 10-bit quantization function, I get a positive R error of the same magnitude (0.0039 -> 39 nits) that you reported. Perhaps this is not such a big deal, compared to the 9977 nits in G, but it appears troubling to me that pure colors can not be round-tripped. |
|
I updated the spreadsheet to fix the quantization function to reflect A.2 in SMPTE-2084-2014, and also to allow selecting the bit-depth. Not surprisingly, the accuracy depends upon the selected bit-depth. Bit-depth of 11 seems to be the sweet spot for preserving the primary colors after clipping. For each bit-depth, we can tweak the LMS-to-RGB matrix to get exact round-trip of the primary colors. I'll attempt that next. |
|
It was easier than I thought to derive a LMS-to-RGB matrix to get exact round-trip of the primary colors. We basically just compute round-trip LMS of the identity RGB matrix, and then compute the matrix inverse of the round-trip LMS. See the attached spreadsheet. |
|
That method only accounts for errors caused by quantization, but in a real workflow, numerical errors can come from many sources, particularly data compression. Consider this scenario:
The general principle that should hold is that errors in the transformed space should result in proportional errors in RGB. |
|
I would disagree with the proportional RGB error metric. The right metric should be perceptual-based. The attached document set has a good treatment of compression errors of IPT-PQ (ICtCp with SMPTE-2084 PQ) compared to Y'CbCr. |
|
I agree that dE2000 is a better measure than RGB per-channel error. Can you provide a reference for computing dE2000 of HDR values? There are only 2^30 code points in 10-bit PQ space, so it should be possible to exhaustively test the conversion error. |
|
This has a DeltaE computation module. https://gitlab.com/standards/HDRTools Look for DistortionMetricDeltaE.cpp. |
|
I noticed that I was using the float-int cast in my quantization function, which was increasing the error considerably. After I applied correct rounding, I am seeing dE2000 of ~0.05 for (R=0,G=1,B=0) with 10-bit quantization and ~0.1 for (R=0,G=1,B=1). (R=0,G=1,B=1) appears to be the worst-case input. The simple YCbCr system has dE2000 of ~0.03 and ~0.01, respectively. The error appears to be stable and does not increase significantly with additional round-trips. Is this a reasonable result? |
|
That seems even better than published results. |
|
I will need to investigate again to ensure I am measuring dE correctly. |
|
I did an exhaustive analysis and found that the overall dE was not worse than NCL for ICtCp as implemented in the commit. |
|
Since one of the aims of ICtCp is to improve color gamut mapping, I suggest you allow ICtCp to be used with any gamut and not just BT.2020. I will use zimg for color gamut mapping experiments and I'm sure others would too if this restriction were removed. |
|
That would require a different RGB-LMS matrix, would it not? I noticed the document in m37266-v2-m37266_IPTPQ_v2.zip proposed for LMS to be a value of color_primaries, but that appears to not have been accepted by JCT-VC. |
|
Now that I think about it, generating ICtCp output with a different input color primaries is already supported by zimg. The input color primaries are first converted to BT.2020 and then ICtCp conversion is performed. The challenge for color gamut mapping is to figure out how to manipulate the ICtCp color space to reduce or expand the color volume. Then the inverse conversion is performed from ICtCp to linear BT.2020 RGB and then to the target color primaries. |
|
@ValZapod I do not understand. Is there some action required from z.lib here? |
|
That's a problem with ffmpeg or ffmpeg's zimg wrapper (I assume you mean using zimg through ffmpeg's zscale filter). I tried that sample, and ffmpeg returns "unspecified" for all relevant colorspace parameters. Is that file even valid? |
|
Hello again, @dzung-hoang . Is there any information about the IPT colorspace? The linked document only states that "proprietary IPT [is] similar to BT.2100 ICtCp." |
|
@sekrit-twc, there is some technical detail in the linked Dolby white paper. |
|
I found the information on page 7. [I P T] = [[0.4 0.2 0.2] [4.4550 -4.8510 0.3960] [0.8056 0.3572 -1.1628]] * [L M S] Is the takeaway that IPT differs from ICtCp in the matrix coefficients step, using the product of the aforementioned three matrices instead of the LMS-ICtCp matrix? |
|
No, the page shows the derivation of ICtCp from IPT. If you were to apply all three matrix multiplies, you would end up with ICtCp.
This info is really background information on how ICtCp was derived from IPT. The exact details are in confidential Dolby Vision documents, which I have not seen. I suppose some more digging around might reveal a public source. I would dig into JVET and JCT-VC documents where Dolby made presentations on IPT and ICtCp. I would also check HDRConvert on github. This is the software JVET and JCT-VC uses to convert different color formats.
Sent from Yahoo Mail for iPhone
On Sunday, November 10, 2019, 1:29 PM, sekrit-twc <notifications@github.com> wrote:
I found the information on page 7.
[I P T] = [[0.4 0.2 0.2] [4.4550-4.85100.3960] [0.80560.3572-1.1628]] * [L M S]
Rotation = [[1 0 0] [0 0.4226 -0.9063] [0 0.9063 0.4226]]
Scalar = [[1 1 1] [1.4 1.4 1.4] [1 1 1]]
Is the takeaway that IPT differs from ICtCp in the matrix coefficients step, using the product of the aforementioned three matrices instead of the LMS-ICtCp matrix?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
There is a typo in your L'M'S' to IPT conversion matrix. As I already replied, Rotation and Scaler conversions are there to convert IPT to ICtCp. The following link confirms the above conversion matrix. https://gitlab.com/standards/HDRTools/blob/master/common/src/ToneMappingBT2390IPT.cpp This should be sufficient information to add support for IPT. |
and others
BT.2100 defines a new color space ICtCp. I can help add support for this to zimg.
The text was updated successfully, but these errors were encountered: