All RPU frames are altered when RPU is injected into a video with different length

[manuelrn]

When I inject an RPU into a HEVC file, if I extract the RPU from the resulting injected HEVC, I get an identical RPU file (same MD5) as the original RPU.

Recently I had to inject an RPU into a HEVC file that was 1 frame longer than the RPU, so I had to use the editor to edit the RPU and duplicate the last frame of the RPU to obtain the same length as the video.

And, as I mentioned above, if I extract the RPU from this injected video I get the same MD5 as the modified RPU that I edited (with 1 more frame). So everything is perfect.

But the new version 0.4.0 has just been published, which automates this process in these types of situations. That is, different lengths of RPU and video.

So I have decided to test version 0.4.0 with that same video that I mentioned above that I used recently (video with 1 frame more than the RPU), so dovi_tool v0.4.0 is supposed to get the last frame of the RPU and simply copy it again below, that is, duplicate the last frame, in order to obtain the same length of the video.

That is, the dovi_tool v0.4.0 should now automatically do exactly the same process that I did manually days ago. And the RPU of the injected video should be identical to the RPU edited by me with the last duplicated frame.

But it's not like that. The resulting RPU edited automatically by the dovi_tool v0.4.0 is different from the RPU edited manually by me. Shouldn't they both be identical?

I have thought that the RPU would be completely identical and only the new last frame would change... But no.

I have reviewed several random frames with the info argument, and all of them have changed compared to their original RPU.
Specifically, I have seen that the vast majority of 'addr' data is altered during this process.

Why is all this data from all frames altered instead of all frames remaining completely intact and just duplicating the last frame?

Thanks and regards!

[quietvoid]

The address is a memory address of the buffer when parsing, it is not data.

I just tried your process and I end up with identical RPU files when comparing an edited (duplicate) RPU and the extracted RPU from an injected video with mismatched length.

So you're only supposed to look at the checksum of the files to see if they're bit identical.

[manuelrn]

The address is a memory address of the buffer when parsing, it is not data.

Okay, but even if it's not data, it should also be kept intact, right?

Or is it correct that the 'addr' data is altered?

I just tried your process and I end up with identical RPU files when comparing an edited (duplicate) RPU and the extracted RPU from an injected video with mismatched length.

So you're only supposed to look at the checksum of the files to see if they're bit identical.

Is it completely identical?

That is, does the 'addr' data also stay original without being altered?

[quietvoid]

No, the addr is runtime dependent when buffers are allocated in memory. It's a pointer, so it will always change.
Like I said it's not data.

It is fine if addr changes, the important is that the files themselves are the same.

[manuelrn]

No, the addr is runtime dependent when buffers are allocated in memory. It's a pointer, so it will always change.

But I have done a lot of tests, and 'addr' has never changed, everything is always kept completely intact.

If I extract an RPU multiple times, all the resulting RPUs are identical.

If I inject an RPU into a HEVC file, then I extract the RPU from the injected HEVC and it is completely identical to the original RPU (including 'addr').

If I edit an RPU to add one more frame at the end, all the frames contained in the new edited RPU (except the new last frame, as it does not exist in the original RPU) are completely identical to the original RPU (including 'addr').

And if I inject the edited RPU into a HEVC file, I extract the RPU from the injected HEVC and it still remains completely identical (including 'addr') to the edited RPU (which is also identical to the original RPU, except the new last frame evidently).

In short, absolutely always, whatever I do, even if I edit the RPU, inject and extract it again, everything always remains intact (including 'addr').

But if instead of previously manually editing the RPU, I let the new dovi_tool v0.4.0 do it automatically, all the 'addr' of all the RPU frames are altered.

Couldn't the automatic process just duplicate the last frame without modifying all the others frames?
In the same way that it is done manually with the editor argument.

Thanks again!

[quietvoid]

The addr is determined when parsing the RPU, not writing it. There's nothing to be done about it and it has nothing to do with the automatic padding when injecting.

The only reasonable way to fix it would be to stop displaying it when using info, and instead just show remaining bits.

[manuelrn]

So 'addr' is not written inside the RPU?
Is it just an information displayed by the info argument?

So what changes inside the RPU must be something else, and that causes the info argument to show altered 'addr'.

Since as I commented previously, the RPUs are altered. That is, the checksum of RPUs does not match, the bin files have different MD5s.
And I thought that this was caused by the 'addr', since it was the only thing that was shown differently by the info argument... But then it must be something else inside the bin file.

Doesn't this happen to you?
Do you need I share my bin files?

[quietvoid]

addr is not written in the file.

When I tested I made sure to only duplicate the last metadata only, then the RPUs were identical.
I can compare the RPUs for you if you want..

[manuelrn]

https://we.tl/t-zwdlLjoDK6

• Original RPU.bin
  The original RPU file.
• Manually edited RPU.bin
  The "Original RPU.bin" file edited manually to duplicate the last frame.
  For this I used:

{
	"duplicate": [
		{
			"source": 157868,
			"offset": 157868,
			"length": 1
		}
	]
}

• Automatically edited RPU.bin
  The "Original RPU.bin" file automatically edited by the new dovi_tool v0.4.0 during the injection process.
  It should be identical to "Manually edited RPU.bin", but it is not. Something changes.

Thanks again!

[manuelrn]

I have more information.

I have removed the last frame from the "Automatically edited RPU.bin" file. For this I used:

{
	"remove": [
		"157869"
	]
}

And I have obtained an RPU identical to the original (Original RPU.bin).

So whatever changes is located in the new last injected frame.

[quietvoid]

Well it appears to just be different metadata..
The first identical frame from the original is 157864, so I don't really know how it ended up last.

The only possibility is because the video has a different presentation order, and that the metadata you think is last is attached to a frame that's decoded before the others.
There's no way to know for sure without debugging with the video.

For example, frame 157864 could be the last presented frame, and is the metadata source for padding when injecting.

[quietvoid]

I tested this again with different videos and it is what I explained previously.
The last metadata taken from the original RPU is not guaranteed to be the last in the final video, depending on the presentation order.

[manuelrn]

Forgiveness for slow to respond. Thanks for answering me.

If you need me to share the video to debug it as you said, tell me. But I think you were already able to reproduce the same case with other videos, right?

So, if I understood you correctly, both RPUs were different because when I do it manually, a frame is duplicated, but when it is done automatically, another different frame is duplicated. Correct?

Thanks!