imagecache: remove redundant ImageSpec from ImageCache internals

[bfraboni]

First draft of backend changes for issue #4436.

Description

• add ImageDims struct to store only a subset of ImageSpec or use it as a view on an existing ImageSpec.
• move ImageDims and ImageSpec ownership to ImageCacheFile and use them as a pool
• add ImageSpec* m_spec in SubimageInfo to access the descriptors that are common across all mip level
• add ImageDims* m_dims in LevelInfo to access the despcriptors that differ per mip level
• add functions to access to mip level dimensions:

const Dimensions& SubimageInfo::leveldims(int miplevel) const;

• do the plumbing across all libtexture to use the new interface

Tests

I performed a batch of testing to compare the memory usage and performance of different strategies using 17 ptex files from the PTex library samples (some from the Moana Island dataset, so they are quite heavy). The baseline numbers are approximates but gives a good estimate of the memory usage.

Baseline	Total (MB)	Specs (MB) (%)	# specs	Subimages (MB) (%)
OIIO 3.0 main	1500	1200 (80%)	4936364	300 (20%)

Method	Total (MB)	Specs (MB) (%)	# specs	Subimages (MB) (%)
No reuse	787,2	454,3 (58%)	4936381	332,5 (42%)
Reuse dist=1	442,2	109,3 (25%)	1193485	332,5 (75%)
Reuse dist=10	368,5	35,6 (10%)	380506	332,5 (90%)
Reuse dist=inf	333,2	0,292 (<1%)	3020	332,5 (>99%)

These numbers shows that the memory is now bounded by the SubimageInfo + LevelInfo structure sizes, and not from the descriptors ImageSpec + Dimensions  with the right amount of reuse across subimages and mip levels.

Checklist:

• I have updated the documentation, if applicable. (Check if there is no
  need to update the documentation, for example if this is a bug fix that
  doesn't change the API.)
• I have ensured that the change is tested somewhere in the testsuite
  (adding new test cases if necessary).

[bfraboni]

Hey @lgritz ,

I'm almost done with this PR, apologies for the massive changes, but I've reworked a bit the ownership of descriptors in the ImageCache.

As we discussed offline, the final version consists in adding a standalone structure, internal to the private image cache, to describe or access a subset of the ImageSpec.

One ImageSpec is allocated per SubimageInfo, and one Dimensions is allocated per LevelInfo if it differs from the ImageSpec at lip level 0.

The owner of the allocated data is the Image cache file itself and it is all released when 'invalidate_file' is called with unique_ptr.
LevelInfo and SubimageInfo only store raw pointers to access the descriptors.

The descriptors are reduced to a minimum if we reuse them across subimages and miplevels. I implemented a 'lazy' reuse scheme that checks up to the 10 previous subimages to find a matching descriptor. This can save up to 90% of memory for some Ptex files.

Note that my tests reusing across all descriptors seems to save further up to 99% of the descriptors memory, but at that point the memory usage is dominated by the size of SubimageInfo and LevelInfo structures.

The LevelInfo footprint has been reduced at its minimum so I can't really compress further and that's what dominates the current version; shared ImageSpec / LevelSpec takes few tens of mb while few millions of SubimageInfo / LevelInfo takes few hundreds of mb now.

I think the PR is close to the final version, If you have some time, could you have a quick look at the code and let me know what you think ?
Thank you 🙏

[bfraboni]

@lgritz I've updated the comment with what's in the PR, the tests are all green, the history is not clean but I wait for your input and the list of required changes before squashing all that together.
Should I do anything else before review ? (I can't really ask on github, the area for review is grayed)
Should it be discussed in the TSC meeting?

[lgritz]

Does it only break unit tests by printing the "oiio:miplevels" in certain test output and we just need to update that test output reference? Do you need exr files to do this trick?

[bfraboni]

It seems to break more than the just output, i.e. for Ptex I just had to update the output text and the test passed again, for Exr it is beyond my understanding so far..

[lgritz]

Sorry it's taken me a while to get to this. My comments are extremely minor, and other than that, this all LGTM. Make those minor changes and rebase to the current main (there seems to be some minor conflict) and then I am ready to merge it.

[bfraboni]

Thanks for the review @lgritz , I implemented your feedbacks in the last commits and fixed the conflicts, I think it is good to go !

[lgritz]

LGTM, thanks for the edits