VRAM savings from removing unused blendshapes not as high as when done through Blender

[kitlith]

Comparisons done using the VRAM Calculator from Thry's VRAM performance tools, using:

• a modified version of the Bestboi Chibi avatar base
• comparing only the Body mesh (which is the mesh that the majority of the VRAM usage comes from)
• with only "Use Ring Finger as Foot Collider" enabled and 0 exclusions
• with the following passes commented out: (I might have gone overboard, but I didn't want it touching my materials for now, damnit)
  • OptimizeMaterialSwapMaterials
  • CreateTextureArrays
  • CombineAndOptimizeMaterials
  • OptimizeMaterialsOnNonSkinnedMeshes
  • SaveOptimizedMaterials

Results:

• Unoptimized (53 blendshapes, 57 bones): 11.39 MiB
• OptimizedCopy from this tool (28 blendshapes, 52 bones): 11.02 MiB
• Blendshapes removed using blender (28 blendshapes, 57 bones): 8.94 MiB
  • I tried to remove exactly the same blendshapes as this tool did.

Unfortunately, this isn't 100% rigorous:

• I commented out some passes, so it's possible that I accidentally commented out a pass that actually reduces the amount of blendshape information kept around?
  • I don't think this is the case, because the blendshapes do actually appear to be removed from the optimized mesh
• The tool is doing more than I did in blender because I only removed/baked blendshapes.
  • but to me that just says that the other optimizations aren't making up for this going wrong?

If you want I can work on making a reproduction case that I can send to you that isn't <my personal avatar>

[d4rkc0d3r]

I don't think the VRAM difference comes from handling blendshapes differently.
I think your body mesh has around 13k vertices and the difference comes from expanding uv0 from 2 wide to 4 wide vectors:

d4rkAvatarOptimizer/src/d4rkAvatarOptimizerEditor.cs

Line 1999 in 88be166

targetUv[0].Add(new Vector4(sourceUv[vertIndex].x, sourceUv[vertIndex].y, meshID, 0));

13k * 2 * 4Bytes ~= 0.1MiB
This is the difference between your blender stripped version and the one the optimizer produces.
As a short explanation on why I expand uv0. I save original meshID in uv0.z so I can convert mesh toggles to shader property toggles. uv0.w is also needed for merging materials by saving the original material id per vertex and then the combined shader can load per material data from arrays.
If you don't have ~13k vertices in that mesh I'd like to get to the bottom of the difference, but if you do you can close this issue.

[kitlith]

I think you might have copied the wrong numbers when checking the difference between blender and this tool, because 11.02 - 8.94 is 2.08, not 0.1 -- while what you've mentioned is a factor that's occurring, it's overshadowed by whatever else is happening.

Have some screenshots (feat: a backported mesh inspector view)

Original:

This tool:

Blender:

Some points:

• The blender version is smaller than originally measured since I removed the unused material that I hadn't originally bothered with.
  • Blender removes unused vertices, whereas this tool only removes unused indices. CombineAndOptimizeMaterials() pass does this, so that's my fault.
• The only sizes measured in this inspector is for the vertices and the indices, which sums to less than 2 MiB, but the model takes closer to 9-11.5 MiB of VRAM, the rest of which I assume is because of skinning and blendshape information which is not measured here.
  • The fact that the measured size in these sections barely varies says to me that the 2MiB in variation I'm seeing is because of one of the other sections, which I assume is the blendshapes, given that's what I touched in blender.
• The 0.15 MiB added to vertex data that you'd expect from expanding UV0 from x2 to x4 is accounted for in the vertex data, as you expected.
  • random thought: could you get away with expanding it to x3 instead of x4? Since the UVs are 4 bytes each, you'll retain the 4 byte size alignment for the vertex attributes either way, and only use half of the additional VRAM.
  • I assume you don't do this because of generalization issues, but you could encode the model ID into the UV coordinates outside of the 0-1 range, and avoid any additional VRAM usage. (I think there's a way to make it more general too, but eh)
  • (I'm not fussed about this for now, though one of the things I do want to eventually experiment with is using meshoptimizer to crunch all the vertex attributes. i might not get much out of the experience, though, judging by this.)

[d4rkc0d3r]

Ok, I'll investigate the blendshape situation further. Idk what kinda math I was doing in my head last time, I was so far off its not even funny.

Blender removes unused vertices, whereas this tool only removes unused indices.

CombineAndOptimizeMaterials() does remove unused vertices too, but you didn't use it.

random thought: could you get away with expanding it to x3 instead of x4?

yes, that would be pretty easy. I just simply hadn't bothered so far since the gains seem pretty small.

[kitlith]

Idk what kinda math I was doing in my head last time

You might have been comparing the un-optimized and the optimized version, instead of optimized and blender. No worries.

CombineAndOptimizeMaterials() does remove unused vertices too, but you didn't use it.

I knew that was eventually going to bite me! Irrelevant for people using the tool as designed, then. I'm glad I mentioned the passes that I disabled. Will edit previous message to indicate this.

tbh i'm kinda waiting for a "actually the blendshape space usage is also fixed by enabling another pass, that's also your fault" and hoping that isn't the case >_>

[d4rkc0d3r]

Ok, I think I figured it out. I added Mesh.Optimize() to the new mesh created in CombineAndOptimizeMaterials().
This fixed it in my test case that I did where I didn't merge meshes or materials via settings. Can you dl newest version and check your case by disabling the merge settings instead of commenting out functions?

[kitlith]

That takes it to 10.46 MiB, which is definitely better, but is still leaving 1.52 MiB on the table compared to removing the blendshapes in blender.

I should really reproduce on not-personal avatar so you can work on it directly.

EDIT: I'm confused now. I setup the reproduction, was getting similar behaviour as with my personal avatar, and then i jiggered some things around and now the optimized avatar's mesh is producing good results, VRAM-wise.

EDIT 2: Okay, current status: If I add the component to the avatar, immediately toggle everything off, then I get 10.27 MiB on my reproduction. If I add the component to the avatar, run (a successful) optimization with default settings it gives 8.58 MiB. If I then toggle everything off and re-optimize, it still produces an 8.58 MiB mesh.

So I guess there's still behavior that depends on a something being toggled on when optimizing, at least once in this component's lifetime.

EDIT 3: It appears that it's Delete Unused Game Objects that's triggering the confusing behaviour? (what? this doesn't feel right at all, looking at the code, which I haven't messed with at all this time. is there more pebkac on my end? :/)

[d4rkc0d3r]

No its not just you, I also have seen very inconsistent behavior regarding VRAM size now. Even on consecutive runs on the same avatar with no changes :(
Sometimes the improvements kick in, sometimes they don't. The tiny saves from only expanding uv0 3 wide always apply though.

[kitlith]

hm. i think my experience has been more consistent than that, but maybe the Delete Unused Game Objects thing was a fluke/red herring. Is it at least consistent for you when using default settings?

[d4rkc0d3r]

Turns out Thry's VRAM calculator takes some memory consumption value from a Unity API. This is not VRAM consumption however. I found this out while doing blendshape performance testing. I've notified them of a way to calculate actual VRAM consumption for meshes.