Scripts - Rigify armature overhaul

[ilcane87]

Note from the future: these scripts are now integrated into the addon, so you don't need to download them from here, just do as instructed in the video tutorials, but don't forget to activate the Rigify addon first, by following these steps:

• In the Blender preferences, activate the Rigify addon (it's already there):
  

• Enable AutoRun Python Scripts from the Save & Load section:
  
  You may need to save the preferences and restart Blender in order for Rigify to run properly.

Rigify Scripts.zip (Updated 2023-04-03)

These scripts will automatically make the KKBSP imported armature Rigify-compliant while preserving all existing joint correction functionalities, meaning you'll be able to generate an elaborate control rig from it that contains a lot of additional features and automations:

I know it looks messy, but everything is actually neatly organized in layers that you can show or hide at will from the side panel.

Warning: these scripts heavily alter the original armature and some vertex groups, which I assume will prevent any of the features of KKBSP that affect them from working, so I advise you to duplicate the whole collection in advance.

If you aren't familiar with Rigify or character rigs in general then the armature may look confusing to you, and you won't be able to leverage all of its features, so I recommend checking out this playlist that will teach you the basics:
https://www.youtube.com/watch?v=-JSFcSxsaTs&list=PLdcL5aF8ZcJv68SSdwxip33M7snakl6Dx
The same youtube channel also contains several videos about working with Rigify armatures for different tasks, like exporting to game engines or retargeting animations, so it's worth spending some time on it if you're interested.

On a side note, expect the hair/accessories layers to be a bit messy since their contents are unpredictable and I had to use a generic algorithm for their controls; you can always go back to the original armature and make a few adjustments and then, while it's selected in Object Mode, click the star-shaped 'Convert for Rigify' button in the 'KKBP Extras' section of the plugin.
And as per Rigify conventions, you'll find all Deform bones in the third to last layer (which isn't shown in the Rig Layers panel), in case you want to adjust some weights.

And that's it, I've tested these scripts with several characters and they work fine, but if you run into any issues you can link to the characters causing them here so I might be able to reproduce them.
Also, let me know if I forgot to carry over any of the features of the original armature so I can make amendments.

[FlailingFog]

Dang, this looks great! The features shown in that playlist definitely trump what's on the current armature. Thanks for writing these scripts, I'll check them out.

[FlailingFog]

Is the generate rig button supposed to be selectable after the first script is run? The button is there and the script looks like it's modifying things but the armature doesn't seem to be recognized as a metarig so I can't use the button.

Nevermind, I got it working on 2.93 (didn't seem to work on 3.0). Apparently I didn't know what I was missing because these extra features are a lot of fun to play around with. In addition to the eye controls, it also looks like you made a ton of controls for the face and chest accessible too!

Also, let me know if I forgot to carry over any of the features of the original armature so I can make amendments.

Does Rigify support three spine bones instead of two? Maybe I didn't dig far enough, but out of the three spine bones (Spine, Chest, Upper chest) I only see access to the Chest and Upper chest. The Hips are parented to the Spine, so that third spine bone is essentially lost. Other than that, I think you hit every feature on the original one.

As usual, feel free to include this in the addon as an extra if you think people could find it useful.

I'll gladly do this! Do you want to make a pull request with these files so you're credited as the original author? I can take care of the addon integration part.

[ilcane87]

@FlailingFog

Nevermind, I got it working on 2.93 (didn't seem to work on 3.0).

I actually haven't even downloaded 3.0 yet, maybe Rigify isn't running correctly on it at this time, or they may have made some changes, I'll check it out.

In addition to the eye controls, it also looks like you made a ton of controls for the face and chest accessible too!

I added a custom shape to anything that could have any kind of deformation effect, be it direct or indirect. In fact the (MCH) layers are entirely made out of bones formerly in layer 10 that have any kind of relation with the deform bones of the face and hair/accessories, but there's probably lots of redundancy there.
All of the bones that have no effect whatsoever end up in the Junk layer of the original armature (they are almost 500, more than half the original bones!), and are deleted on the generated rig when you run the After Each Generate script, which is why Junk is empty there, so if you want to preserve some of them you can just move them to a different layer in the original armature and then hit Generate again (and run the subsequent script).

Does Rigify support three spine bones instead of two? Maybe I didn't dig far enough, but out of the three spine bones (Spine, Chest, Upper chest) I only see access to the Chest and Upper chest. The Hips are parented to the Spine, so that third spine bone is essentially lost.

Three bones is actually the minimum length of the Rigify spine, you're saying the bone called Spine is missing, or did I misunderstand?
You can check out the actual structure by showing layer 29 (the Deform layer):

For the second spine bone Rigify generates an FK control and Tweak control below it, in the Torso (Tweak) layer:

I didn't change any parenting when it comes to the spine bones so I'd be surprised if I skipped anything.

I'll gladly do this! Do you want to make a pull request with these files so you're credited as the original author? I can take care of the addon integration part.

Honestly I'd rather not be credited, this is something I did for my own use and released it on a whim, so while I'm going to try and maintain it I don't plan to spend extra time on issues that wouldn't affect my own work, so I'd prefer it if people didn't come looking for me when you make the next public release.
Consider this an anonymous contribution to give something back for the amazing work that's being done on KKBSP. :)

[ilcane87]

I actually haven't even downloaded 3.0 yet, maybe Rigify isn't running correctly on it at this time, or they may have made some changes, I'll check it out.

Just uploaded the updated version in the OP, now it's compatible with both 2.93 and 3.0, all they changed is adding more granularity to the custom shape transform parameters (a welcome change, actually).
Also, apparently Rigify doesn't like the 'mmd_bone' custom property anymore, so I had to None them all or the generation would fail.
But they did make a nice addition by letting you setup a script to run automatically after the rig generation, which is perfectly suitable for the "After Each Rigify Generate" script:

If you follow the Rigify workflow of making changes on the metarig and then generating from it again and again this could save you one step each time.

[FlailingFog]

For the second spine bone Rigify generates an FK control and Tweak control below it, in the Torso (Tweak) layer:

What I meant was, the Spine_fk bone seems to be doing something similar to the Hips_fk bone. They're both affecting the lower body, but I was trying to get the Spine_fk to affect the upper body. If you check the original armature you can see what I'm talking about. The Spine (along with the Chest and Upper chest) affects the upper body and the Pelvis affects the lower body. The DEF-Spine bone on the Rigify armature looks like it's doing just that, and the chest bone in the Torso section only seems to affect the Chest_fk bone / Upper Chest_fk bone, so am I missing something obvious that allows the Spine_fk bone to affect the upper body?

this is something I did for my own use and released it on a whim,

Okay no problem, I can understand that. Thanks again for sharing these!

[ilcane87]

What I meant was, the Spine_fk bone seems to be doing something similar to the Hips_fk bone. They're both affecting the lower body, but I was trying to get the Spine_fk to affect the upper body. If you check the original armature you can see what I'm talking about. The Spine (along with the Chest and Upper chest) affects the upper body and the Pelvis affects the lower body. The DEF-Spine bone on the Rigify armature looks like it's doing just that, and the chest bone in the Torso section only seems to affect the Chest_fk bone / Upper Chest_fk bone, so am I missing something obvious that allows the Spine_fk bone to affect the upper body?

Ah, I see what you were referring to now.
By default Rigify sets the pivot point of the spine FK controls after the second bone in the chain, which is the "Spine" bone, meaning every bone before the pivot will move the lower part of the body with it; this setting facilitates animation, because it gives you two bones to control the lower body and two to control the upper, but it can be easily changed in a few ways from the metarig.
You can find the options by selecting the first bone of the chain in Pose mode (Rigify options tend to be on the first bone of any structure component), and then checking the bone properties tab:

The "pivot position" option is what I was talking about, so simply setting it to 1 instead of 2 will give you back the original behavior of the Spine bone when manipulating the Spine_fk control; the problem is, you will instead lose the current behavior.
If you feel like the previous behavior is more suitable as default, I can easily set it up like that in the script.
By the way, Rigify also provides the fun option "Custom Pivot Control": when selected, the generated rig will have an additional bone that you can position at will, and when you rotate it the whole spine will rotate using that as its pivot point:

The limbs have that option as well.

[FlailingFog]

I've added a wrapper for your scripts into the plugin.
Click the button in the Extras panel once and it'll do steps 5-6. Every click after that will do steps 7-10

The "pivot position" option is what I was talking about, so simply setting it to 1 instead of 2 will give you back the original behavior of the Spine bone when manipulating the Spine_fk control; the problem is, you will instead lose the current behavior. If you feel like the previous behavior is more suitable as default, I can easily set it up like that in the script.

That's what I was looking for, thanks! I personally prefer it like this, but if position 2 is the default Rigify setting then I think it'd be better to leave it alone.

[ilcane87]

I've added a wrapper for your scripts into the plugin.
Click the button in the Extras panel once and it'll do steps 5-6. Every click after that will do steps 7-10

Well done, it's functional and I'll be able to release updated files as they are without worrying about the plugin side of things.

[ilcane87]

@poisenbery

The scripts don't alter any weights, they are in fact designed to work with the original weights.
The only exception to this rule are the wrist weights, a part of which is divided among the palm bones that didn't exist beforehand, but even those are unchanged.

Can you post comparison pictures between the original weights and the resulting weights from using the scripts, so that I may understand what's going on?

[poisenbery]

@ilcane87 Sorry, I deleted the original post because I noticed that's pretty much exactly what you did.

However, I'm still having some issues, specifically with the legs area. I haven't tested everywhere else yet.
Top is original skeleton, bottom is after rigify. Hips cave in whenever I move the legs out like that.
Its happening on all of my models.
Not sure if 3.0 might be the issue?

More extreme examples:

[ilcane87]

@poisenbery

Thanks for posting this, there are definitely issues with both the "cf_s_leg_R/L" and the "cf_s_siri_R/L" bones when moving the legs, probably caused by differences in the leg bone axes compared to the original.
I'll be working on a fix today.

[poisenbery]

@ilcane87 No problem!

Really appreciate the fact that this even exists. You're doing a lot of people a huge favor by having made this!

[ilcane87]

I double checked a few things and also found erratic behaviors with the waist and shoulder correction bones, but now everything should work like it does in the original armature.
I updated the zip file in the OP, let me know if you find any more issues, otherwise @FlailingFog can update the plugin files.

[poisenbery]

@ilcane87 im having a couple of issues:

I repacked everything, and when I click the button for this, I get an error:.

Python: Traceback (most recent call last):
  File "C:\Users\AREALHUMANPERSON\AppData\Roaming\Blender Foundation\Blender\3.0\scripts\addons\KK-Blender-Porter-Pack-5-merger\extras\rigifywrapper.py", line 68, in execute
    exec(text.as_string())
  File "<string>", line 786
    constraint.invert_z = True
TabError: inconsistent use of tabs and spaces in indentation

location: <unknown location>:-1

That script only has 78 lines of code, so it's actually referencing part of the "Before First Rigify Generate -Public.py" script.

So running that in the script editor, I now get this:

Python:   File "\Before First Rigify Generate - Public.py", line 786
    constraint.invert_z = True
TabError: inconsistent use of tabs and spaces in indentation

location: <unknown location>:-1

I decided to investigate and see if there were spaces used instead of tabs, or vice versa. I found everything to be consistent. I decided to erase all of the tabs and then...put them back in.

Re-ran the script and it worked. Very strange.

I saved the "new" version and retested on a different model, and it worked without issues.

I researched the issue and it said that python only allows a user to use tabs OR spaces for indentation. I'm kind of baffled on why I was getting this error when I found no stray spaces at all in the code. [but i also don't really know anything about coding]

[ilcane87]

@poisenbery

Darn, I'm sorry, that was 100% my fault.
The problematic line was the last one I added to the public file, and I did it from an external editor that doesn't convert tabs to spaces, unlike the Blender editor. And of course I didn't test it after that last addition.
I updated the corrected zip file in the OP, but all I did was the same thing you did so you don't need to download it again.

[ilcane87]

Howdy, I bring you new years gifts (and gifs I guess).
I added a bunch of features to the eyes to facilitate animation and retargeting from different sources, they are all optional and can be activated and tweaked from the Rig UI properties just like Rigify features.
Here's the rundown:

• Limit location (Left Eye Handle, Right Eye Handle, Eyes Handle):
  
  This is self-explanatory, and will be useful to take full advantage of the other features; the default values for the limits aren't going to be perfect since the eyes sizes and shapes can vary greatly, so take a minute to tweak them.

• Eyelids automation (Eyes Handle):
  
  The default values attempt to reproduce the eyelids motion from the Koikatsu in-game editor, but you can tweak all parameters according to your tastes.

• Eye speed correction (Left Eye Handle, Right Eye Handle):
  
  This may not be obvious at first glance, but were you ever annoyed that the pupils tend to not reach their final left/right position at the same time, and require manual tweaking? This property saves you the effort, whenever you move the Eyes Handle bone the speed of each individual pupil will be tweaked so that they smoothly reach their side positions at the same time on both sides (this depends on the limits set as per the first feature on this list).

• Eyeball controls (Eyeballs, Left Eyeball, Right Eyeball):
  
  These are an alternative way to control the eyes with rotation instead of location like you'd usually do with the eye handles; not very useful when animating manually, but this allows for retargeting eye motions from armatures that use that kind of eye controls.
  Here's a retargeting example from a VMD animation:
  
  Depending on your retargeting tool, you may want to retarget either the common eyeballs bone, or the left and right ones, but not both at the same time, or the source motion may apply twice (this is caused by the source armature, not the target one).
  Also, remember that different sources may require different speed settings to match them, so start from an animation that has a full range of eye motions and adjust the speed property accordingly.
  By the way these controls are also affected by the eye speed correction from the previous point.

• Target tracking (Eyeballs):
  
  Once enabled, your character's eyes will always point to the Eyes track target bone, which can be moved freely, and has custom properties to switch its parent between the head, torso, root or none. Enabling eye position limits is necessary to get good results with this.
  It also has an optional nearby target correction feature:
  

And that's it, all custom properties have tooltips to explain them but feel free to ask me if anything is unclear.
Until @FlailingFog updates the plugin, you can download the new scripts from the link in the OP and replace the existing ones.

[miyabi-95]

looks very promising! i'm still an amateur with blender, so i was wondering if this something that can allow me to apply an armature to my character and stay with her when i export her to unity? reason being i have accessories that don't seem to have non-deform bones from the start so if this automatically assigns bones to the entire mesh it may be useful to me.

[ilcane87]

@miyabi-95

Hello, I'm afraid I'm not very knowledgeable when it comes to exporting to game engines, since that's outside the scope of my development, but I can give you a few pointers.
First of all, the effect of these scripts on the character accessories (and hair, since it behaves the same way) is that their deform bones are moved to layer 29, and they get a non-deform copy (with a custom shape) that can be manually controlled in layer 0, with additional control bones in layer 1 if the original bones have at least one chain, and additional controls in layer 2 if the original bones had parents in the original layer 10.

But I don't think that's what matters when you are exporting to Unity, I think accessories are supposed to work even if you're just using the default armature of KKBPP, whether their bones are deform or not. If you're having trouble with the procedure shown in FlailingFog's videos, you can open a new issue and ask for help.
But if what you want is to export the character with the Rigify rig generated with these scripts, then I recommend checking out this playlist that should have all the answers:
https://www.youtube.com/playlist?list=PLdcL5aF8ZcJvCyqWeCBYVGKbQgrQngen3

[BugMonkey42335]

Yo @ilcane87 while testing your rigify scripts I have found an issue, they don't seem to work on male cards.

I've tested both vanilla and modded male cards but they all give this same error.

Traceback (most recent call last):
  File "<string>", line 1645, in <module>
  File "<string>", line 313, in main
  File "<string>", line 173, in arrangeTripleWidgetSet
TypeError: unsupported operand type(s) for +: 'NoneType' and 'NoneType'

Failure

[ilcane87]

@BugMonkey42335

Hello, this is not surprising considering I've never tried a male character before.
I exported the default one from Koikatsu but I can't get the "Import KK Shader and Textures" step to work without errors, is there a trick to it for male characters?

Anyway I threw in a quick fix for that problem, you can get the updated scripts from the zip file uploaded in the first post, but I expect more issues to arise since I couldn't test it properly.

Edit: nevermind, I just hadn't updated the exporter for a while, things seem to be working fine aside from some breast bones not having any effect for obvious reasons.

[koshisan]

Hi!
Thank you for your great work!

I am a bit confused what part of it is actually in the latest release and for what extra scripts are required - especially with Blender 3...

Although I am kind of new to Blender, I followed the guide as closely as possible, but I still end up with things looking like this:

Is this a bug/missing steps in the documentation (it says to parent with empty weights, but there is no such option - just automatic or envelope) or am I just stupid? ;)

[ilcane87]

Hi @koshisan, if you're using the latest release of the addon (5.0.1) you don't need anything else, it already includes the latest version of these Rigify scripts, and it works just fine in Blender 3.0.
The instructions in the first post here are only for using the scripts separately from the addon, so you shouldn't follow them.

Here's what you should do, starting from a new Blender file:

1. Import the model with the CATS plugin, using the big Import Model button
2. In the Blender Porter Pack plugin tab, click Finalize PMX File
3. Separate and rename the hair mesh as 'Hair'
4. Click Import KK Shader and Textures and select the model folder
5. Click Convert and Apply colors to Shader and select the model folder
6. In Object Mode with the armature selected and active, go to the Extras of the plugin tab and click the star icon beside Convert for Rigify
7. Again in Object Mode with the armature selected, click the star icon a second time
8. Parent all objects to the new generated armature called 'rig' with Empty Groups, as shown below:
   

What you should Not do is edit anything else prior to completing the 8 steps, don't even move or resize any object.
It's not clear from your picture because of the color theme, but to me it looks like the character isn't centered at the world origin (X=0, Y=0, Z=0), meaning you may have changed its location, and that would have caused the scripts to misbehave.
That is just a theory though, if even following the 8 steps and doing nothing else you get strange results, try a vanilla Koikatsu character and see if that works.

If it turns out it's a specific character's problem, you can post it here and I'll take a look.

[koshisan]

@ilcane87 Thank you for your quick reply!

I indeed may have moved the model, since I was fiddeling around with it quite a bit before I got the rig to generate. However, on a vanilla model strictly following the steps above I still have the same problem:

There is some dependency between the bones, but when posing the arms for example, the hands seem to be pinned in space and the mesh just gets streched. Maybe I am just grabbing her wrong? Rigged test models, however, seem to work fine for me...

[ilcane87]

@koshisan

Since the rig generated correctly, I think your problem is that you aren't used to manipulating a control rig: for example, I see from the picture that you moved the tweak bones of the arm, which are only for cartoony deformations or subtle shape corrections. The bone you're supposed to move for the arm is the big red square that encompasses the hand, in the Arm.L IK and Arm.R IK layers (you should hide any layer you are not currently using for clarity).

But that just scratches the surface, to become able to take advantage of the features of a Rigify rig, I strongly recommend checking out this video for a quick overview:
https://www.youtube.com/watch?v=vKgH5zXIYmM&list=PLdcL5aF8ZcJv68SSdwxip33M7snakl6Dx&index=14

[koshisan]

@ilcane87 Thanks again! It was a combination of that and parenting to the wrong rig, I guess. I am finally getting somewhere:

However, it is a real pain to find the bone/joint you want and move it around in blender (since support for 3d input devices is limited). You wouldn't happen to have a better software in mind I could export to for better posing? The final result would be printed on a 3d printer, so I don't really care about colors/textures...

[ilcane87]

@koshisan

I'm afraid I'm only familiar with Blender, you could look into retargeting existing animations to your character or exporting to game engines (I linked to useful playlists for those purposes in this thread), but if all you want is poses then that may be more trouble than it's worth, especially for a beginner.
If you'd like to create your own motion capture data, I know of Rokoko (https://www.rokoko.com/) which has tools for that, but the poses will be applied to a different armature, so you'll still need to use retargeting tools to transfer them to your Koikatsu character.

If you are fine with Koikatsu poses, this addon you've been using has a quick way to apply them to your character, as explained at the end of part 4 of the newest tutorial by FlailingFog:
https://www.youtube.com/watch?v=XFt12n7ByBI
Remember though that if you want to use that method, you must not convert your armature to a Rigify rig, so you should skip steps 6, 7 and 8 from the previous list.

But with everything considered, if you want to pose and print the character you exported to Blender, I believe the most efficient way is to just get used to making poses yourself, reproducing them from references and following tutorials; your speed will pick up fast and it won't feel awkward anymore if you spend some time on it.

[koshisan]

@ilcane87 Thank you for your detailed reply, even though it is a bit off topic here...
You are probably right and I just have to bite the bullet with blender. I did shortly look into Daz3d and even Poser, but that adds the problem of importing/exporting, while not making posing significiantly easier...

If anyone else has a similiar problem: Posing one limb at a time and hiding all other layers helps significantly, especially if you work one something with controls all over the place, like hair.

[poisenbery]

This is extremely off topic. If you dont know how to use blender, github issue tracking is NOT the place to post about that. This forum is meant to address bugs with code. It is NOT a beginner help blog. I suggest you post your questions in help forums, not bug trackers.

…

On Wed, Feb 9, 2022, 6:04 PM Jens ***@***.***> wrote: @ilcane87 <https://github.com/ilcane87> Thank you for your detailed reply, even though it is a bit off topic here... You are probably right and I just have to bite the bullet with blender. I did shortly look into Daz3d and even Poser, but that adds the problem of importing/exporting, while not making posing significiantly easier... If anyone else has a similiar problem: Posing one limb at a time and hiding all other layers helps significantly, especially if you work one something with controls all over the place, like hair. — Reply to this email directly, view it on GitHub <#78 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASWU22E5ZIM5CFJOSK24FULU2MMJVANCNFSM5JPT6Z7Q> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[koshisan]

@poisenbery I am well aware! Please see my initial comment: since it was unclear in the documentation how much of the scripts was already in the released version. I admit that the discussion got a bit side-tracked, but possible errors/missing information in the project documentation does belong in the issue tracker - especially since this issue was still open and it was therefore unclear how much of this was actually implemented!

[ilcane87]

@MediaMoots
I double-checked on Koikatsu Studio and it seems that v5 is the accurate one after all.

I posted a new issue for easier tracking: #258

[ilcane87]

Another question for either @MediaMoots or @FlailingFog, probably the latter: in v5 of the plugin (left) the accessories that have bone chains in their armature are imported with those chains intact, while v6 (right) has them disconnected:

I can't really connect them myself since I have no way to discern whether they're supposed to be connected or just parented without connection. Is there a way to get back the original behavior, or can you at least point me to the old code that managed the bone connections?
Rigify armature aside, I think having them connected was useful in the default armature as well.

Edit: it's actually a question for MediaMoots since I tried exporting the same character with the old version of KKBP_Exporter and the bone connections are already there when importing it with CATS.

[FlailingFog]

If you want to do it on the blender side, you should be able to connect the children of the N_Move bone for each ca_slot accessory slot, or leave it alone if it's only a single child.

[ilcane87]

@FlailingFog
Oh that's nice, didn't expect the information to be available in that form.

I should be able to handle this myself then, thanks a lot.

[ilcane87]

@MediaMoots
I spent several hours on it but unfortunately the workaround proposed by @FlailingFog won't cut it, not all bones follow the same pattern (some for example are supposed to be inverted and then connected), and even more importantly that pattern only applies to that type of accessory, while others are entirely variable:

It makes sense that you can't deduce the necessary information from the bone hierarchy alone, considering accessories can be freely rigged and then imported in Koikatsu.

So instead, would it be possible to restore the behavior of the earlier versions of KKBP_Exporter when it comes to the hair/accessory bones?
Unless it's a system limitation, I don't see any benefits to having those bones disconnected even in the default armature, it makes it really tricky to understand how to move them compared to how it used to be, especially when there's many of them overlapped together.

[MediaMoots]

@ilcane87

Hmm, this is V5 raw export though?

Bones are not connected even in V5 Exporter

[ilcane87]

@MediaMoots
You made me realize something, only by importing with the CATS plugin I get the bone connections (for the skirt too), unlike with mmd_tools, and I get them even with v6 exports, so KKBP_Exporter is not the issue after all.

I guess my next question is: is it possible to use a manually imported model with the KKBP addon?
I could with the old version, but the new one only has its own Import button, which as far as I can tell works like mmd_tools and differently from CATS, which is why the bone connections get lost.

Edit: nevermind, I found the relevant function in the CATS repository, apparently the bone connections are just an estimate, pretty much an extension to FlailingFog's criteria, while I previously thought it was information contained in the exported file.

[ilcane87]

Just uploaded the new version in the first post.

Changelog:

• 'Rigified' the new rigged tongue;
• Moved the eyes and rigged tongue bones to their own layers to reduce clutter in the face layers;
• Fixed the skirt bones alignment when converted, and also added the sixth bone (cf_j_sk_##_05) to the chains, just in case;
• Characters without skirt bones are now supported;
• Connected hair/accessory bones in the same way the CATS plugin does, and changed their widgets from 'circle' to 'limb' type, so the connections with their MCH parents are more apparent;
• Enabled a small new Rigify adjustment to the toe bones for Blender 3.2+ (this one: https://www.youtube.com/watch?v=H80AjLWgECY );
• Enabled a small new Rigify feature to the limb bones for Blender 3.3+, which allows you to scale the arms and legs uniformly by scaling the gear control bone at their base;
• Added back a custom shape to the anus bone, since it's not there anymore in the current version of the plugin (I assume it's an oversight);
• Now the script that is run after the Rigify conversion automatically changes parents and armature modifier targets to the generated rig for all mesh objects in the scene that are related to its metarig; it does it without relying on object names so it should be safe even with multiple characters in the same scene (it detects its metarig from a common bone with a random alphanumeric string in its name). In other words, there's no more need to parent objects with empty groups.

@FlailingFog
I noticed you changed the file names and a few lines of code in the Rigify scripts for v6, so I tried to emulate that and it does work when replacing them in the addon zip file, though I didn't change the 'rotate' functions to use bmesh since they seemed to work in their current state, but if it's necessary for some other reason let me know and I'll change them next time.

[FlailingFog]

Awesome! Thanks for these updates. I'll add them in the next release.
Initially I was running your scripts by loading the script into the text editor, but when I tried to attach this to the end of the new automatic import process I ran into context errors. They didn't make much sense, so I tried integrating the scripts into the plugin to get rid of them. I recall this fixed some things but one thing I couldn't fix was the context errors with the rotate function, so I replaced it with a bmesh command and it worked without issue. I'm not sure why the rotate function is working now, but since it is there's no point in using bmesh for it.

[ilcane87]

Hi @FlailingFog, I'm writing here since it's slightly Rigify related, but I noticed the other day that you committed a useful pose library script, and since I had intended to test retargeting from Koikatsu for some time, I finally got down to it using your instructions as a starting point, and I can give you a few pointers to hopefully solve some issues (assuming you haven't already).

First of all you mentioned a problem retargeting bone location with Rokoko, but while I'm not using it myself, from the tutorial video (the dinosaur part at the end) it looks like location is definitely retargeted, so what I think happened to you is something else related to the source armature.
Specifically, depending on how you go about it when importing, you can get two different armatures, you can see the difference below:

I assume the one on the right is the issue you are having, and it's happening because the small 'cf_j_hips' bone, which acts as the root of the armature and is responsible for moving it around, is connected to its parent, which is the huge bone below it:

Since connected bones cannot move away from their parents, even though the location keyframes are there, they have no effect, and the problem will propagate to the target armature, so one way to fix it would be to simply disconnect it through the script before retargeting.

But the thing is, if you just import the complete armature instead (the one on the left), the issue won't be there in the first place, and the additional bones may actually be useful in some cases.
What determines which armature you get is simply this field in the exporter:

The value in the picture is what I think you've been using, since in the instructions you wrote this:
but __don't__ export the mesh along with it. This can be avoided by closing the body 00 tab before exporting
But if instead of that value you choose 'p_cf_body_bone', you won't export the mesh even if you don't close its tab, and you'll get the full armature without the hips bone problem; in fact that's what you did in the video tutorial for the last version.

Also, I wanted to mention a few more bone pairs that should be added to your retargeting list (which was missing one of the shoulders btw):

cf_j_shoulder_R;Right shoulder
cf_j_waist02;cf_j_waist02
cf_j_kokan;cf_j_kokan
cf_j_ana;cf_j_ana
cf_j_siri_L;Left Buttock handle
cf_j_siri_R;Right Buttock handle
cf_d_bust00;Breasts handle
cf_j_bust01_L;Left Breast handle
cf_j_bust01_R;Right Breast handle
cf_j_bust02_L;cf_j_bust02_L
cf_j_bust02_R;cf_j_bust02_R
cf_j_bust03_L;cf_j_bust03_L
cf_j_bust03_R;cf_j_bust03_R
cf_j_bnip02root_L;cf_j_bnip02root_L
cf_j_bnip02root_R;cf_j_bnip02root_R
cf_j_bnip02_L;cf_j_bnip02_L
cf_j_bnip02_R;cf_j_bnip02_R

The left names are the Koikatsu names and the right ones are the Rigify armature names, though many of them are the same.

Note that each bone of that list except the waist and shoulder should have its location and also Scale retargeted as well.
If it's not, you won't get correct results for several animations found in the following folders:

C:\...\Koikatsu\abdata\h\anim
C:\...\Koikatsu\abdata\studio\anime
C:\...\Koikatsu\abdata\adv\motion\controller\adv
(some mods may also contain animations in their zip file)

Now, even with the addon I'm using, Auto Rig Pro, while you have a checkbox for retargeting location in addition to rotation, there is no option to retarget scale; however, with that addon you can jump to a middle-step of the retargeting process where all bone constraints are applied and can be changed manually, so I'm just using a script to add the 'Copy Scale' constraints (with 'Pose Space') at that stage of the process and everything works out, though it may not be as easy with Rokoko.

Lastly, while creating a pose/animation library is really neat, be warned that models with a different height compared to the source armature you are using at the time of retargeting will have erratic behavior, especially with the hips location (their feet might slide or leave the ground), because when retargeting the source armature should always be resized so that the height of its hips bone roughly matches the one of the target armature in order to get accurate results.

[FlailingFog]

I assume the one on the right is the issue you are having, and it's happening because the small 'cf_j_hips' bone, which acts as the root of the armature and is responsible for moving it around, is connected to its parent, which is the huge bone below it:

This is good info, but I’m having a slightly different issue. I’ve been using the body_bone entry for the export option, and the hips bone is acting as the root like it should. I just didn’t realize the mesh wasn't exported with it because of the tooltip on the export button, so you’re right you don’t need to unload the mesh:

The actual issue I’m having is that some poses didn’t completely match up after retargeting (like \abdata\studio\anime\04\gravure_00\isu00)

Initially I thought it was because I couldn’t get the Rigify armature’s fk bones to move, but looking into it again I don’t think that’s the cause of this. None of the main arm/elbow/wrist bones move on the source armature for this particular pose. They also don’t scale in this pose, only the helper bones (like the cf_d_ ones) move and scale. I also tried retargeting the pose after importing with the Koikatsu armature option on the KKBP panel, which is supposed to replicate the in-game armature, but I got the same result so I don't think it has anything to do with the Rigify armature. Something else must be causing the hands to be off, but I still don’t know what yet

Also, I wanted to mention a few more bone pairs that should be added to your retargeting list (which was missing one of the shoulders btw):

Thanks I’ll add these in.

be warned that models with a different height compared to the source armature you are using at the time of retargeting will have erratic behavior

This is true, but I personally use models that are all around the same height

[ilcane87]

Something else must be causing the hands to be off, but I still don’t know what yet

Oh right, I also noticed the arms/hands being a bit offset compared to the source in some motions, so it's not a Rokoko problem either, I assume it's a slight difference in the exported armature of KKBP compared to SB3.
I've been able to solve it manually by either very slightly altering the rest pose orientation of the arms in the source armature before retargeting, or slightly offsetting the entire animation curve for those bones' location in the target armature after retargeting, both of which could be easily done in the script.

I've yet to verify whether the same change solves the issue for all animations though, I'll let you know after I've experimented a bit more.

[MediaMoots]

The actual issue I’m having is that some poses didn’t completely match up after retargeting (like

Yup, @ilcane87 is right, this is caused by the ShapeInfoBase scripts inside KK.

That script has a few inherited scripts like ShapeBodyInfoFemale and ShapeHandInfoFemale.
These use the cf_d and some joint bones to create the character.
This means that all the sliders in the Chara Maker control the values that these scripts use to change the shape of the character.

The script loads curve values from, in the case of the body, chara/oo_base.unity3d/cf_anmShapeBody
SB3U can parse the file. The data is stored as a table, and each row is for a different bone.
Also, the bone names in the file are mapped to other bones in the script, they do not directly correspond to the bones in the rig.

So, when we export from KK, we are exporting a changed mesh and not the original mesh from SB3U.
Even Chika is not the default character.

Maybe one way to fix this would be to somehow get a delta between the bone transforms of the original rig and the exported rig, then apply the delta to the animation after retargeting (or before).

What do you guys think?

[ilcane87]

@MediaMoots
Hi, thanks for the info, it's good to know what happens behind the scenes.
But maybe there's no need to go that deep, as far as I've seen the retargeted animations match reasonably well for anything except the arms, and I think the reason is simple:

If you overlap the SB3U model with the KKBP one, they match almost perfectly except for the arms length, specifically the forearms in KKBP are slightly shorter.
I've tried simply increasing the length of the KKBP forearms (by increasing the size of the forearm bone in pose mode, then saving it as shape key and applying it as base) so that it roughly matches the SB3U one, and this is the difference in retargeting results:

(I had that different model setup for retargeting so I used it instead of Chika, but the body is pretty much the same)
While this matches the SB3U model it still doesn't perfectly match the Koikatsu Studio result, but I've tried to bind the KKBP mesh to the SB3U armature and the result is roughly the same, so either the motion file isn't entirely accurate, or the mesh/armature isn't, but it may not be worth looking into for that small a difference.

Maybe just slightly lengthening the forearms of the KKBP models when they are exported/imported would be enough?

[MediaMoots]

@ilcane87
Good idea, but what if the character has longer forearms, or weird proportions?
This is a good manual fix but I'm not sure how it can be done automatically?

[MediaMoots]

There is another interesting thing, for some animations in KK there are 3 versions L.M.S (Large, Medium, Small) and the game blends between them according to the character's height...

[MediaMoots]

V3.99 can now export the shape values.
Just for reference, but maybe they could be used for something.

[ilcane87]

Good idea, but what if the character has longer forearms, or weird proportions?
This is a good manual fix but I'm not sure how it can be done automatically?

Rather then correcting the forearms proportionally to the upper arms, I was thinking along the lines of lengthening just the forearms by 5% (or whatever ratio we determine) of their original length, so even weirdly proportioned characters will still be weird.
In a script I'm sure it could be done in many ways, one would be by posing the forearm bone location in FK mode and then applying the pose to the mesh, like I did manually here:

There is another interesting thing, for some animations in KK there are 3 versions L.M.S (Large, Medium, Small) and the game blends between them according to the character's height...

Oh so that was the meaning, something to be aware of when deciding what to retarget I guess.

V3.99 can now export the shape values.

Are these related to the sizes?

[ilcane87]

my gif

Oops, I realized I made a mistake, it's not the X location but the X scale that must be increased in the forearm bone, otherwise you end up lengthening the upper arm instead.

Edit: there are actually several bones that are parented to the forearm bone in the standard armature, and they end up being deformed as well, so there's a bit of tampering to do during the operations.
If you decide to take this route I can provide a script for it, it'll be easier than trying to describe every step.

[FlailingFog]

V3.99 can now export the shape values.

Is this the scale of each bone relative to the SB3U armature? And if so, is there a list that correlates each value to a specific bone? I'm also guessing a value of 0.5 is no change?

Oops, I realized I made a mistake, it's not the X location but the X scale that must be increased in the forearm bone, otherwise you end up lengthening the upper arm instead.

I'm not sure on scaling the forearms... It works in this case, but characters with even shorter arms (like ones that are an entire hand length short of the reference armature) will need to be stretched out kind of far.

Maybe one way to fix this would be to somehow get a delta between the bone transforms of the original rig and the exported rig, then apply the delta to the animation after retargeting (or before).

The only poses that would have this issue would probably be ones where the hands line up, so maybe the hand location's distance/location relative to some point on the armature (ex the upper chest) could be found on the imported fbx armature, then the animation could be retargeted, then the hand FK animation could be converted to an IK animation, then the distance of the IK from the upper chest on the reference armature could be increased or decreased to fit to the size of the target model. Something like this where the reference armature arms are the black line bones and the target armature arms are the octahedral bones. It won't work for every pose though, only ones where the hands are supposed to be together...

[ilcane87]

I'm not sure on scaling the forearms... It works in this case, but characters with even shorter arms (like ones that are an entire hand length short of the reference armature) will need to be stretched out kind of far.

That wasn't meant to solve the animation issue for any character, but rather to solve what seems to be a forearms size issue for models imported with KKBP compared to Koikatsu, that's why I said it'd be something like +5% for any character, the fact that it'd fix animations for default sized characters was just a bonus.
Or at least I assumed from what MediaMoots said that the SB3U default proportions are more reliable than KKBP since it takes into account the shape values, but I may have misunderstood.

As far as the animations go, I think it's correct that weirdly proportioned characters will have weird result that require manual fixing, that's what happens in Koikatsu as well as far as I've seen, trying to find an algorithm that makes any animation visually pleasing for any character setup may be an exercise in futility.

[ilcane87]

Hi @FlailingFog, I just uploaded a small update to the scripts in the first post, it adds compatibility with the head mod (#349) and a few other minor tweaks.

[FlailingFog]

I just uploaded a small update to the scripts in the first post, it adds compatibility with the head mod (#349) and a few other minor tweaks.

Thanks, I've added your changes to the master branch.

I'm not sure on scaling the forearms... It works in this case, but characters with even shorter arms (like ones that are an entire hand length short of the reference armature) will need to be stretched out kind of far.

I wound up trying this and I was wrong. It was on a sample size of 1, but even short armed characters seem to look correct with the 5% scale increase. I'll see if I can add that as a toggle to the asset library creation script

[ilcane87]

even short armed characters seem to look correct with the 5% scale increase

Nice, I didn't expect that myself to be honest.
If that means the 5% thing (which is a random number btw) really makes the models more faithful to their appearance in Koikatsu, maybe it should be the way they are exported in the first place, rather than just an after-the-fact tweak for animations?
Meaning, maybe it should be done on the exporter side of things? No idea if that degree of control is even possible though.

[ilcane87]

Hi @FlailingFog, I uploaded an update to the scripts, I fixed a handful of small bugs and console errors, and added a head tracking mechanism similar to the one for the eyes.

There's a new control in the Torso layer, in front of the head, which contains a few properties in the Item panel, one is for limiting the range of rotation of the head, and its angle values:

The default values should be fine for non-weird models.

Then there's the head tracking property which will override the head rotation to follow the target instead:

Like for the eyes target, the parent of the head target can be changed between head, torso, root, or none.

Some use cases for this could be to track the camera with both head and eyes, or easily changing the facing direction during animations with lots of body movement, but be warned that some flipping will occur if the character rotates too far out of tracking range, such as when spinning around.

[FlailingFog]

Thanks, I'll add in your updates. Head tracking is a pretty sweet feature.
I also realized I forgot to do the 5% scaling during the retargeting of animations... I'll add that in as well.