Building MM and Debugging ModDBInterop #37
Comments
|
I use vs community 2019, the github action uses this as well. Not surprised that 2022 has Reference errors but I always struggle to fix them. If you follow the github action you should be able to get it to build https://github.com/jmquigs/ModelMod/blob/master/.github/workflows/dotnet-desktop.yml You can ignore the TPLib for now, that lib isn't needed (its just binaries of d3dx and similar files that I didn't want to host in a public repo). |
|
Okay I figured out the problem. I had to install the required .NET targetting packs. But since the .NET 4.0, 4.5 etc. are out of support for VS 2022, I had to add VS2019 to the installer and add only the targetting pack components. The app is building and running correctly now. I'm going to try to implement proper tangent and binormal calculation using the reference site in the script comment to see if this fixes the light deflection issues. |
|
I realised the same thing you did in #10, i.e. that we can export the tangent space directly from blender instead of calculating it ourselves. I have written my own extraction method which ensures every loop normal/tangent/binormal has a unique vertex index, and are then also sorted by those same indices. This should ensure each vertex is assigned the correct normal, tangent and binormal. Now we have tn and bn lines in addition to vn lines, which should be able to be imported by MM. This is something I will work on another day. Since I have edited the export script, this branch will now conflict with the other one I made a pull request for. Should I merge the two branches when I have finished the edits to the MM code? |
|
That's cool! I'm interested in seeing if those blender-generated vectors work better. I was disappointed with the results I got from the DXMesh library, but I couldn't be sure if it was just because I had a "dirty mesh" with duplicate verts or faces and something like that. You can go ahead and merge it into your existing PR if that is the easiest. It could just be a new feature of moving to the more recent version of blender. One thing I have strived for is to make sure the mmobj format is stable - for instance you don't need to reexport mmobj files when upgrading to a new version of MM. So any changes to the structure of the mmobj need corresponding changes in the F# loader so that it is able to load both old and new versions. This would similarly apply for this change. if the bn and tn lines are available, then fine use that data, otherwise the F# code |
|
@jmquigs You're correct that the new scripts are not compatible with Blender 2.7x. I think it may be possible to have install.py recognise the Blender version you're using and install the appropriate scripts, but it's a question of if it is worth trying to implement this. The code changes I made to the normals writing protocol should be cross compatible, they would just need copying over. How you want to proceed here is up to you. I feel the majority of potential users will be on the latest Blender version, so maybe making the 2.7x scripts optional is best? Though, without modifying install.py, they would probably have to be installed manually. That, or they could be contained in their own individual nested zip files. I am just making some changes to the F# code. I have added tangent and binormal resize arrays and log lines to Also, when I build the app and try to launch GW2 I get a popup with the error EDIT: After adding the bin with the x64 DX dlls to the repo, the game now launches, but MM does not load correctly. The log outputs this: |
|
I'm trying to figure out what components are missing from the build version that prevents it from working like the release. The setup instructions in the dev guide appear to be outdated as there is no longer a solution named ModelMod in the project. Unfortunately Im struggling to troubleshoot this myself due to my inexperience with VS. |
|
@Frosferes2 You can get a "managed code version mismatch" if the F# code does not match the version the rust code expects. Both are 3 in the committed code. Since I presume you are not rebuilding the rust stuff, I would expect that to be "3". You can try printing the versions that are being compared in the F# code before it throws the exception to see what rust is passing in. Its important for these versions to to be the same, because if they aren't the rust code won't have the proper layout of the F# interop interfaces, which can lead to mysterious crashes. |
|
@jmquigs Initialising logging early in Interop and printing the native versions for the Rust and F# code shows a discrepancy as you said: the Rust code is version 2 and the F# is 3. I am using the d3d11.dll from the MM Beta 1.2.0.66, which I assume the native version is read from when the CLR is initialised. The compile date of the rust code reported by the log file aligns with when this release was published: Removing the fail condition allows the CLR to initialise regardless, and the MM UI loads in game. But I assume this is not best practice, since you said version differences can lead to unexpected errors. Do you know what the differences are between native Rust versions 2 and 3? If I were to build the version 3 Rust binaries myself, how would I go about doing it? |
|
@jmquigs I figured out how to build the Rust binaries and moved and renamed the newly generated hook_core.dll to the MM bin. The build is now fully working and I have been able to start debugging. Exciting news! I've discovered the source of the lighting bug in the F# code and it is not related to the tangents after all. When WriteElement is called, both the vertex index and vertex normal index are extracted from the element index record v. For some reason I am not sure of, the vertex normal indices are out of sync with the actual vertex indices in this record. I wrote both these values to the log and found that the vertex normal index values would not exceed around 300 and would often repeat, despite the mesh containing several thousand vertices. This would explain why all the normals face a similar direction: they are all copies of only the first few vertex normals. I changed both modmNormal and modmBinormalTangent to call the normal and tangent vectors respectively using the vertex index instead of the normal index and this seems to almost completely fix the lighting. There is still some weirdness, but this might be due to my texture or it could be related to the axis basis of the vectors as I mentioned in a past comment. I say this because it seems to me like the lighting is correct, but oriented in the wrong direction. I will have to test this further. This method will cause an error if the length of the vertex list does not exactly match the normals list, and I still want to implement the pre-calculated tangent space from Blender. It will take some more F# learning to implement this intelligently though. |
|
@jmquigs Sorry for the incessant comments. I just wanted to let you know that I believe I've managed to fix the issues with blend weight mode. Changing line 466 from UNorm to UInt allows the mode to load. However, after doing this I got the same bug as I mentioned in this comment from nearly 2 years ago. After some investigation, it turns out the problem was with writeMeshBW. Occasionally when adding a new vertex to the mesh, its weights will be rounded in the byte array such that the sum of the array is not 255. When this happens, the vertex does not have a valid binding and stretches off in world space. To correct this, we can add the difference between 255 and the array sum to the highest (first) weight value. The sum of a byte array is also a byte, and so will overflow if over 255. To solve this we can initialize the weight array as ints, and then convert to byte when passing to the binary writer. This seems to have resolved the issue as none of my remodels have has this stretching problem since implementing this. I would like to commit the changes I've made to a new PR soon. The changes to the managed code can exist independently of the Python stuff, as I'm not sure how you would like to handle the Blender scripts. I'm still not too familiar with Git, so I don't want to flood the page with tiny PRs. When you can, please let me know how you want to go about this. |
|
@jmquigs Hi again. I just discovered that the position transformations that MM uses to translate between the coordinate systems for OGL and DX is incorrect. At present, going from Blender to DX gives MM the transformations: Blender's coordinate system is right handed, whereas DX is left handed. There is therefore no way to map one onto the other without one axis being mirrored. If this were applied to the current transformation list, it would look like: This also means that the snapshot transformations are incorrect. Which expedites the need to have working snapshotting for DX11. Problem is, I really don't know what to do to fix this. So I would need some leads from you if I were to attempt it. |
|
Hey @Frosferes2 sorry I've been not responding, my life over the past few weeks has been much more complicated than I'd like. I've actually known about that inverted X for quite a while now but never fixed it. I think what be involved is for the snapshot code to come up with a new transform convention name and export the verts using that fixed convention. For backwards compatibility we'd still want to be able to load the old transform though (but they could continue to use the unfixed axis). Additional on a re-export the python code could optionally fix the export to use the new transform. I'd least that is how it would look in a perfect world. The DX11 snapshotting code is actually sort of working as far as I recall though its been months since I looked at it. The issue is that with DX11 the game needs to be started with a special flag so that snapshottable data is copied into system memory. This is enabled with a "precopy" flag as seen here: https://github.com/jmquigs/ModelMod/blob/master/Native/hook_core/src/hook_device_d3d11.rs#L697 . Its ok to run the game in snapshot mode all the time, as it garbage collects data after a period of time, its just a bit of a performance drain, especially when the game is loading a lot of assets. I had started to modify the launcher so that it could be started in either precopy or "playback" mode but never finished that work. I got stuck on what kind of design would be optimal to pass the flag to the rust code, a silly thing in retrospect. Its still pretty unlikely that I'll be able to get to it soon. But if you wanted to play around with snapshotting yourself you'd need to start there. You may need to build the rust code as I'm not sure if the beta version I made had the latest changes to enable that. |
|
Hi @jmquigs don't worry about not responding, life takes priority after all. I hope things settle down for you soon. I have added a new 'flip' keyword to the 3D transform parser, similar to what is used in the 2D UV transform. This instructs the transformer to take the negative value of whatever vector element is being flipped. After doing this, changing an existing Blender mesh's transform strings from On that note, I have been able to get the lighting on the imported mesh accurate enough that I can't tell if any further discrepancies are because of the transform basis or something to do with the mesh export step. This is something I will look into when snapshotting is going. I have been able to build the Rust code with Cargo locally by fudging the Git hash. I'll have a go at setting this precopy flag manually and see what happens. I have aimed to ensure the changes I made to the F# code are backwards compatible with previous mods. I take it you're not going to be available to review them at the moment given IRL commitments, right? Perhaps I will push my changes onto my MM branch and submit a pull request once everything is done. We can discuss individual code changes is said PR later, hopefully when you have more time. |
|
@jmquigs indeed you were right. Forcing Some initial observations about the snapshotting:
|
|
Hey @Frosferes2 , glad you got it sort of working I always build in release build. If it crashes in debug, well that may or may not be a problem. Possibly the rust code is panicking, which would crash the whole game. I never use debug because its too slow. For iterative dev builds I use this script under git bash to copy the dll and run the game: On other issues:
Of course feel free to fork and branch so that you can commit your work. I won't be offended, in fact I'm happy someone is finally working on it other than me. The project certainly has some limitations but its still good for some use cases, I'd like to see it continue to work with gw2 at least. |
|
@jmquigs I've been rather enjoying working on this project. It's been a great way to learn about lower level languages and the IDE/Compiler/Git workflow. I'm just wondering what the theoretical limits of MM are. Maybe if I look into the Rust side of the code more I will learn about how the renderer works and what is/isn't possible Face normal direction is based on the order of vertices in the triangle. When you invert an axis in a mesh, you change its handedness. To translate a triangle made for a left handed coordinate basis to a right handed one, you also have to reverse the order of vertices in the triangle, otherwise the triangle will be facing the wrong direction. I've already implemented this, so don't worry too much about it. I will look at the mesh and texture stuff later. Out of curiosity, when you say some geometry is inaccessible by MM, does that mean inaccessible with the current implementation or is it just completely impossible? If I modify a mesh which has flowing cloth on part of it, like the Dark Tyrant outfit, will it lose this component or is it rendered seperately? I assume the _VB file generated by the snapshot is the DX11 equivelant of _VBDecl? Are you aware of any documentation of this sort of file and what needs to be done to port the mod loader over? Knowing where the launcher checks for this file when creating a mod would also be useful. |
|
@jmquigs Hi again, hope you enjoyed the holiday season! I was poking around the screenshotted textures to try and figure out what the issues with them were. Turns out that most of the textures aren't actually broken, they were simply in a format that most image editors couldn't parse. Visual Studio has its own image editor that is capable of handling the more arcane DDS formats and was able to open almost all of the textures that were extracted. These can then be saved as some lossless format like png for external editing. Inspecting the textures in VS shows that Texture0 is the combined diffuse/transparency maps, and Texture1 is the normal map with no alpha channel. Something I noticed with the normal maps was that they were an odd color compared to the light blue neutral that is typical of this kind of map: Most other maps I've seen appear to be misc color maps or some other kind of data I'm unsure of in various formats. The one exception being Texture2, which sometimes appears when snapshotting certain objects. This texture is significantly larger than others attached to the same object (16MB vs 1MB) and could not be opened by the VS image editor. Inspecting this texture with the VS hex editor shows that the entire file is filled with empty data (which translates to blank white pixels), except for a couple of lines near the beginning which state the image format. The file also ends on an incomplete line, which seems to indicate that the generation process was abruptly cut off. Perhaps like you say, d3dx is unable to interpret the data in this texture slot and just generates junk. If I were to guess what this map is supposed to be, it could be metallic or specular data, as it seems to generate when snapshotting armor with reflective components, but I can't say for sure. |
|
Hi @Frosferes2 , cool that most of the textures appear to be valid at least in visual studio. That's a good sign that even the old d3dx is snapping them right, it just doesn't know how to interpret them. Perhaps there are some transforms that could be applied on export (and then again on load) to change them in and out of a format that blender and other tools can understand. Unfortunately I'm very weak on texture stuff. Some people working on the reshade project might be able to help more, since I presume they run into this stuff all the time with what they do. I agree that green normal map is very unusual I've never seen one like that before. I agree it sound like the 16MB texture does have an format that is unrecognized. It might be a previous backbuffer itself (which could be used for some reflection effects), or, if the format is totally unknown even the size could be invalid. While I' here let me answer some questions from your earlier post:
Usually the data is present, but because its fully transformed into world space, including its exact animation frame, it can't really be modded, because this violates an assumption MM makes, which is that the base mesh is in the unanimated rest pose. So a mod can be swapped out with that and the shader will animate with no problems. Its a bit hard to explain but, if you tried to mod a particular frame of the cloth, what you'd likely find is on subsequent frames, the delta vectors between vert and mod positions would get so far out of whack that you'd have a polygon dance on the screen. The only way around this would be to copy that data back to the CPU on each and every frame, do a computation to figure out where the mesh is relative to some current mesh, apply a mod, and then copy it back to GPU. Which would be very difficult and slow in a game like GW2. Years ago I did experiment with this for GW1 (which makes much heavier use of CPU-animation), but that code is gone now and it never worked great to begin with.
The only docs are the rust code and the snapshot code in F# sadly. But I believe its just a copy of the raw data structures that I read out of d3d11. But yes, the decl and layout contain the same type of information, basically the offset into each vertex structure of where things like positions, normal, textures can be found. One irritating thing about DX11 in GW2 is that it periodically just randomly change these (usually after a level swap) in a running game for meshes that were already loaded, it changes the format around (even though its still functionally identical) and MM doesn't notice. This can cause mods to stop appearing that were once working in that session. A full reload sometimes fixes this as it can force MM to update its cached formats. |
|
@jmquigs I agree it would be great to be able to talk with some of the Reshade guys about texture/lighting topics. There is still some weirdness related to the vertex normals that has an impact on the lighting. Even after translating the normals corectly into the DirectX coordinate basis, they still require some additional rotations to appear correctly. I wonder if the coordinate system used by game's rendering shader is somehow offset from the basis used by the world space. If this is the case, it may necessitate additional xform instructions in the mmobj file. The issue of the flipped normal maps is also something I would like to explore, along with some other strange texture types I have discovered in snapshotting. It's difficult to know what to do with the VB files. All the VBDecl file I have seen have been the same across all my old mods and are very short. By contrast, the VB files are very variable in length, which is dependent on the size of the extracted mesh. I assume this data is raw binary and can't be opened in some other app? I suppose if I want to work on this I will have to dive more into the snapshotting code to understand the data structure more. Given that MM is now mostly doing what I want it to, I haven't had as much personal motivation to investigate this. I have been able to create new mods by substituting the missing VBDecl file with one from my previous mods, but this is obviously a jerryrig solution. One other thing is that I have developed a few Blender scripts to help with creating MM mods. These aren't part of a full addon, but could be incorporated into one if desired. Let me know if you find these interesting: I have one which generates an armature for an imported mesh by estimating bone positions based on the weighted mean positions of the vertices in each group. This allows for easier weight painting, along with the other utilities of having a vertex deforming armature to work with. The bones are unstructured, as I can think of no way to estimate what the bone heirarchies should look like from the available data, but this script automates the most important and tedious parts. Creating mods for different armor sets can be challenging because various faces are culled from the body mesh whenever a piece is equipped, and the resulting body mod has variable vertex group index ordering, making it difficult to use the same Blender mesh for both mods. This is something which I think you mentioned as being a problem. I have developed a name profile system which stores different possible names for each bone in an armature as custom property strings, which can be freely swapped between using a script. When the armature a mesh is skinned to changes the name of its bones, the names of the vertex deforming groups in the mesh are updated automatically, which is very handy. A new armature can be generated from an imported reference mesh which has been culled by armor. I made another script which can match the bones in this reference armature to the bones in the armature used by the main body based on the relative distances between the reference and target bones. It then adds the names of these reference bones to the target bones as a name profile. This means the same body can be used for multiple mods by swapping between name profiles. The necessary faces can then be culled using a mask modifier. I was thinking of a similar script to use the name profile system for bone symmetrisation as well. |
I've spent some time familiarising myself with VS and F#. If I can get MM to build successfully, I can move on to debugging the issues I brought up earlier.
I'm using VS Community 2022. When I attempt to open and build the MMDotNet solution, I get notifications that I'm missing specific versions of the .NETFramework reference assemblies (4.0, 4.5 etc.). But when I try to install these, the installer says they're already part of my OS and won't proceed. The reference manager says the targetted assemblies were not found.
Is there a way to solve the missing references using Paket or the Github build options you mentioned before? Apologies for the elementary questions. Once this is solved there shouldn't be any more barriers to debugging.
On a side note, I've made a Blender script to generate a akeleton for imported MM meshes. Bone positions are approximated using the weighted mean position of all the vertices in a given vertex group, where mean weights come from the vertex weights. If you are interested, I can share it with you, though I'm not really sure how to do so via Github. I also want to make a partner script that can use this skeleton to symmetrize the vertex groups for more convenient weight painting.
The text was updated successfully, but these errors were encountered: