UE5: MindMaker (DL single node) gives (KeyError: '_init_setup_model') on play

[OkMast]

After clicking create project for UE5 (previously it would allow project creation for other versions, and the free plugin was also available for other versions, but now it shows support for only UE5 on the marketplace), it shows the following attached error.

However, when I download and run the example project (cartpole), it runs as it should but on UE4.25, as Ai plugin is manually present in the plugins folder.
I have tried one too many things on different systems, but now I am more confused than ever if the error is environment on my system related, related to the changes in the 'supported versions' of the Mindmaker, or something else.

[krumiaa]

This key error is caused by underscores in all the blueprint custom parameters for each DRL algorithm in the UE5 version of mindmaker. They must be removed and replaced with spaces in blueprints. For some reason the new release of the SocketIO json library for UE5 is not processing underscores. I believe I replaced all the underscores in the experimental UE5 release which can be downloaded along with the UE4 version here. https://unrealengine.com/marketplace/en-US/product/neurostudio-self-learning-ai/questions
Please let me know if this fixes the issue.

[OkMast]

I downloaded the experimental UE5 release project you have linked above (epic games); that was the one I had used the last time around too. The issue is still the same, and I have attached a video for your reference so we are on the same page regarding the problem.

errorVideo.mp4

I am not sure exactly where I am supposed to fix the underscore problem in the blueprint custom parameters for each DRL algorithm. Maybe if you could go over it step by step in a video or something?

[krumiaa]

did you make sure to use all of the new portions of the plugin for ue5, including the new mindmaker.exe? if you open cartpole ue5 example, and then go to A2C custom structure in the content directory, you will notice all of the underscores have been removed from the parameters as compared with previous versions. This should solve the key error. Perhapse I forgot to remove some underscores though, what specific algorithm are you attempting to train with?

…

On Thu, May 26, 2022, 8:13 AM OkMast ***@***.***> wrote: I downloaded the experimental UE5 release project you have linked above (epic games); that was the one I had used the last time around too. The issue is still the same, and I have attached a video for your reference so we are on the same page regarding the problem. https://user-images.githubusercontent.com/94696318/170494265-c8d08228-7829-43a9-8c99-89f5928826b9.mp4 I am not sure exactly where I am supposed to fix the underscore problem in the blueprint custom parameters for each DRL algorithm. Maybe if you could go over it step by step in a video or something? — Reply to this email directly, view it on GitHub <#20 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABI6N2MZ4CZBIL4XRLHN7NTVL52H7ANCNFSM5WZ3MX6Q> . You are receiving this because you commented.Message ID: ***@***.***>

[OkMast]

As of now, I am just trying to run the cartpole example, which uses PPO2 algorithm.
I have removed all underscores from all structures, and fair enough the error for "_init_setup_model" went away. However, it was replaced by the following error in the activation function parameter:

Note that I have removed ALL the underscores in all the structures.

[krumiaa]

I will try running with same setup as you and see if I can replicate the issue. Are any of the other algorithms working when you change it from PPO2 to something else? I have tried a few and they all seemed to work, maybe I zip and share the project files with you to make sure we are both using identical setup, though if you downloaded the UE marketplace version, that should already be the case. What release of UE5 are you using?

…

On Tue, May 31, 2022 at 4:54 AM OkMast ***@***.***> wrote: As of now, I am just trying to run the cartpole example, which uses PPO2 algorithm. I have removed all underscores from all structures, and fair enough the error for "_init_setup_model" went away. However, it was replaced by the following error in the activation function parameter: [image: error_actfunc] <https://user-images.githubusercontent.com/94696318/171134081-f455df65-b10b-4f4f-a012-1cd3391440ef.PNG> Note that I have removed ALL the underscores in all the structures. — Reply to this email directly, view it on GitHub <#20 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABI6N2L4YHGSEOQQR4FSXU3VMXHUZANCNFSM5WZ3MX6Q> . You are receiving this because you commented.Message ID: ***@***.***>

[krumiaa]

perhapse the mindmaker.exe in your content directory is still the older version, this would happen if you used the new ue5 plugin but copied over your existing content folder that included the older mindmaker.exe file. in addition to removing underscores, some of the parameter names got changed slightly in the new mindmaker version so it could be looking for those and throwing this error. Did you try using the marketplace ue5 mindmaker DRL example cart pole exactly as is without any changes, also removing the old version of the UE5 plugin if you had one?

…

On Tue, May 31, 2022 at 4:54 AM OkMast ***@***.***> wrote: As of now, I am just trying to run the cartpole example, which uses PPO2 algorithm. I have removed all underscores from all structures, and fair enough the error for "_init_setup_model" went away. However, it was replaced by the following error in the activation function parameter: [image: error_actfunc] <https://user-images.githubusercontent.com/94696318/171134081-f455df65-b10b-4f4f-a012-1cd3391440ef.PNG> Note that I have removed ALL the underscores in all the structures. — Reply to this email directly, view it on GitHub <#20 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABI6N2L4YHGSEOQQR4FSXU3VMXHUZANCNFSM5WZ3MX6Q> . You are receiving this because you commented.Message ID: ***@***.***>

[An-nym-us]

Any fix yet, I'm still having the same problems, as the person above, removed all underscores, and followed all other directions listed.

[OkMast]

Are any of the other algorithms working when you change it from PPO2 to something else? I have tried a few and they all seemed to work, maybe I zip and share the project files with you to make sure we are both using identical setup, though if you downloaded the UE marketplace version, that should already be the case. What release of UE5 are you using?

I have tried to run different algorithms, none of them worked. I am using 5.0.1 version of UE5. (Python 3.7.0 environment)

perhapse the mindmaker.exe in your content directory is still the older version

This is not possible since I have used another, completely new CPU and environment which had no prior installations of this project or even UE. I installed everything from scratch and then followed your directions exactly.
I also did everything else you have asked about in your second text above.
Maybe if you zip and share your project like you mentioned above on my email: shizaaman11@gmail.com, and I try and run it on my end

[krumiaa]

will do, sending you a zip to your email. curious to get to the bottom of this

[krumiaa]

UPDATE: Just released a version of this for UE 5.0.2 which can be downloaded from the marketplace. https://www.unrealengine.com/marketplace/en-US/product/neurostudio-self-learning-ai
make sure you are using the updated example files with DRL content, this has _ removed

I'm thinking this may have fixed this issue. Still wasn’t able to replicate but if it now works, please let me know by updating this issue.

[Malados]

always the same error on my side

[oshirokn]

Same for me too

[OkMast]

error in the zip file you emailed:

The updated version on epic games gives the same error as the 'act_func' error above- that is, after I yet again personally removed all the underscores (that were again not removed for this new version) in all the structs. Tried all the other things you asked me to try too.

[krumiaa]

Was finally able to replicate the issue and hopefully the fix is now up and working. Please try updating again from the marketplace. The issue seems to be that the project files on the marketplace included both the older UE 4.27 version of mindmaker and a version for UE 5, but when one created a new project from UE launcher rather than downloading the zipped project files with both folders, the launcher was defaulting to the UE 4.27 version of mindmaker, and running it inside UE 5. I thought it would automatically use the appropriate version but apparently not. Apologies for that, Im still getting to know the marketplace system and it seems things may have changed with the release of UE 5.

[krumiaa]

circling back to make sure this issue was fixed by the latest update on the marketplace, can anyone comment?

[oshirokn]

Marketplace version works fine for me. Thanks for the update!

[krumiaa]

Great, closing this for now in that case

[Malados]

Hey, i try to use your update but i have a new error. Maybe my fault, i'm not in my main pc. I thought it was python problem so i uninstall / install and try some tricks but not result. Any idea ?

[OkMast]

Yes it works fine now, thank you Aaron! :)

Maybe my fault, i'm not in my main pc.

Yeah that happens with me too (the mindmaker.exe file closing immediately) on one of my computers, but it works fine on another. This is probably something related to our setup, or the installed dependencies. I've been trying to fix that issue for a while now with the previous functional version of mindmaker, even reinstalled my entire python and visual studio setup; tell me if you manage to fix it in the future. In the meantime, use another computer.

[An-nym-us]

I'm having the same error as Malados. What version of python are you using for those that got it to work. Im using python 3.9.

[krumiaa]

This happened for me before and I think its related to Microsoft OpenMPI, since this is required for DDPG. Can download MPI here

https://www.microsoft.com/en-us/download/details.aspx?id=57467

and for python

conda install mpi4py

Please confirm if this is the fix, will publish an update

[EliasNacef]

Hey ! Just to let you know i had the same problem and installing mpi4py and MPI solved the mindmaker.exe crash

[An-nym-us]

thank, you, was able to find the link to the MPI. Was able to get mindmaker up and running. Some of the Reinfomrent learning algorithms still do not function while others do. I'm assuming this will get fixed in later updates.

[Malados]

This happened for me before and I think its related to Microsoft OpenMPI, since this is required for DDPG. Can download MPI here

https://www.microsoft.com/en-us/download/details.aspx?id=57467

and for python

conda install mpi4py

Please confirm if this is the fix, will publish an update

works for me, thx you !

[Devanshu-singh-VR]

I am still have the problem, the mindmaker.exe is still crashing after installing all the stuff.
How "conda install mpi4py" is working? because UE5 uses its own python environment.

Here are the steps I did to run the UE5:

1. I installed the mindmaker DLR plugin.
2. I created a new project out of it.
3. I downloaded the mindmaker learning engine.
   this one -> (MindMaker DRL Engine Single Node (Stable Baselines: A2C, DQN, PPO, ACER, TD3, SAC) - Win64)
4. then I copy paste all the files to mindmaker folder as written in the mindmaker github.
5. I installed the Microsoft OpenMPI.
6. and mpi4py

if there anything still left to do? Kindly let me know

[krumiaa]

You shouldnt need OpenMPI for the UE 5.1 version of the project because its stablebaselines 3 which is using pytorch. Im not sure why its crashing, try this example project and tell me if it runs https://drive.google.com/file/d/12UOj5qe9mVtgWccph_P2MCUnwg04JP_m/view?usp=share_link

…

On Mon, Feb 20, 2023 at 7:58 AM Devanshu Singh ***@***.***> wrote: I am still have the problem, the mindmaker.exe is still crashing after installing all the stuff. How "conda install mpi4py" is working? because UE5 uses its own python environment. Here are the steps I did to run the UE5: 1. I installed the mindmaker DLR plugin. 2. I created a new project out of it. 3. I downloaded the mindmaker learning engine. this one -> (MindMaker DRL Engine Single Node (Stable Baselines: A2C, DQN, PPO, ACER, TD3, SAC) - Win64) 4. then I copy paste all the files to mindmaker folder as written in the mindmaker github. 5. I installed the Microsoft OpenMPI. 6. and mpi4py if there anything still left to do? Kindly let me know — Reply to this email directly, view it on GitHub <#20 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABI6N2OTX55FL7MGKMHAMTDWYOBCVANCNFSM5WZ3MX6Q> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>

[Devanshu-singh-VR]

thanks @krumiaa, I tried the unreal engine 5.1 and its working fine.

The issue was coming in the unreal engine 5.0. But I think now I will use the UE5.1 rather than UE5.0.
Thanks once again.

[sadadsads]

thanks @krumiaa, I tried the unreal engine 5.1 and its working fine.

The issue was coming in the unreal engine 5.0. But I think now I will use the UE5.1 rather than UE5.0. Thanks once again.

can you share me with the mindmaker ai Plugin for ue 5.1 ，i can only find 5.0 ，5.2，5.3 online