Run unity plugin on cpu ?

[FrenchDoge]

I installed with success openpose last version, and tried to run unity plugin. When I open the scene and click on "play", it crashes few seconds later.

I think it's because my computer have components not strong enough. When I installed openpose, i had to make it run in CPU only mode, my graphic card was not powerful enough. I tried to find a way to tell unity to run on CPU but it seems to run on GPU no matter what I do.

This screen capture might help you understand better my problem :
https://imgur.com/a/0LyN7lE
this photo has been taken just before the crash, it's always the same messages. (I hope the link works)

I have checked the FAQ, which tells me (if I'm right) to modify the file "OpenPoseUserScript", situated in the folder :
openpose_unity_plugin-master\OpenPosePlugin\Assets\OpenPose\Examples\Scripts\

For example, it tells me to deactivate hands and face, and to reduce net resolution. I did it modifying these lines in the code : (tell me if I did something wrong)

32 :

        public bool
            handEnabled = false,
            faceEnabled = false;
        public Vector2Int
            netResolution = new Vector2Int(-1, 16),
            handResolution = new Vector2Int(16, 16),
            faceResolution = new Vector2Int(16, 16);

90 :

        private void UserConfigureOpenPose(){
            OPWrapper.OPConfigurePose(
                /* poseMode */ PoseMode.Enabled, /* netInputSize */ netResolution, /* outputSize */ null,
                /* keypointScaleMode */ ScaleMode.InputResolution,
                /* gpuNumber */ -1, /* gpuNumberStart */ 0, /* scalesNumber */ 1, /* scaleGap */ 0.3f,
                /* renderMode */ RenderMode.Cpu, /* poseModel */ PoseModel.BODY_25,
                /* blendOriginalFrame */ true, /* alphaKeypoint */ 0.6f, /* alphaHeatMap */ 0.7f,
                /* defaultPartToRender */ 0, /* modelFolder */ null,
                /* heatMapTypes */ HeatMapType.None, /* heatMapScaleMode */ ScaleMode.UnsignedChar,
                /* addPartCandidates */ false, /* renderThreshold */ renderThreshold, /* numberPeopleMax */ maxPeople,
                /* maximizePositives */ false, /* fpsMax fps_max */ -1.0,
                /* protoTxtPath */ "", /* caffeModelPath */ "", /* upsamplingRatio */ 0f);

            OPWrapper.OPConfigureHand(
                /* enable */ handEnabled, /* detector */ Detector.Body, /* netInputSize */ handResolution,
                /* scalesNumber */ 1, /* scaleRange */ 0.4f, /* renderMode */ RenderMode.None,
                /* alphaKeypoint */ 0.6f, /* alphaHeatMap */ 0.7f, /* renderThreshold */ 0.2f);

            OPWrapper.OPConfigureFace(
                /* enable */ faceEnabled, /* detector */ Detector.Body, 
                /* netInputSize */ faceResolution, /* renderMode */ RenderMode.None,
                /* alphaKeypoint */ 0.6f, /* alphaHeatMap */ 0.7f, /* renderThreshold */ 0.4f);

However, unity keeps crashing. I thought changing renderMode.Gpu to RenderMode.Cpu would tell unity to run on CPU. I don't know what to do more.
I've checked every topic or issue or FAQ I could find that seemed to have a link with my problem. Please don't just answer by giving me a link and be a bit more specific, I'm afraid I need more specific help, I'm just an intern trying to do my best openpose is quite new to me ^^

thanks for your help

Issue Summary

Unity does not seem to run using the parameters I have chosen

Executed Command (if any)

Play button pressed on demo scene from unity

OpenPose Output (if any)

Unity output : console, here is a screen capture taken before the crash.
https://imgur.com/a/0LyN7lE

Errors (if any)

None: unity crashes before.

Type of Issue

You might select multiple topics, delete the rest:

• Execution error
  unity crashes.

Your System Configuration

1. Whole console output Unity output : console, here is a screen capture taken before the crash.
   https://imgur.com/a/0LyN7lE

2. OpenPose version: OpenPose Unity Plugin default

3. General configuration:

   • Operating system : Windows 10, 64-bit
   • Release or Debug mode : release
   • Compiler : VS 2017 community

4. Non-default settings:

   • Any custom OpenPose configuration with respect to the default version? : no

5. 3rd-party software:

   • Caffe version: Default from OpenPose
   • OpenCV version: OpenPose default

6. If GPU mode issue: I'm using the CPU_ONLY option, but just in case :

   • CUDA version : 10
   • cuDNN version: 7.5
   • GPU model : Nvidia Geforce GTX 760

7. If CPU-only mode issue:

   • CPU brand & model: Intel Core i7-4700K CPU @ 3.4GHz 3.4GHz
   • Total RAM memory available: 16GB

8. If Windows system:

   • Default OpenPose DLL or Portable demo or compiled library? Not demo, Last version of Openpose installed directly downloaded at the root of the github. Installed following the installation guide

[ZiuTinyat]

Hi,

CPU mode should work normally as GPU mode, and your configuration also seems correct.

One possible reason would be, when you modify the parameters in the code, did you also change them in the inspector? Unity inspector will overwrite the initialized value in the script.

If things are still not working, I suggest trying openpose binary in your machine first, to check if your pre-requisites are all satisfied. (e.g. CUDA and CUDNN)

[FrenchDoge]

Thank you for your answer

I don't change any parameter in unity when i run the demo, but don't hesitate to ask me for screen captures if you need to check the default configurations of specific components of the unity demo.
Also the openpose quick start demo works correctly, as long as I reduce the net resolution in the command prompt, for example : "--net-resolution -1x64" (with 1x64 I have 6 fps, which is fine for my work by the way)

Unfortunately Unity still crashes, I don't know if you have any other idea left, thanks for your help anyway

[FrenchDoge]

I feel a bit lost right now, so excuse me if my question sounds a bit dumb, but until now, I assumed that the unity_plugin-master zip file was the reunification of all the files already built.
But I just noticed that some folders were missing in the root folder, in comparison with the openpose master folder, so I was wondering if it was truly pre-built as I thought or if running demo.unity create the needed build files or if there is a step I missed (I am certain that I followed correctly the installation guide, but still)

[FrenchDoge]

I just noticed that after the configuration with CMake, at the lie "CUDA_SDK_ROOT_DIR" the path has not been found. Error message : "CUDA_SDK_ROOT_DIR-NOTFOUND". I guess it's a problem so I'm searching a solution but so far I didn't find the right path in windows. Only found ubuntu solutions. If you know it please tell me ^^

[FrenchDoge]

I have successfully ran the demo scene by changing gpuNumber from -1 to 0 in "OpenPoseUserScript". So only the camera run without the pose tracking, that makes me think it's clearly the tracking calculations that cause the crash. Moreover, I checked my computer performances while running with gpuNumber at -1, and the GPU has a quick spike at 100% of capacity. So my question is : Do you know a way to reduce that component performances ?

On the other hand, I tried to modify net resolution in the same file, but when I run the demo, it's by default in 368x368. Is the plugin not modifying the values ? or is it just a graphic "bug" ? Since reducing the resolution is the only way for me to launch the openpose demo, I assume that I need to do it also in unity

[ZiuTinyat]

In the current demo, the UI won't reflect the change of parameters in codes. That is, you can change the parameters through the UI but the change of parameters in codes won't be updated in the UI. It's kinda UI "bug" but I just ignored that in order to reduce the complexity of codes.

Back to your problem, CUDA SDK is not installed in Windows by default and you need to download it according to your GPU version, as well as CUDNN required by OpenPose. (But you said your openpose quickstart works normally so I assume that you already have them?)

So I still think it is most possibly your configuration. You may try hardcoding the resolution into the function first:
private void UserConfigureOpenPose(){ OPWrapper.OPConfigurePose( /* poseMode */ PoseMode.Enabled, /* netInputSize */ new Vector2Int(-1, 16), /* outputSize */ null, /* keypointScaleMode */ ScaleMode.InputResolution, /* gpuNumber */ -1, /* gpuNumberStart */ 0, /* scalesNumber */ 1, /* scaleGap */ 0.3f, /* renderMode */ RenderMode.Gpu, /* poseModel */ PoseModel.BODY_25, /* blendOriginalFrame */ true, /* alphaKeypoint */ 0.6f, /* alphaHeatMap */ 0.7f, /* defaultPartToRender */ 0, /* modelFolder */ null, /* heatMapTypes */ HeatMapType.None, /* heatMapScaleMode */ ScaleMode.UnsignedChar, /* addPartCandidates */ false, /* renderThreshold */ renderThreshold, /* numberPeopleMax */ maxPeople, /* maximizePositives */ false, /* fpsMax fps_max */ -1.0, /* protoTxtPath */ "", /* caffeModelPath */ "", /* upsamplingRatio */ 0f);

[FrenchDoge]

Thank you for your answer,

Your hardcoding solution works. If I replace "netResolution" by "new Vector2Int(-1, 64)" the plugin runs correctly and construct the unity skeleton over the image as intended.
Something weird that is worth noticing i think, is that if I replace "netResolution" by "new Vector2Int(-1, 16)", the camera runs but without drawing the skeleton. Same for openpose Demo : the option "--net_resolution -1x64" draws the skeleton but not "--net_resolution -1x16"

About the CUDA/CUDNN issue, I run correctly the openpose demo, but I use the CPU_ONLY mode. in GPU mode the computer can't create the images correctly. It behaves like if the calculations were too big to produce outputs (the window opens but stays black). I suppose back then that my graphic component was just not strong enough (the installation guide recommend titan X, mine is less powerful) but I'm not sure about that.
According to you, is my CPU truly more powerful that my GPU, or should my GPU be powerful enough ? in the second case it could mean that I just didn't install CUDA/CUDNN correctly, even if I think I followed the installation guide correctly.

(sorry If I don't use the correct IT terms, I'm an intern, not yet a professional ^^)

[FrenchDoge]

I forgot to say something : Unity runs correctly with the hardcoding solution, as long as I don't try to use face/hands recognition. If I do, the plugin crashes. I tried to replace handResolution/faceResolution by new Vector2Int(64, 64) but then in unity, an error occurs :

"Assets/OpenPose/Examples/Scripts/OpenPoseUserScript.cs(103,23): error CS1502: The best overloaded method match for `OpenPose.OPWrapper.OPConfigureHand(bool, OpenPose.Detector, UnityEngine.Vector2Int?, int, float, OpenPose.RenderMode, float, float, float)' has some invalid arguments"

[ZiuTinyat]

The hardcoding way suggests that the initialization value in scripts is overwritten by value in Unity inspector. You can change the value in the inspector rather than in code to properly set the parameters in you previous case. Shown in the screenshot.

It's normal that openpose crashes when your GPU has not enough memory. You can try reducing the resolution for body/hand/face but 16 is too small to correctly detect. Usually 64 is sufficient enough to render something.

[FrenchDoge]

I would indeed need to modify the values in the inspector, but I have a weird bug : my "Openpose User Script" window is not displayed like yours.

As you can see, the window is not filled with the values and it says "default references will only be applied in edit mode". But I actually am in edit mode, am I not ?

I'm already searching on the internet a similar issue but for now I haven't find the solution, so if you have a clue, I'd be grateful ^^

[FrenchDoge]

Ok i "solved" it : I tried to open it in the "project" window but I have the same as you if I open the file situated in the "hierarchy" window.

Well thank you @ZiuTinyat for your help and your patience, it seems like all my problems are solved !

[ZiuTinyat]

That's how it works. Hope you enjoy using it!

[ecremasco]

I seem to have the same problem as @FrenchDoge, but I couldn't solve it doing what was suggested here.

• I can run the testBinary with no problem after I changed the resolution to 1x80.

• I tried changing the hand and face resolution to 1x64, 1x16 and 1x80 in the Unity Inspector and I also replaced the UserConfigureOpenPose() by the suggestion made by @ZiuTinyat .

Is there anything else I can try?

I only need to track the hands, so if I could disable everything else and keep only the hand detection, it would work for me.

Issue Summary

Unity crashed running the Demo Scene

Executed Command (if any)

Play Demo Scene

OpenPose Output (if any)

Unity closes after I run the Demo.

Errors (if any)

Unity crashed.

Type of Issue
Unity crashes on trying to execute the Demo.

Your System Configuration

OpenPose version: OpenPose Unity Plugin default

General configuration:
    Operating system : Windows 10, 64-bit
    Release or Debug mode : release
    Compiler : VS 2015 Community

Non-default settings:
    Any custom OpenPose configuration with respect to the default version? : No

3rd-party software:
    Caffe version: Default from OpenPose
    OpenCV version: OpenPose default

If GPU mode issue: 
    CUDA version : 10
    cuDNN version: 7.5
    GPU model : Nvidia Geforce GTX 960

[aanexplus]

I seem to have the same problem as @FrenchDoge, but I couldn't solve it doing what was suggested here.

• I can run the testBinary with no problem after I changed the resolution to 1x80.
• I tried changing the hand and face resolution to 1x64, 1x16 and 1x80 in the Unity Inspector and I also replaced the UserConfigureOpenPose() by the suggestion made by @ZiuTinyat .

Is there anything else I can try?

I only need to track the hands, so if I could disable everything else and keep only the hand detection, it would work for me.

Issue Summary

Unity crashed running the Demo Scene

Executed Command (if any)

Play Demo Scene

OpenPose Output (if any)

Unity closes after I run the Demo.

Errors (if any)

Unity crashed.

Type of Issue
Unity crashes on trying to execute the Demo.

Your System Configuration

OpenPose version: OpenPose Unity Plugin default

General configuration:
    Operating system : Windows 10, 64-bit
    Release or Debug mode : release
    Compiler : VS 2015 Community

Non-default settings:
    Any custom OpenPose configuration with respect to the default version? : No

3rd-party software:
    Caffe version: Default from OpenPose
    OpenCV version: OpenPose default

If GPU mode issue: 
    CUDA version : 10
    cuDNN version: 7.5
    GPU model : Nvidia Geforce GTX 960

Hey,
What I did was to close Chrome browser and that prevents Unity from crashing. I will suggest you close any heavy applications.
Best of luck

[KoenL-1747481]

Same issue as most people here. Still crashes even with 64 x 64. I have 2GB VRAM which is not enough. If anyone has any working solution by now, i'd love to hear it :).