Camera doesn't move with car while running DreyeVR_AI.py

[aureliuszi]

Hello Gustavo (and DReyeVR team)!

The camera of DReyeVR_AI.py does not seem to move with the car.

I am running both the 2D and VR version of DReyeVR with the following commands:

CarlaUE4.exe OR CarlaUE4.exe -vr
python DReyeVR_AI.py

Here is the output after the python command:

WARNING: Version mismatch detected: You are trying to connect to a simulator that might be incompatible with this API
WARNING: Client API version = 0.9.13
WARNING: Simulator API version = 0.9.13-1-g8854804f4-dirty
INFO: Found the required file in cache! Carla/Maps/TM/Town03.bin
Successfully set autopilot on ego vehicle
successfully spawned 10 vehicles

Below is an image of what I see as the car turns without repositioning myself. I have to physically move my body/ the camera in order to see the front of the car again.

[GustavoSilvera]

Hi @aureliuszi, you are right, this does happen when you switch which "pawn" is possessing the EgoVehicle at a time. I imagine you pressed 3 on the keyboard as the documentation suggests.

This is due to the camera in "autopilot-mode" being fixed relative to the world's rotation, rather than the vehicle's rotation. We are aware of this bug and are working on a fix in the next update :)

[GustavoSilvera]

Also, to clarify, the camera does still "move" translation-wise with the vehicle correct? It is just the rotation of the camera that is not following the vehicle.

[ethicallab]

That's correct! I am "moving" with the car, but not rotating with it.

[GustavoSilvera]

@aureliuszi @ethicallab would you mind trying out the latest commit in the dev branch? I believe these changes fix the autopilot camera issue and VR spawn position issue.

[aureliuszi]

Hi @GustavoSilvera! To clarify, I would have to: reclone the DreyeVR repo; rerun the make carla CARLA=../carla and
make sr SR=../scenario_runner commands; and rebuild the package (make package) to see these changes? Thanks!

[GustavoSilvera]

Hi @aureliuszi from the DReyeVR repo you can simply git checkout dev and git pull to grab the latest commit, then you'd need to run make carla CARLA=../carla as per usual (this commit doesn't include any changes for scenario-runner so you can omit that step), then you should rebuild everything with make PythonAPI && make package to get the newest changes.

Caution, running make carla CARLA=../carla will overwrite the corresponding files so if you have any changes to these files in your carla directory that you want to keep, you should make sure to back them up somehow (we have Carla fork and use several custom DReyeVR branches)

[aureliuszi]

Ran everything. Got this error on make package:

     Creating library C:\Users\kp\Documents\G\carla9.12\carla\Unreal\CarlaUE4\Intermediate\Build\Win64\UE4Editor\Development\CarlaUE4\UE4Editor-CarlaUE4.lib and object C:\Users\kp\Documents\G\carla9.12\carla\Unreal\CarlaUE4\Intermediate\Build\Win64\UE4Editor\Development\CarlaUE4\UE4Editor-CarlaUE4.exp

-[Package]: [ERROR] There was a problem while building the CarlaUE4Editor.
          [ERROR] Please read the screen log for more information.
make: *** [package] Error 1```

[GustavoSilvera]

Uh oh, this is a new one. Try make launch before make package.

[GustavoSilvera]

I think it might require that you build the editor (CarlaUE4Editor) before the package (CarlaUE4) which is done through make launch.

[aureliuszi]

Hmmm, just tried make launch and this happened:

C:\Users\kp\Documents\G\carla9.12\carla\Unreal\CarlaUE4\Plugins\Carla\Source\Carla/Game/CarlaHUD.h(54): warning C4996: 'UWheeledVehicleMovementComponent': PhysX is deprecated. Use the UChaosWheeledVehicleMovementComponent fron the ChaosVehiclePhysics Plugin. Please update your code to the new API before upgrading to the next release, otherwise your project will no longer compile.
 C:/Users/kp/Documents/G/carla9.12/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Game/CarlaHUD.cpp(23): warning C4996: 'UWheeledVehicleMovementComponent::DrawDebug': PhysX is deprecated. Use the UChaosWheeledVehicleMovementComponent fron the ChaosVehiclePhysics Plugin. Please update your code to the new API before upgrading to the next release, otherwise your project will no longer compile.
make: *** [CarlaUE4Editor] Error 6

[aureliuszi]

I'm going to try cleaning everything and rebuilding.

[aureliuszi]

It's a similar error -

I ran make clean and make rebuild and then make launch

[28/32] UE4Editor-CarlaUE4.lib
     Creating library C:\Users\kparodi\Documents\G\carla9.12\carla\Unreal\CarlaUE4\Intermediate\Build\Win64\UE4Editor\Development\CarlaUE4\UE4Editor-CarlaUE4.lib and object C:\Users\kparodi\Documents\G\carla9.12\carla\Unreal\CarlaUE4\Intermediate\Build\Win64\UE4Editor\Development\CarlaUE4\UE4Editor-CarlaUE4.exp
make: *** [CarlaUE4Editor] Error 6

[GustavoSilvera]

Very strange. Can you upload the full log for make launch?

[aureliuszi]

Where is the log? Just the stuff in the x64 prompt?

[GustavoSilvera]

Yes, you can right click the titlebar of the cmd prompt to select all and copy it into a txt

[aureliuszi]

make_launch.txt

Here you go, the stuff that printed after make launch was quite outrageous. Here is as much as could be copied and pasted

[GustavoSilvera]

Oh I see the error, it is specifically this error C2955: 'DReyeVRDataRecorder': use of class template requires template argument list and it is because the new changes have this file deleted (part of another refactor). The solution is to delete this file in your Carla directory. Specifically delete this file: C:/Users/kparodi/Documents/G/carla9.12/carla/Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Recorder/DReyeVRRecorder.cpp

[GustavoSilvera]

This would also be fixed by recloning carla itself and then make installing from dev directly to that (since it would not include DReyeVRRecorder.cpp in the first place.

[aureliuszi]

That was very fast, thank you! Things appear to be building now and I'll have an update for you shortly.

[aureliuszi]

So make launch worked fine. I was trying to run make carla CARLA=../carla and got this:

C:\Users\kparodi\Documents\G\carla9.12\DReyeVR>make carla CARLA=../carla
process_begin: CreateProcess(NULL, bash C:\Users\kparodi\Documents\G\carla9.12\DReyeVR\Scripts\install.sh --carla ../carla, ...) failed.
make (e=2): The system cannot find the file specified.
make: *** [install] Error 2

[aureliuszi]

You can see the contents of Scripts appears to contain what we need...

C:\Users\kparodi\Documents\G\carla9.12\DReyeVR\Scripts>dir
 Directory of C:\Users\kparodi\Documents\G\carla9.12\DReyeVR\Scripts

05/02/2022  02:25 PM    <DIR>          .
05/02/2022  02:25 PM    <DIR>          ..
04/23/2022  10:04 AM               880 DReyeVR.mk.help
04/23/2022  10:04 AM             1,519 install.sh
05/02/2022  02:25 PM             3,089 installers.sh
04/23/2022  10:04 AM             1,593 patch_sranipal.sh
04/23/2022  10:04 AM             2,268 utils.sh

[GustavoSilvera]

Interesting, is this make carla CARLA=../carla on Windows batch (cmd prompt) or WSL?

[GustavoSilvera]

These scripts are designed to be run in bash (eg. installers.sh) so I recommend you run make carla CARLA=../carla in WSL.

[aureliuszi]

x64 Native Tools Command Prompt for VS 2019

[GustavoSilvera]

Yeah, run the make carla CARLA=../carla in WSL, but then run make launch/make package/make PythonAPI in x64 Native Tools Command Prompt for VS 2019

[aureliuszi]

Issue appears to be solved for 2D mode, however when I play the sim in VR it no longer works. In fact, if I even attempt to make launch and view the editor in VR, I no longer have the option of opening VR Preview. I tried both the -vr mode form the package and the make launch editor

[GustavoSilvera]

Interesting, can you check that the SteamVR plugin is installed and enabled in the project?

[aureliuszi]

Fixed, it was a problem on Steam's End (the SteamVR plugin was installed). Restarting my computer stopped it from glitching. Additionally, in VR, the camera now stays where I first put it! Thank you so much for your help on this :D

[GustavoSilvera]

Great news! This should fix this issue (#14) and #15 (and #21) but I will refrain from closing the issues until everything is extensively tested and upstreamed in main. This should happen soon, but I wanted to get you the critical bugfixes early.

[aureliuszi]

Thanks so much! Appreciated!

[GustavoSilvera]

Hey, @aureliuszi please check out the upgrading-DReyeVR section of the documentation to pull from main to get all the latest bugfixes. This issue should be fixed with those changes.