problem with installation FSDS from source in ubuntu 20.04

[saikumar-kolla]

I am unable to build the missing modules again. I am using Unreal Engine 4.27.2 version and Ubuntu 20.04

In the instructions this was mentioned and i clicked yes.

Now this message popped up. can any one help me with this like what are the steps i need to do or what i need to check.

[wouter-heerwegh]

Hi @saikumar-kolla,

Have you installed Airsim aswell? If so, do you know with which version of clang?

It has been a while since I built the sim from source, and I would have to instal Unreal engine again.

I'll try to build it again tomorrow and Tuesday.

In the meantime, it could be good to get a full build log, so that I can analyze it a bit.

[wouter-heerwegh]

@saikumar-kolla,

I have ran into the issue before. When I run ~/UnrealEngine/Engine/Binaries/ThirdParty/Mono/Linux/bin/mono ~/UnrealEngine/Engine/Binaries/DotNET/UnrealBuildTool.exe Development Linux -Project=/home/wouter/Formula-Student-Driverless-Simulator/UE4Project/FSOnline.uproject -TargetType=Editor -Progress, the logs reveal that an undefined symbol is used in librpc.a (this is a library that contains a protocol to communicate between the simulator and the ros bridges.

The undefined symbol in question is pthread_cond_clockwait. This is a function that is introduced in glibc v2.30, released in August of 2019. Unreal engine crosscompiles their engine with a different toolchain, to support a wide variety of platforms. The toolchain is for CentOS 7, which contains glibc 2.17, and thus pthread_cond_clockwait is not defined.

I'm going to try to look into building librpc with the Unreal engine toolchain, as this should solve the issues that we have, but in the meantime, there are other things you could do. For instance build librpc inside a docker container that runs ubuntu 18.04, or a different OS with an older glibc version than 2.30. Another option is to fully build the project without the toolchain provided by Unreal

I'll let you know if I have some progress

[saikumar-kolla]

@wouter-heerwegh
I have run the setup.sh and build.sh files present inside Airsim. There are the logs i got
setup_sh.txt
build_sh.txt

After that when i ran FSOnline.uproject with UE i got this log
Fsonlinelog.txt

[wouter-heerwegh]

@saikumar-kolla,
Can't conclude much from the logs yet, but I'm guessing that if you run the following command with the paths changed to your specific case, you will see the pthread_cond_clockwait undefined symbol error.

~/UnrealEngine/Engine/Binaries/ThirdParty/Mono/Linux/bin/mono ~/UnrealEngine/Engine/Binaries/DotNET/UnrealBuildTool.exe Development Linux -Project=/home/wouter/Formula-Student-Driverless-Simulator/UE4Project/FSOnline.uproject -TargetType=Editor -Progress

[saikumar-kolla]

@wouter-heerwegh
Yes, I am getting the same error

building.txt

[wouter-heerwegh]

Hi @saikumar-kolla,

I made a cross compile toolchain, that could used to build the airsim libraries, so that they can be used by both your system and unreal engine. You can download the toolchain from my google drive here. I'll do some tests and patch the build and setup scripts tonight or tomorrow, but in case you can't wait, feel free to already compile.

[saikumar-kolla]

@wouter-heerwegh
Sorry to ask this question but, I don't have any idea about how to compile a toolchain. i am assuming that in Airsim folder i have steup.sh and build.sh. Do i need to modify those files.
or
Can i just wait for two days and clone the repository again and run setup and build in Airsim folder

[wouter-heerwegh]

If you want to compile with the toolchain, you will have to do the following:

1. Download the toolchain and extract the zip on your system
2. Add the /bin folder to the PATH env variable (export PATH="$PATH:path_to_toolchain/bin)
3. Change the CC and CXX env variables in the build.sh file, for instance here in the file:

export CC="x86_64-centos7-linux-gnu-gcc"
export CXX="x86_64-centos7-linux-gnu-g++"

I haven't fully tested if everything works with this, but the pthread_cond_clockwait symbol is omitted in the static rpc library when building with the toolchain.

If you're not sure about making these changes, I will be updating the scripts to build Airsim, so you could indeed pull the changes later.

[wouter-heerwegh]

I tried building the unreal engine project with the libraries compiled with my toolchain, but it seems there is still an issue, I will need some more time to try and fix the toolchain. In the meantime, what worked for me was building airsim in an Ubuntu 18.04 docker container, and then continuing as usual. So if you don't want to wait, it's already something you can try.

[wouter-heerwegh]

Wait @saikumar-kolla,

Could you convert all clang mentions inside the build.sh and setup.sh scripts in Airsim to 8?
So for instance:

#install clang and build tools
VERSION=$(lsb_release -rs | cut -d. -f1)
# Since Ubuntu 17 clang is part of the core repository
# See https://packages.ubuntu.com/search?keywords=clang-8
if [ "$VERSION" -lt "20" ]; then
    wget https://apt.llvm.org/llvm.sh
    chmod +x llvm.sh
    sudo ./llvm.sh 12
fi
sudo apt-get install -y clang-12 clang++-12 libc++-12-dev libc++abi-12-dev

becomes:

#install clang and build tools
VERSION=$(lsb_release -rs | cut -d. -f1)
# Since Ubuntu 17 clang is part of the core repository
# See https://packages.ubuntu.com/search?keywords=clang-8
if [ "$VERSION" -lt "20" ]; then
    wget https://apt.llvm.org/llvm.sh
    chmod +x llvm.sh
    sudo ./llvm.sh 8
fi
sudo apt-get install -y clang-8 clang++-8 libc++-8-dev libc++abi-8-dev

[saikumar-kolla]

@wouter-heerwegh
I am getting the same error.

[wouter-heerwegh]

@saikumar-kolla, did you fully remove the build_debug folder before rebuilding?

[saikumar-kolla]

Yes i removed the build_debug folder.
these are the logs i got while running setup and build.
build_sh.txt
setup_sh.txt

[wouter-heerwegh]

Hi @saikumar-kolla,

In the build log, it looks like the CC and CXX variables are still set to the cross compile toolchain. Could you remove these and replace them with clang-8 and clang++-8?

Does that fix it?

[saikumar-kolla]

these are the contents of build and setup files
build.txt
setup.txt

[wouter-heerwegh]

These lines (34-37) should be changed to clang-8 and clang++-8

else
    export CC="x86_64-centos7-linux-gnu-gcc"
    export CXX="x86_64-centos7-linux-gnu-g++"
fi

[saikumar-kolla]

@wouter-heerwegh
I have changed the variables to clang-8 and clang++-8 as you told me and i got this log
build_sh.txt

[wouter-heerwegh]

Looks good. If you now run the Unreal project and rebuild the Airsim and Blocks modules, does it succeed?

[saikumar-kolla]

@wouter-heerwegh i think i am getting the same error while rebuilding again. This is the log i got by running the FSOnline.uproject
FSOnline_uproject.txt

[wouter-heerwegh]

Strange. Could you clean the full repo (git checkout . && git clean -xdf this will reset the entire repo, so make sure to save your code if needed), make the changes into Airsim again to convert clang 12 to 8 and rebuild. Just to make sure that everything is regenerated completely.

[saikumar-kolla]

@wouter-heerwegh
I am sorry but it's not working for me. I am getting the same error even after doing the above steps you mentioned.

[wouter-heerwegh]

If you want, we can set up a meeting to have a look at it together. Feel free to send me some dates and times on which you are available. My email address is wouter.heerwegh@formulaelectric.be.

[saikumar-kolla]

@wouter-heerwegh
It's opening for me now. I don't know why i just double clicked on FSOnline.uproject and it popped with build using Unreal Engine and its opening. I think the problem is with GenerateProjectFiles.sh in UEProject folder. the directory for unreal Engine is not set

[wouter-heerwegh]

Okay, so with clang 8 it works, right? Could you check if it works when you use this command with absolute paths?

~/UnrealEngine/Engine/Binaries/Linux/UE4Editor ~/Formula-Student-Driverless-Simulator/UE4Project/FSOnline.uproject

[saikumar-kolla]

yes it is working.

[wouter-heerwegh]

Great! You can leave this issue open for now, I will make some changes to Airsim to make sure that clang 8 is used when using Ubuntu 20.04

[saikumar-kolla]

@wouter-heerwegh
Thanks for the help.

[PanterSoft]

Hello, I have tried several of the solutions which are mentioned here but none of these works for me.

Here are my steps I did.

git clone https://github.com/FS-Driverless/Formula-Student-Driverless-Simulator.git --recurse-submodules -b v2.2.0 ./setup.sh > setup.txt && ./build.sh > build.txt

build.txt
setup.txt

Can someone maybe help ?

[wouter-heerwegh]

Hi @PanterSoft,

It seems like Airsim built, make sure that before you build, you remove build_debug, otherwise It might skip some compilation steps.

I'm guessing with this built, you can't open the unreal project?

If not, could you run this command and post the output (make sure to change the paths)?

~/UnrealEngine/Engine/Binaries/ThirdParty/Mono/Linux/bin/mono ~/UnrealEngine/Engine/Binaries/DotNET/UnrealBuildTool.exe Development Linux -Project=/home/wouter/Formula-Student-Driverless-Simulator/UE4Project/FSOnline.uproject -TargetType=Editor -Progress

If that worked succesfully, try to open the project again. For some reason this worked for me. I cannot seem to package the project on Ubuntu 20.04 though. I'm going to try to port the project to UE5, and see if this has less issues.

If you do want to develop on the simulator and package, I highly recommend using Windows instead. It's easier to debug and you can cross compile for linux.

[PanterSoft]

Hi @wouter-heerwegh

Thanks for your response. I also got it working now. I had to combine a few of the answers here.

First of all I had to change from Clang12 to clang8 in setup.sh and build.sh

Starting at line 32 in setup.sh:
sudo ./llvm.sh 8

and line 34:
sudo apt-get install -y clang-8 clang++-8 libc++-8-dev libc++abi-8-dev

in build.sh I had to change line 35 and 36 to:
export CC="clang-8"
export CXX="clang++-8"

and then in the AirSim folder:
setup.sh && build.sh

after that I had to run your provided command
~/UnrealEngine/Engine/Binaries/ThirdParty/Mono/Linux/bin/mono ~/UnrealEngine/Engine/Binaries/DotNET/UnrealBuildTool.exe Development Linux -Project=/home/YOUR_USERNAME/Formula-Student-Driverless-Simulator/UE4Project/FSOnline.uproject -TargetType=Editor -Progress

after that It started normally like described in the instructions.

But I will also probably try using Windows for that in the Future if it gives me some problems in the future.
thanks for your help @wouter-heerwegh

[wouter-heerwegh]

Hi @PanterSoft

Quick thing to note, for other people coming across this issue, you should run setup.sh && build.sh before running the mono command. It will otherwise not find the Airsim library and fail.

Thanks for the response, glad it worked.