Coordinate frames do not move when using ArduCopter with ROS

[bn2021]

Bug report

• AirSim Version/#commit: 1.4 and #b9c26181abfd473a5dc2f6a7bd1c1cd0149740f0
• UE/Unity version: N/A
• autopilot version: N/A
• OS Version: Ubuntu 18.04 LTS

What's the issue you encountered?

I could not find anything about it, so I am not sure if this is a kind of intended behavior or a bug.

Using AirSim together with ROS works perfectly fine for me if SimpleFlight is set as VehicleType in the settings.json (Multirotor SimMode): Coordinate frames move as expected in rviz when flying around and sensor and image data is published on topics.

However, frames do not move as I would expect if ArduCopter as VehicleType is used. Then most of the frames, including the odom_local_ned frame, are located within the world frame (0, 0, 0) or in the location that was defined in the settings.json. Only some of the camera frames move.

Note: The AirSim ROS node was built from source, for the simulation itself one of the binaries is used.

Settings

Minimal settings:

{
  "SettingsVersion": 1.2,
  "SimMode": "Multirotor",
  "Vehicles": {
    "copter": {
      "VehicleType": "ArduCopter",
      "UdpPort": 9003,
      "ControlPort": 9002
    }
  }
}

How can the issue be reproduced?

1. Start AirSim
2. Run the AirSim-ROS-Node
3. For visualization run rviz: roslaunch airsim_ros_pkgs rviz.launch
4. Move vehicle (e.g. simple takeoff), using rosservice for SimpleFlight and ardupilot for ArduCopter
5. If SimpleFlight is used, the odom_local_ned frame moves. For ArduCopter it does not move.

Thank you for your great work! Maybe you can help or have an idea here. Thank you!

[rajat2004]

I would say its a bit of intended behaviour. The AirSim-ArduPilot integration doesn't have state info flowing from AP to Airsim, which is supposed to be used for the movement APIs. It would be possible to add code so that the sensor data being sent by Airsim is also used for the state info APIs used by AirSim, however the data should be what the flight controller/firmware thinks the values are, which can be different from the simulated data. Therefore, the AP integration just returns NaN for most of the APIs, see ArduCopterApi.hpp

Instead, for controlling the vehicle and gathering such information, you can use pymavlink or Mavros for connecting to ArduPilot itself rather than through AirSim. Another intention of going about it this way was that these methods will be applicable for real vehicles as well, rather than the AirSim APIs which won't be. For some docs on using ROS with AP, see https://ardupilot.org/dev/docs/ros.html

Hope this helps!

[bn2021]

Thank you @rajat2004!

Is it also intended behavior that some camera frames are moving using AirSim ROS with ArduCopter while other frames do not move? Are these camera poses reliable given your described case that a flight controller/ firmware may assume different data?

Thank you again and best regards!

[rajat2004]

Hi, sorry for the late reply.
The camera poses are present since the current ArduCopter (or other vehicle wrappers also) don't modify the image capture part, that is handled separately in the UE4 and Unity plugin part rather than in AirLib. Therefore, the info in these isn't removed which is probably what's causing some of those to move while others don't. As mentioned earlier, some of this info could be added in the APIs also (which currently return NaN), however it could differ from the firmware's estimation.

I myself haven't tested this, so I feel it would be wrong to arrive at a conclusion without some testing. For checking the divergence between the estimated and Airsim's position data, the GPS sensor API could be used. If you want, I could add some code to implement the data gathering APIs if it makes things simpler for testing.

[bn2021]

Hi @rajat2004,
I am also sorry for the late reply!

Thank you for your answer and especially for your offer to implement test code! However, for now, the simulated position data is sufficient for me.

In general, it could be good to have some kind of ground truth from the game engine in addition to the simulated data and independent of the vehicle type used in order to evaluate things. For example by offering additional coordinate frames.

Thank you again and best regards!

[rajat2004]

The simGetGroundTruthKinematics/Environment do work, and all the sensor APIs as well. APIs which get the estimated data such as getMultirotorState, and movement APIs aren't implemented

I'm not sure what do you mean by additional coordinate frames, like GPS and NED data? They're included in the ground truth data. Or something else

[bn2021]

Sorry again for the late reply!

I guess the environment topic is kind of what I meant. Thank you! :)

[3zuli]

Hello, I am also having this problem. I'm trying to use AirSim to simulate an existing drone system based on Ardupilot and ROS. We have several nodes for mission management, trajectory generation etc. and the drone is controlled using Mavros. My understanding was that AirSim with Ardupilot SITL can be used as a drop-in replacement of the physical drone hardware and flight controller, Mavros can be configured to communicate with SITL and the rest of the system can seamlessly interact with the simulated drone.

I'm also running the airsim_node from airsim_ros_pkgs which I use to obtain the depth camera images. The problem is that position estimates from Ardupilot (/mavros/global_position/local, /mavros/local_position/odom) and the DepthCameraCenter_body frame from AirSim start in the same location, but they quickly diverge as the drone flies. The topic /airsim_node/Drone/odom_local_ned is being published, however the position is always zero. The position from /airsim_node/Drone/environment corresponds to the DepthCameraCenter_body frame, so at least this part seems to be working.

I don't understand why the Ardupilot and AirSim positons aren't identical. Based on what @rajat2004 said it seems that the positions from Ardupilot are not being fed back into AirSim, but honestly that doesn't make sense to me.

[3zuli]

I think I was able to narrow down the issue. AirSim sends a JSON-encoded message to Ardupilot SITL which contains sensor data (IMU, GPS, barometer etc.) based on the current state estimated using its model of the drone physics. SITL should then update its state estimate using the received data and compute new motor PWM values which are then sent back to AirSim. It appears that this is the only input that AirSim uses to update the physics model, which would lead me to believe that there is a mismatch in the model or in control allocation matrix between AirSim and SITL.

I am still under the assumption that AirSim and SITL poses should match up. The SimpleFlightApi implementation of the getKinematicsEstimated() function returns the state estimated by the simulated flight controller. Similarly, MavLinkMultirotorApi returns the actual state received from the vehicle via MavLink. However in ArduCopterApi the function returns empty values and prints the "Not Implemented: getKinematicsEstimated" message. I think this is also the reason why the /airsim_node/Drone/odom_local_ned topic contains only zeros.

Would this issue be fixed if getKinematicsEstimated() returned the actual pose from SITL? Adding the sending of pose data from SITL to AirSim would be a bit cumbersome under the current implementation of ArduCopterApi, but doable. I could also try to use the ArduCopterSoloApi vehicle type, but I'm not sure if it's compatible with Ardupilot SITL. Can somebody help confirm my assumptions?

[tridge]

I suspect you're right that implementing getKinematicsEstimated() likely would work, but the point from @rajat2004 is that you really should be pointing ROS at ArduPilot directly, not at AirSim. That way the ROS code you develop will work on a real vehicle without modification.
Note that you can set AHRS_EKF_TYPE=10 in ArduPilot if you don't want any discrepancy between truth and estimated state in ArduPilot (that tells ArduPilot to use an EKF-shim that just passes through the simulated data, bypassing the EKF state estimation). Obviously this setting only works in a simulated environment.

[3zuli]

Thank you @tridge for the suggestions. I tried setting AHRS_EKF_TYPE to 11 (as per the docs) and it does somewhat help, but there's still a discrepancy between the AirSim pose and Ardupilot pose. At least it appears that the discrepancy is more constant, before this the poses just kept drifting apart. I think I will have to try to implement getKinematicsEstimated to have it return the pose estimate from SITL.

... but the point from @rajat2004 is that you really should be pointing ROS at ArduPilot directly, not at AirSim. That way the ROS code you develop will work on a real vehicle without modification.

That is precisely what we're already doing, our mavros node is configured to connect to the SITL simulator and the rest of the system behaves as if it was connected to an actual flight controller. I'm using the AirSim ROS node on top of that to get the depth camera data.

[rajat2004]

@3zuli Sorry for the late reply. I'm not very clear about why implementing getKinematicsEstimated will help since you already have the ROS node working with ArduPilot. AFAIK, AirSim doesn't use the estimated kinematics from the flight controller apart from returning the info through APIs, it's not used in the internal physics engine.
SimpleFlight doesn't have any estimation, and just returns the ground truth info as the estimated values. And ArduCopterSolo is no longer compatible with ArduPilot, haven't gotten around to removing it. I'll be happy to help as well if estimated kinematics are needed, can discuss more here or some other platform such as Discord also