Rover: add a line-following mode

[peterbarker]

Issue details

Rover lacks a line-following mode, a fairly common thing to do with a Rover

This will be hampered by the fact that we don't currently support a distinct sensor type that gives you a hotter/colder for a line. At least adding an analogue light sensor would be required. Maybe something as complicated as https://www.sparkfun.com/products/12787 Discuss before implementing....

Version

All

Platform

[ ] All
[ ] AntennaTracker
[ ] Copter
[ ] Plane
[ X] Rover
[ ] Submarine

[techytoes]

Hey, @peterbarker Can I take this issue?
I am new to this organization so can you please elaborate required changes and relevant files to work with.

[peterbarker]

@firedranzer I apologise that I haven't actually investigated this issue all that much myself.

It's seemed a big hole in our lineup for quite some time!

I'm going to go ahead on the assumption we're doing a "simple" line following thing - so a simple sensor and simple code. No piping video data through OpenCV here!

There are basically two parts to the problem.

The first is that you need to be able to tell whether you're on the boundary of a line or not. There are light sensors suitable for that - research required. The sensor should be chosen with the requirement we have to put it into ArduPilot, of course. The code has to be written for this device. An analogue sensor would probably be simplest to interface code-wise but kind of yucky in other ways. An i2c device may be better.

The second part is to create a driving mode that uses the data from that sensor. Rover's structure should make this fairly straight forward.

It would actually be nice to have a simulated light sensor. However, since I don't actually have one at the moment, I don't really know what shape that data would take. With a simulated light sensor the driving mode can be tested quickly, easily, and with enough work repeatably. Once a sensor (from part1) is chosen, writing a simulated equivalent for it from the data sheet should be possible.

[rmackay9]

To add a bit on Peter's advice, I think it could be done something like this:

• write an AP_LineFollow sensor driver (call it what you want of course). Perhaps this wiki page may help getting the low-level code working. Hopefully this new sensor driver returns a number telling us where the line is. For example "-100" might mean the line is at the very left edge of the sensor, "0" means straight ahead, "+100" means to the far right. It might be best to have a parameter in the library to allow the user to specify a scaler to allow converting this sensor number to an angle (i.e. if this scalar parameter is "0.1) then "-100" = line is 10 degrees from straight ahead).
• write a new Rover drive mode that accepts pilot throttle input for speed but accepts no steering input. The steering input should instead be based upon the output of the new AP_LineFollow driver. Instead of simply sending this output straight to the AP_MotorsUGV::set_steering() method like we do in Manual mode we should probably use either AR_AttitudeControl::get_steering_out_rate() like we do in ACRO or calc_steering_to_heading like we do in GUIDED.

[i-radwan]

I would like to work on this issue 😄
After doing some research I've found an I2C line following sensor https://www.sparkfun.com/products/13582
I've read some of the Ardupilot code and I think that I've understood the flow of sensors reading. I've also read some of the sensor Arduino library functions to see what do they return.

Now I think that I've to add a new category in the libraries folder in order to add the line follower sensor to it. The driver will use the I2C interface same as other drivers but for now, the driver will update the status depending on some dummy values until the simulator support this sensor (if you agree on it).
Then I have to add the mode that makes use of this driver and I think after that I could run the simulator to see the rover move randomly. Please let me know if something is wrong or missing, or if you have any suggestions. Thanks.

[peterbarker]

On Thu, 1 Mar 2018, Ibrahim Radwan wrote: @peterbarker would act as reflectance array sensor, so what about using analog sensors?

analogue sensors are fine. Just fiddly.

[peterbarker]

@hemoali Everything sounds good there.

One of the best ways to do this without the hardware is to write a simulator for the hardware!

This patch:

peterbarker@251c11e

is from my support-sds021-particle-sensors branch:
https://github.com/peterbarker/ardupilot/commits/sds021

[i-radwan]

@peterbarker So you think I've to start by adding the simulated sensor to SITL and then implement the above approach? and if I've to add this simulated sensor, should I read something first or understanding the code you wrote in your own repo would be enough?

[peterbarker]

@hemoali Yep.

[codeoftherings]

If no one is on it, can I take this issue? (New here)

[rmackay9]

@codeoftherings, it's open, go for it!

[peterbarker]

@hemoali - any movement on this one? @codeoftherings is interested in chasing this.

[rmackay9]

I think more than one person can work on it .. first one to get their PR into master wins!

[codeoftherings]

@rmackay9 I tried working on it but seem to hit a dead end. I need to be more familiar with the libraries and get some hardware. Can someone else take it up or give any guidance

[Nkg18]

Hello, I am Nilesh. Is this issue still open?
If yes, I would like to work on it.

[peterbarker]

@Nkg18 Well, we don't have a line-following mode yet, so yes!

[MohamedAliRashad]

But how to add a simulated sensor to SITL ?
i think this will be a very hard task, Right ?
@peterbarker @rmackay9

[peterbarker]

On Sun, 24 Feb 2019, mohamed ali rashad wrote: But how to add a simulated sensor to SITL ? i think this will be a very hard task, Right ?

Well, that depends on your current level of skill, surely :-) By the end of it it will be easier next time! It's not particularly difficult in several respects: - it's only simulation, so if you make a mistake you don't break real hardware - it's simulation, which makes iterating code revisions faster - it's simulation, so you can automate your testing Here's where we read from other simulated analogue devices: https://github.com/ardupilot/ardupilot/blob/master/libraries/AP_HAL_SITL/AnalogIn.cpp#L29 You'd make something similar to a rangefinder. First cut: you would modify the voltage by working out your distance from a point in local-NED based on your home or origin location. Second cut: modify voltage based on distance from straight line between two points. A more elaborate simulation might draw virtual lines between your mission waypoints - but that's in a different level of code. Or provide an input file of points to a SIM_LineDetector device (see SIM_VICON). Note that the current simulated analogue devices do not lend themselves to that.

[DaHaiHuha]

Hi, I am Cheng Alen. I saw this is still open, but why there is a × on platform rover? If possible, I would like to work on it, interested in rover

[peterbarker]

@DaHaiHuha That just indicates we think the code is only likely to be applicable to Rover - at least in the short term.

[DaHaiHuha]

@peterbarker wow, thank you! But i wonder why this brilliant open project did not include an interface associating the dronekit and direct throttle control, i mean , whenever i want to make a car by myself, the direct control by hand or simply code is always the first thing i would consider; However, after reading the dev docs of both Ardupilot and Rover, i admit that the pid & GNSS feedback control is great, but all the other developers develop the direct control of rover by themselves? Forgive my ignorance and lack of experience, thank you in advance!

[peterbarker]

On Mon, 25 Mar 2019, ChengDahai wrote: @peterbarker wow, thank you! But i wonder why this brilliant open project did not include an interface associating the dronekit and direct throttle control, i mean , whenever i want to make a car by myself, the direct control by hand or simply code is always the first thing i would consider; However, after reading the dev docs of both Ardupilot and Rover, i admit that the pid & GNSS feedback control is great, but all the other developers develop the direct control of rover by themselves? Forgive my ignorance and lack of experience, thank you in advance!

Oh, we allow external control of the vehicle via mavlink, don't fear there. Look at Rover's "GUIDED" mode. This PR isn't about external control, however; it would be nice to have something onboard :-)

[DaHaiHuha]

Oh, we allow external control of the vehicle via mavlink, don't fear there. Look at Rover's "GUIDED" mode. This PR isn't about external control, however; it would be nice to have something onboard :-)

Yes, but i wonder whether there is a direct way to control the acceleration speed, even via mavlink. This could cause some damage due to some unexpected situations such as being block by a stick lying on the road~

[rishabsingh3003]

Hi @peterbarker! I would like to work on this PR. Seems like no one is in it for now!
I was planning to simulate an analog sensor in SITL. Going with your suggestion, I am going to modify the voltage based on the distance of the vehicle from a straight line (formed between home and a particular way point). Is that okay?
After spending a while on this, I couldn't figure out how to access way points fed from mavproxy to my simulated analog device code. Could you elaborate on how I could do this? To form a line, and hence work out the voltage from that line, I would need those point coordinates. Thanks

[peterbarker]

On Wed, 6 Nov 2019, rishabsingh3003 wrote: I was planning to simulate an analog sensor in SITL. Going with your suggestion, I am going to modify the voltage based on the distance of the vehicle from a straight line (formed between home and a particular way point). Is that okay?

Yep, sounds good.

After spending a while on this, I couldn't figure out how to access way points fed from mavproxy to my simulated analog device code. Could you elaborate on how I could do this? To form a line, and hence work out the voltage from that line, I would need those point coordinates. Thanks

That's going to be a little tough as waypoints are at a significantly different level of code than your hardware device is. I can see the attraction in being able to use the mission library to synthesise your lines, but it's all sorts of layering violations to do it. Might still be worth doing :-) Might I suggest to start with you simply project a line out from the vehicle's home position? That will let you get the sensor written and tested quickly and easily. Peter

[rishabsingh3003]

As per your suggestions, I projected a (1 km) line from the vehicle's home position. The simulated sensor detects where the line is (left or right) with respect to the vehicle and returns a voltage proportional to the distance of the sensor to the line.
I could not find a sensor that will directly tell you where the line is with respect to the vehicle, therefore a minimum of 2 Infrared Sensors (Or some other sensor to detect the line) would be needed on the far edges of the vehicle so that the line can be detected and kept between those two sensors.
After trying for some time, I could not figure out how to accurately place the two sensors on the far left and right edge of the vehicle. Therefore, I just simulated 1 sensor that would do the job stated above. I need a little help in this to make the simulation more accurate (i.e get the two sensors in place).
The voltage is then mapped and fed to AR_AttitudeControl::get_steering_out_rate() like @rmackay9 suggested. The throttle input is taken from the user. In the end it does work, but needs more work and certain parameters also need to be defined that can be changed by the user. Should I raise a PR or fix these issues first?
Here is my branch for the mode:
https://github.com/rishabsingh3003/ardupilot/tree/Line-Follower
To simulate, in sim_vehicle.py , type:

mode 17
arm throttle
rc 3 1700

A sample GIF of the simulation is posted below:

[rishabsingh3003]

@peterbarker The problems stated above were fixed. The vehicle now calculates the error between its location and the line between two sensors. However, as time goes on, the oscillations that the vehicle makes keeps on growing larger and larger in amplitude (That is, the oscillations become more and more aggressive as time goes on), which eventually makes the vehicle go far away from the line, where the sensor cannot detect it due to sensor range limitations. To make the vehicle travel smoothly and follow the line, some sort of controller would eventually be needed. Do you recommend I put a PID(or PI?) controller on the calculated error from the sensor?

[peterbarker]

@rishabsingh3003 The question on control is really one for @rmackay9 , I'm afraid.

OTOH, what you have here appears to be a working line-following implementation, which is awesome!

Just looked at the commit history, and it looks like it does need to be cleaned up a little - but it would be great to get some sort of PR happening so people can start to comment on your code easily. I think that demonstration gif you pasted in above shows that this is mergeable, so long as it doesn't impact other functionality. It's something you (and others in the future!) can improve upon!

One thing I've noticed is that you're going a git pull from master to update your code to the latest developments in master. That's not actually typical git workflow; instead you git rebase your changes onto master. Our master branch consists only of a linear timeline - any merge commits we have in our master branch are accidents. A git rebase -i on master should let you fix your current commit topology.

Additionally, we have a one-commit-per-directory policy; look through the history to see what I mean there. We have a tool to help split the top commit up: Tools/gittools/git-subsystems-split, so you can squash all of your commits together then run that tool to clean things up quickly.

If you're new to git - make sure you take an archive of your entire ardupilot git repository before playing massive games with git! Backups are cheap and fast.

If you'd like help with the cleanup, do let me know!