OWL working principle

[gnaegit]

I have a question concerning the general working principle of the OWL. If I get it right the different algorithms detect weed in a picture. And then the location of the weed in the picture is used to decide which nozzles to turn ON? If so, is only one dimension of the picture used to decide which nozzle to turn on or is also the dimension in the direction of travel used to decide when to turn ON the nozzle?
Could the OWL be adapted such that it could be used together with a robot arm which has a tool mounted? I guess this would require that the location of the weed can be estimated from the image coordinates and that the location of the weed can then be transmitted to the robot arm to move to the weed location?

[geezacoleman]

That's correct, determining, happens at this step on lines 378 - 288 of owl.py:

 # loop over the ID/weed centres from contours
for ID, centre in enumerate(weedCentres):
    # if they are in activation region the spray them
    if centre[1] > self.yAct:
        sprayTime = time.time()
        for i in range(self.nozzleNum):
            # determine which lane needs to be activated
            if int(self.laneCoords[i]) <= centre[0] < int(self.laneCoords[i] + self.laneWidth):
                # log a spray job with the controller using the nozzle, delay, timestamp and spray duration
                # if GPS is used/speed control, delay can be updated automatically based on forward speed
                self.controller.receive(nozzle=i, delay=delay, timeStamp=sprayTime, duration=sprayDur)

It just uses the 'weedCentre', or the coordinate of the centre of the bounding box, to determine which nozzle to activate. As you can see, there is the idea to use location information and speed to make the on/off more dynamic, but that hasn't been implemented just yet.

I worked on something similar previously, we used image coordinates to target weeds with a 3 axis CNC machine. A robot arm would be slightly more complex I imagine? The key thing you need to do here is map how each weedCentre coordinate in the image corresponds to its position in the real world. You would need to correct for lens distortions, convert the pixel coordinates to real world positions (we just used a simple FOV width/pixels to work out the width per pixel) and then move the arm. Doable, but slightly more complex.

[gnaegit]

Thanks for the fast response!

[gnaegit]

I think a feature like this would make the OWL more useful for tools that require a high precision and accuracy.
In the end it would not be that important if a robot arm or a 3 axis CNC machine would be used to move the tool as the problem could be solved separately and just the location would be needed as an input.

[geezacoleman]

You could definitely take weedCentres for this purpose, the issue is reliably translating pixel coordinates to on-ground coordinates.

What sort of feature did you have in mind?