Problems with precision when using linalg.angle_between

[caiolaytynher]

Context

This might not be really a bug, but a consequence of floating point precision that the programmer has to pay attention to when using the procedure. However, in case this was uncaught and is actually undesired, I might as well show what problem I ran into.

PC and Odin specs

Odin: dev-2024-07:fc5ce30
OS: Windows 10 Professional (version: 22H2), build 19045.4651
CPU: AMD Ryzen 5 5500U with Radeon Graphics
RAM: 5989 MiB
Backend: LLVM 18.1.8

How I ran into it (you can skip this if you want)

I was doing a project with 2D boids and I was using linalg.angle_between to determine the angle between the direction that the boids were moving. However, suddenly some boids started to disappear for apparently no reason. After I checked for NaN values, I found out that really specific values for 2D vectors were causing the issue.

Expected Behavior

linalg.angle_between should give the correct angle values for any valid [2]f32.

Current Behavior

For some [2]f32 values, whose angle between them is close to $\pi$ rad, linalg.angle_between returns NaN.

Failure Information

linalg.angle_between calls for math.dot internally after calling linalg.normalize0 on both [2]f32s. For some specific values, math.dot overshoots 1 because of floating point imprecision, which causes a problem on the subsequent call for math.acos, that returns NaN as there's no arc whose $\cos$ is greater than 1.

Steps to Reproduce

Just run the following:

import "core:fmt"
import "core:math/linalg"

main :: proc() {
	v := [2]f32{46.856773, -17.434246}
	u := [2]f32{-46.856297, 17.442307}
	fmt.println(linalg.angle_between(v, u)) // NaN
}

[Kelimion]

Fixed. Clamped the dot before handing it to acos.