Skip to content

thonzyk/float-precision-problems

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Float Precision Problems

Description float64 float16 Symbolic Term
Small Numbers 1.000e-08 0.000e+00 $10^{-8}$
Big Numbers 7.000e+04 inf $70000$
Long Sum 1.000e+04 9.992e+03 $\sum_{k=1}^{10^{5}} 0.1$
Catastrophic Cancellation 1.000e-04 0.000e+00 $1 - (1 - 10^{-4})$

About

A single minimal script showcasing common floating-point precision issues. It compares float64 and float16 results in NumPy for underflow, overflow, long summation, and catastrophic cancellation, producing a compact table that highlights where reduced precision fails.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors