Exploring floating-point precision 01 October 2017
ε. The fifth letter of the Greek alphabet. In calculus, an arbitrarily small positive quantity. In formal language theory, the empty string. In the theory of computation, the empty transition of an automaton. In the ISO C Standard,
1.19e-07 for single precision and
2.22e-16 for double precision.
The other day I was attempting to use
FLT_EPSILON (which I later learned was laughably incorrect) when the Swift 4 compiler emitted a warning saying that
FLT_EPSILON is deprecated and to use
.ulpOfOne instead. What the hell is
ulpOfOne? I read the documentation and then everything made sense — ha, just kidding. The
FloatingPoint.ulpOfOne docs generously describe the static variable as the unit in the last place of 1.0 — whatever that means. Let’s find out.