Floating-Point Precision Calculator (IEEE 754 Float, Double, Half, bfloat16)
How coarse the number line is around any value in half, bfloat16, single or double precision: the gap to the next number, machine epsilon, reliable digits and the largest integer that is still exact.
Results
What this tool does
Floating-point numbers are spaced evenly only inside each power of two: the gap doubles every time the value does, but stays the same fraction of the value. That is why 0.1 + 0.2 is not exactly 0.3, why a 32-bit float stops counting whole numbers at 16 777 216, and why a JavaScript number is exact for integers only up to 2⁵³ (9 007 199 254 740 992). Above that the gap in a double is 2, then 4, then 8. Use the gap, not the number of digits shown, to judge how far a result can be trusted.
Formula
gap (ULP) = 2^(⌊log₂|x|⌋ − p) · ε = 2^−p · digits ≈ (p + 1) log₁₀ 2 · largest gap-free integer = 2^(p+1) (p = 10, 7, 23, 52 fraction bits)
Variables
| Symbol | Meaning | Unit |
|---|---|---|
fm | Number format | — |
x | Value to inspect | — |
U | Gap to the next representable number at that value | — |
R | That gap as a fraction of the value | — |
EP | Machine epsilon (gap just above 1) | — |
DG | Reliable decimal digits | — |
MI | Largest integer before whole numbers start to skip | — |
MV | Largest finite value | — |
Worked example
- Number format4
- Value to inspect0.1
- Gap to the next representable number at that value1.38778e-17
- That gap as a fraction of the value1.38778e-16
- Machine epsilon (gap just above 1)2.22045e-16
- Reliable decimal digits15.95
- Largest integer before whole numbers start to skip9.0072e+15
- Largest finite value1.79769e+308
Limitations
- The formula assumes ideal conditions: no friction losses, no air resistance and no efficiency losses unless you enter them.
- The calculation runs at full precision and only the display is rounded. If you copy an intermediate value and retype it, small differences can appear.