Read more

Using Rationals to avoid rounding errors in calculations

Martin Schaflitzl
November 20, 2023Software engineer at makandra GmbH

Ruby has the class Rational Show snapshot which allows you to store exact fractions. Any calculation on these variables will now use fractional calculations internally, until you convert the result to another data type or do a calculation which requires an implicit conversion.

Example use case:

Illustration online protection

Rails Long Term Support

Rails LTS provides security patches for old versions of Ruby on Rails (2.3, 3.2, 4.2 and 5.2)

  • Prevents you from data breaches and liability risks
  • Upgrade at your own pace
  • Works with modern Rubies
Read more Show snapshot

Lets say you want to store the conversion factor from MJ to kWh in a variable, which is 1/3.6. Using BigDecimals for this seems like a good idea, it usually helps with rounding errors over a float, but there are cases where you still have to worry about rounding errors.

conversion_factor = BigDecimal('1') / BigDecimal('3.6')
# => 0.277777777777777777777777777777777778e0

Here the conversion factor got rounded after some decimal places. If you now use this value now to convert a Value from MJ to kWh it can lead to ugly rounding errors.

BigDecimal('7.2') * conversion_factor
# => 0.20000000000000000000000000000000000016e1

This might be OK for you as the value is still very accurate. When you later round the value, e.g. for displaying it might not be an issue, but can be annoying to deal with during development and in tests.

To avoid this use Rationals

conversion_factor = Rational('1') / Rational('3.6')
# => (5/18)
result = Rational('7.2') * conversion_factor
# => (2/1)
# => 0.2e1


The 0 in result.to_d(0) is the required precision parameter Show snapshot , which is applied to the BigDecimal return value.

Using Rationals has some pitfalls you should be aware of:

  • Multiplying a Rational with a BigDecimal still produces rounding errors.

    BigDecimal('7.2') * (Rational('1') / Rational('3.6'))
    # => 0.20000000000000000016e1
  • Rationals don't like floats very much, when initializing a Rational with a float you don't get accurate values.

    float_rational = Rational(0.1)
    # => (3602879701896397/36028797018963968)
    # => 0.100000000000000005551115123125782702e0
Posted by Martin Schaflitzl to makandra dev (2023-11-20 16:11)