Fixing Graticule's "distance" for edge cases

Ever seen this error when using Graticule?

Numerical argument out of domain - acos

Similarly to the to_sql problem for some edge cases, Graticule::Distance::Spherical.distance (and possibly those of Graticule's other distance computation classes) is subject to Float rounding errors.

This can cause the above error, when the arc cosine of something slightly more than 1.0 is to be computed, e.g. for the (zero) distance between the same coordinates (applies only for some).

How to fix

So, similar to the SQL fix, we just force the value we compute the arc cosine of into [-1; 1] bounds.

Put this into a config/initializers/graticule_spherical_distance.rb initializer:

Graticule::Distance::Spherical.class_eval do
  def self.distance(from, to, units = :miles)
    from_longitude  = from.longitude.to_radians
    from_latitude   = from.latitude.to_radians
    to_longitude    = to.longitude.to_radians
    to_latitude     = to.latitude.to_radians

    input = [ [ (
      Math.sin(from_latitude) *
      Math.sin(to_latitude) +
      Math.cos(from_latitude) *
      Math.cos(to_latitude) *
      Math.cos(to_longitude - from_longitude)
    ), -1 ].max, 1 ].min
    Math.acos(input) * Graticule::Distance::EARTH_RADIUS[units.to_sym]
  end
end

Spec

Here is a spec to add to your application:

describe Graticule::Distance::Spherical do

  describe '.distance' do
    it 'should work for edge case zero-distance computation' do
      from = stub :latitude => BigDecimal('0.4991657542 5E2'), :longitude => BigDecimal('-0.1169188713 5E3')
      to = stub :latitude => BigDecimal('0.4991657542 5E2'), :longitude => BigDecimal('-0.1169188713 5E3')

      Graticule::Distance::Spherical.distance(from, to).should == 0
    end
  end

end
Arne Hartherz About 12 years ago