Posted over 9 years ago. Visible to the public.

Fixing Graticule's "distance" for edge cases

Ever seen this error when using Graticule?

Numerical argument out of domain - acos

Similarly to the to_sql problem for some edge cases, Graticule::Distance::Spherical.distance (and possibly those of Graticule's other distance computation classes) is subject to Float rounding errors.

This can cause the above error, when the arc cosine of something slightly more than 1.0 is to be computed, e.g. for the (zero) distance between the same coordinates (applies only for some).

How to fix

So, similar to the SQL fix, we just force the value we compute the arc cosine of into [-1; 1] bounds.

Put this into a config/initializers/graticule_spherical_distance.rb initializer:

Graticule::Distance::Spherical.class_eval do def self.distance(from, to, units = :miles) from_longitude = from.longitude.to_radians from_latitude = from.latitude.to_radians to_longitude = to.longitude.to_radians to_latitude = to.latitude.to_radians input = [ [ ( Math.sin(from_latitude) * Math.sin(to_latitude) + Math.cos(from_latitude) * Math.cos(to_latitude) * Math.cos(to_longitude - from_longitude) ), -1 ].max, 1 ].min Math.acos(input) * Graticule::Distance::EARTH_RADIUS[units.to_sym] end end


Here is a spec to add to your application:

describe Graticule::Distance::Spherical do describe '.distance' do it 'should work for edge case zero-distance computation' do from = stub :latitude => BigDecimal('0.4991657542 5E2'), :longitude => BigDecimal('-0.1169188713 5E3') to = stub :latitude => BigDecimal('0.4991657542 5E2'), :longitude => BigDecimal('-0.1169188713 5E3') Graticule::Distance::Spherical.distance(from, to).should == 0 end end end

Your development team has a full backlog of feature requests, chores and refactoring coupled with deadlines? We are familiar with that. With our "DevOps as a Service" offering, we support developer teams with infrastructure and operations expertise.

Owner of this card:

Arne Hartherz
Last edit:
over 9 years ago
corner, case
About this deck:
We are makandra and do test-driven, agile Ruby on Rails software development.
License for source code
Posted by Arne Hartherz to makandra dev
This website uses short-lived cookies to improve usability.
Accept or learn more