Posted over 9 years ago. Visible to the public.

# Fixing Graticule's "distance" for edge cases

Ever seen this error when using Graticule?

```Copy```Numerical argument out of domain - acos
``````

Similarly to the `to_sql` problem for some edge cases, `Graticule::Distance::Spherical.distance` (and possibly those of Graticule's other distance computation classes) is subject to `Float` rounding errors.

This can cause the above error, when the arc cosine of something slightly more than 1.0 is to be computed, e.g. for the (zero) distance between the same coordinates (applies only for some).

## How to fix

So, similar to the SQL fix, we just force the value we compute the arc cosine of into [-1; 1] bounds.

Put this into a `config/initializers/graticule_spherical_distance.rb` initializer:

```Copy```Graticule::Distance::Spherical.class_eval do
def self.distance(from, to, units = :miles)

input = [ [ (
Math.sin(from_latitude) *
Math.sin(to_latitude) +
Math.cos(from_latitude) *
Math.cos(to_latitude) *
Math.cos(to_longitude - from_longitude)
), -1 ].max, 1 ].min
end
end
``````

## Spec

```Copy```describe Graticule::Distance::Spherical do

describe '.distance' do
it 'should work for edge case zero-distance computation' do
from = stub :latitude => BigDecimal('0.4991657542 5E2'), :longitude => BigDecimal('-0.1169188713 5E3')
to = stub :latitude => BigDecimal('0.4991657542 5E2'), :longitude => BigDecimal('-0.1169188713 5E3')

Graticule::Distance::Spherical.distance(from, to).should == 0
end
end

end
``````

Your development team has a full backlog of feature requests, chores and refactoring coupled with deadlines? We are familiar with that. With our "DevOps as a Service" offering, we support developer teams with infrastructure and operations expertise.

Last edit:
over 9 years ago
Keywords:
corner, case