If you worked with a system Ruby before switching to RVM, this system Ruby will be in your way when you switch between projects with/without RVM.
It's hard to get rid of your system Ruby entirely, but you can tell RVM to just use a given Ruby by default, e.g.:
rvm --default use 1.8.7
You need to re-open existing terminals for the changes to take effect.
Note that this will not actually remove the ruby
package from your system, it just isn't used anymore.