To update your Rubygems to the latest available version, type the following:

 gem update --system

Note that you have a separate Rubygems installation for each Ruby version in your RVM or rbenv setup. Updating one does not update the others.

Ruby 1.8.7

If you are using Ruby 1.8.7 you cannot use the latest version of Rubygems. Type the following to get the latest version that is compatible with 1.8.7:

 gem update --system 1.8.30

Updating RubyGems for all installed Ruby versions

Because of occasional security issues in RubyGems binaries, you should keep them updated across all Rubies on your machines. [1]

Here is how to do that in batch.
Afterwards, list your installed versions to confirm it worked.

rbenv

Here is a one-line bash script that will update all, and fall back to 1.8.30 for Ruby 1.8.7:

for i in `rbenv versions --bare`; do rbenv shell $i; echo $i; if [ ${i:0:4} = 1.8. ]; then gem update --system 1.8.30; else gem update --system; fi; done; rbenv shell --unset;

You could also use rbenv-each for this.

RVM

Use RVM's rvm all do:

rvm all do gem update --system

As mentioned above, the latest version will not work on Ruby 1.8.7. Manually switch to all such versions and use the compatible version like this:

rvm use 1.8.7
rvm rubygems latest-1.8 --force

[1] Note that Ruby 1.8.7 can not use the latest version or it would throw a NoMethodError: undefined method source_index for Gem:Module. We use the latest compatible version instead.