Normally you can list all gems of the current ruby version with gem list, which also includes the gems of you Gemfile. These can be uninstalled with gem uninstall gemname.

List and uninstall a gem installed via Bundler from Github

Growing Rails Applications in Practice

Check out our e-book. Learn to structure large Ruby on Rails codebases with the tools you already know and love.

• Introduce design conventions for controllers and user-facing models
• Create a system for growth
• Build applications to last

Read more Show archive.org snapshot

This does not work for gems installed directly from Github. They do not appear in gem list.

Show all gems installed via Github by bundler:

ls ~/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/

Remove a gem installed via Github by Bundler:

rm -rf ~/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/bundler/gemname-xyz
rm -rf ~/.rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/cache/bundler/gemname-1.2.3

This might also be true for any other gem installed via self defined Git source and not standard Rubygems.