Posted 4 months ago. Visible to the public. Repeats.

Automatically validating dependency licenses with LicenseFinder

Open-source software (OSS) is great. Anyone can use virtually any open-source code in their projects.

Well, it depends. Licenses can make things difficult, especially when you are developing closed-source software. Since some OSS licenses even require the employing application to be open-sourced as well (looking at you, GPL), you cannot use such software in a closed-source project.

To be sure on this, we have developed a project-level integration of Pivotal's excellent license_finder that validates all dependencies during a regular test run. It will protect you from accidentally adding libraries with problematic licenses.

Installation

  1. Add gem 'license_finder' to the develoment dependencies of your project and run bundle install

  2. Add spec/license_finder_spec.rb with

    Copy
    describe 'license_finder' do describe 'action_items' do before :all do # rubocop:disable RSpec/BeforeAfterAll @stdout, @stderr, @status = Open3.capture3('license_finder action_items') end it 'activates all expected package managers' do package_managers = @stdout.scan(/^LicenseFinder::(\w+).*is active/).flatten expect(package_managers).to contain_exactly 'Bundler', 'Please list all expected package managers here' end it 'has no dependencies with unapproved licenses' do expect(@status).to be_success, @stdout end end end

    The first spec ensures no package manager is accidentally skipped (especially in the fragile Bower integration). Whenever a library with an unapproved (or restricted) license is added to the project, the second spec will fail.

    Run the spec now and update its list of package managers.

  3. Optional: Create an alias to simplify running LicenseFinder.

    Copy
    echo "alias lf='bundle exec license_finder'" >> ~/.bash_aliases # Store alias source ~/.bash_aliases # Load stored aliases

Excluding dependency groups

LicenseFinder can exclude certain dependency groups from its report, for example development and test. This only works with certain package managers (among which Bundler and Yarn). You can do so with:

Copy
bundle exec license_finder ignored_groups add $GROUP --why $REASON

Running the "action items" report

LicenseFinder runs as part of the test suite. Run manually with

Copy
bundle exec license_finder

It will detect all known package managers automatically (among which Bundler and Yarn) and check the license of all dependencies.

This is how the output may look when all dependencies are approved:

Image

This is how it may look when dependencies need approval (it is also the output of the failing licenses spec):

Image

Read on to learn what to do about the output. All decisions are written to doc/dependency_decisions.yml. Please always state the reason for your decision with --why.

Permitting a license

When you are absolutely sure that a license can be used in any circumstances, you may permit it. Ask someone if you are not sure. Github license pages or tldrlegal.com give a good overview about a license's permissions, limitations and conditions.

Permit a license with

Copy
bundle exec license_finder permitted_licenses add $LICENSE --why $REASON

Dependencies with this license will automatically be considered approved, and will no more show up in the action items list. If you know there is a license that can never be used in your project, restrict it with

Copy
bundle exec license_finder restricted_licenses add AGPL --why 'Requires to open-source the whole application'

Approving a single library

Some libraries will bring a license that has not been white listed. If your usage is valid with regard to that library's license, you can approve a single library without generally white-listing its license. GPL-licensed libraries are a candidate for this.

Copy
bundle exec license_finder approvals add $DEPENDENCY --why $REASON

Setting the license when unknown

When a license is reported as "unknown", you need to detect it manually. Check if current versions sport a license. Check package.json, composer.json, README.

Once you know the license, tell LicenseFinder with

Copy
bundle exec license_finder licenses add $DEPENDENCY $LICENSE --why $LICENSE_FILE_URL

Hard-to-find licenses

  • Rubygem ntlm-http: bundle exec license_finder licenses add ntlm-http ruby --why https://github.com/trampoline/ntlm-http/blob/9a61eaf20bd93ba035c17a79df0e9c814ffe6f2e/lib/net/ntlm_http.rb

Adding a hidden dependency

Sometimes there may be a dependency the package manager does not know about, e.g. a copied tracking snippet or some nested dependency. Tell LicenseFinder with

Copy
bundle exec license_finder dependencies add $DEPENDENCY $LICENSE $VERSION --why $REASON

Use sparingly, as manually added dependencies are not updated automatically (obviously).

Remarks

  • License identifiers (e.g. LGPL-2.1) follow the SPDX definitions
  • Some license identifiers are followed by an asterisk. This denotes an inferred license.
  • While it is possible to approve specific versions only, we decided to approve libraries as a whole. This is to keep maintenance efforts to a minimum.
  • LicenseFinder also has a concept of decision inheritance. While we're using it in few projects only, we don't need this functionality.
  • You can specify a --who $NAME to each decision to state who made the decision. However, Git logs will usually reveal that.
  • Most "add" commands above offer a "list" command that prints all registered entries.
  • Use license_finder help [command] for help.
  • If you made a mistake (e.g. forgot to pass a reason), you can open doc/dependency_decisions.yml and edit the last entry.

A word on bower-rails

Bower-rails is a Bower wrapper that simplifies Rails integration. Unfortunately, it makes it harder to integrate Bower with LicenseFinder. To get things running, follow these steps:

  1. Symlink bower-rails's bower.json to the project root: ln -s vendor/assets/bower.json
  2. Create a .bowerrc file at the project root with
    Copy
    { "comment": "Together with the bower.json symlink, this file integrates bower-rails with license_finder.", "directory": "vendor/assets/bower_components" }
  3. Remember to add Bower to the expected package managers in spec/license_finder_spec.rb.

A word on vendor/asset-libs

Manually vendored libraries in vendor/asset-libs/ are not discovered by LicenseFinder. Convert each such library to a local Ruby gem by following these steps:

  1. Move all contents of vendor/asset-libs/$library-$version to a new sub-directory vendor/asset-libs/$library-$version/lib/assets/javascripts. That's simply a path sprockets will consider, it is no problem if it contains non-JS files.
  2. Create $library-$version/local-$library.gemspec with
    Copy
    Gem::Specification.new do |s| s.name = 'local-$library' s.summary = 'local copy of $library' s.homepage = $homepage s.version = $version s.licenses = [$license] s.authors = [$author] s.files = Dir.chdir(__dir__) { Dir.glob('**/*') } s.add_runtime_dependency 'rails', '< 100' # Silence warnings by adding version end
    This is a minimal gemspec.
  3. Create $library-$version/lib/local-$library.rb with
    Copy
    module Local$Library class Engine < ::Rails::Engine end end
    Having the gem contain a Rails engine activates Sprockets, which will retrieve assets from the gem.
  4. Rename $library-$version to local-$library
  5. Find a suitable section in your Gemfile, then add
    Copy
    gem 'local-$library', path: 'vendor/asset-libs/local-$library'

Finally, remove the asset path customization from config/initializers/assets.rb:

Copy
-# Load asset libs folders -Rails.application.config.assets.paths += Dir[Rails.root.join('vendor/asset-libs/*')].sort_by { |dir| -dir.size }

makandra has been working exclusively with Ruby on Rails since 2007. Our laser focus on a single technology has made us a leader in this space.

Owner of this card:

Avatar
Dominik Schöler
Last edit:
about 1 month ago
by Dominik Schöler
Attachments:
approval_required.png, approved.png
About this deck:
We are makandra and do test-driven, agile Ruby on Rails software development.
License for source code
Posted by Dominik Schöler to makandra dev
This website uses short-lived cookies to improve usability.
Accept or learn more