285 Frontend build pipelines in Rails [3d]

While working on a Rails application, your code base will grow a collection of different file types including:

• Ruby (business logic)
• HTML fragments (layouts and views)
• CSS/Sass/SCSS (styles)
• JavaScript (client-side behavior)
• Static media (images, fonts, videos)

Except for the Ruby part, all these files are shipped to the browser. In this card we'll focus specifically on CSS, JS, fonts and static media files which are often summed up as assets.

You might already have noticed that Rails modifies those assets before delivering them to the client. For example, you'll never see SASS anywhere in your browser. Rails offers various asset processing and delivery mechanisms which will be covered in this card.

Important

Work on this lesson in advisor mode.

Learning goals

• You can explain what a frontend build pipeline does — bundling, transpiling, fingerprinting, postprocessing — and why the browser never sees your source files as you wrote them.
• You can explain fingerprinting and cache headers, and how one asset can reference another (e.g. an image from a stylesheet) without knowing its fingerprint.
• You can work with our pipeline, esbuild: read its configuration, add an entrypoint, import files and npm packages, add a Sass or image asset, and find the build output.
• You can add and update JavaScript libraries with our package manager (yarn 1) and explain what package.json and the lock file are for.
• You can find your way around the legacy pipelines of older projects — the asset pipeline (Sprockets) and Webpacker — well enough to see where files live and how they are imported.
• You can explain what the alternatives to esbuild trade off, and why we don't use them by default.

Overview: frontend management systems

Over time, we adapted various asset management systems. You already got to know esbuild, which is used in your MovieDB and our most recent applications. This is our preferred solution and should be your main focus while working on this card.

As we maintain more than a hundred applications, many of them still use the asset pipeline and Webpacker. You should roughly understand how they work, but don't invest more than a day to skim over the asset pipeline and webpacker.

Rails 1-7 with the Asset Pipeline (Sprockets)

The asset pipeline is Rails' original mechanism on how stylesheets, javascripts and images from your /assets folder are processed and delivered to the browser

• Used in all our projects from 2009-2019, for example makandracards
  • Because of its simplicity, we might still use it in projects with a very narrow scope
• Has no separate build process in development. Instead the Rails server directly compiles the asset whenever it renders a javascript_tag, stylesheet_tag or image_tag.
• It still has issues on integrating well with vendor libraries or the npm ecosystem:
  • import and export statements (ESM) are not supported
  • because of this, you can only add libraries that are packed in a single file and register themselves globally on the window object
  • Even though ES6-Support required for ESM Show archive.org snapshot has been added with sprockets v4, the asset pipeline still has issues with the support of many JavaScript libraries
• Postprocessing is mostly limited to minification, transpilation is not possible
• Understand by looking at an example project and resources below:
  • What is sprockets Show archive.org snapshot and how does it relate to the asset pipeline?
  • What does require, require_tree, require_self do?
  • Why are image-url and font-url necessary for sass urls?
  • When importing variables or mixins into a .sass file, we need to use @import other_file.
    • Note that we cannot use //= require 'other_file'
      • //= require is a sprockets statement that would copy and duplicate the content file in your bundle each time you use it.
    • only SASS @imports are aware of the current state of your variable definitions
• Have a look at the goals section above focusing on the asset pipeline

Rails 5+ with Webpacker

Webpacker has many more moving parts than the asset pipeline, but allowed us to use ES6 modules and npm packages (through yarn Show archive.org snapshot ) a lot earlier than it was introduced for sprockets. Even after that sprockets is still missing many modern bundling features and has issues with library support. Webpacker is a wrapper around a JavaScript project called webpack Show archive.org snapshot

• Used in all our projects from 2019-2022, for example Studyflix
  • Rails has ended its support for further development Show archive.org snapshot
  • Studyflix no longer uses Webpacker, but plain Webpack
• It comes with a powerful configuration / transformation system and offers many plugins
  • Because of this, it ticks every box for a desirable frontend management system
  • It's also slow and very complex. This makes it hard to debug any issues or understand the pipeline as a whole.
• Understand by looking at an example project and resources below:
  • What "packs" are
  • How to use the webpack dev server
  • How to add and remove npm packages with yarn
  • How import and export of relative paths and npm packages work (ES6 module system)
• Have a look at the goals section above focusing on webpacker

Rails 6+ with esbuild

esbuild offers us similar features like webpacker while being blazingly fast and easier to understand.

• Used in all our projects since 2022, for example your MovieDB
• It currently offers less plugins in comparison to Webpacker, but we're very happy with this solution
• Understand how sprockets integrates with esbuild
• Have a look at the goals section above focusing on esbuild.

Rails 7 with import maps Show archive.org snapshot and jsbundling Show archive.org snapshot

Import maps let you import JavaScript modules using logical names. You can build modern JavaScript applications using ESM libraries without the need for transpiling or bundling. This frees you from needing Webpack and Yarn.

• This is an experimental solution that could become the default in future Rails versions.
• It lacks any transpilation/transformation and bundling features. They reason that bundling is no longer necessary with HTTP/2, but in practice we still need some form of minification etc.
• Don't spend more than 15 minutes looking at this approach as we're currently not using it. You should mainly know that it exists.

Resources

Read what's new to you, skim what's familiar, skip what you already master. Stop when you can meet the learning goals.

Your agent can also generate an overview, a tutorial or an explanation for anything here, tailored to what you already know. Just ask.

Our pipeline: esbuild

• 📄 esbuild: getting started Show archive.org snapshot and the API Show archive.org snapshot — bundling, transpiling, loaders, entry points
• 📄 jsbundling-rails Show archive.org snapshot and cssbundling-rails Show archive.org snapshot — how Rails wires a bundler like esbuild into bin/dev and the asset helpers
• 📄 How to make your application assets cachable in Rails — our card; fingerprinting and cache headers
• 📄 JavaScript start-up optimization: network Show archive.org snapshot — why bundle size and count matter
• 📄 yarn 1 documentation Show archive.org snapshot — the package manager movie-db-base uses
• ▶️ Tschüss Webpacker Show archive.org snapshot (2021) and esbuild Show archive.org snapshot (2022, notes) — internal talks, in our library

History and legacy pipelines

• 📄 The assets pipeline history Show archive.org snapshot — Sprockets → Webpacker → bundlers, in one article
• 📄 Rails Guide (7.0): The Asset Pipeline Show archive.org snapshot — Sprockets, as you will meet it in older projects
• 📄 Webpack(er): a primer — our card; and Webpacker has been retired Show archive.org snapshot

Alternatives

• 📄 Propshaft Show archive.org snapshot and importmap-rails Show archive.org snapshot — the Rails 8 default; Rails Guide: The Asset Pipeline Show archive.org snapshot now describes this setup
• 📄 Modern web apps without JavaScript bundling or transpiling Show archive.org snapshot — DHH's case for no-build
• 📄 Vite Ruby Show archive.org snapshot — the Vite integration for Rails

Alternatives

We build with esbuild; you will meet these elsewhere:

• Propshaft, import maps, "no build" — the Rails default since Rails 7/8: no bundling, no transpiling, browsers load ES modules directly. Poor developer experience for our needs (no Sass, no JSX, awkward debugging); we don't use it.
• Sprockets / the asset pipeline — the complete Ruby-based solution; legacy projects only.
• Vite (via vite-ruby) — a bundler comparable to esbuild with a richer plugin ecosystem; some teams prefer it.
• pnpm — a faster, stricter package manager; we use it alongside yarn 1 in some projects.

See Our Rails stack for the reasoning behind these choices.