When Rails Ejected Node and the History of Compiled JS in Rails

I was with Hampton Lintorn-Catlin when he asked the Rails core team about whether or not SASS, a hugely popular 14-year old CSS preprocessor engine that Hampton was co-inventor of, if the future of Rails & SASS together would include a permanent switch to adopt the Javascript-based (NPM) compiler (newer + slower). The C++ based alternative (older but faster), was based on an implementation being switched away from in favor of another based on the Dart compiler.

The rationale, Hampton argued, was that the core maintainers of lib-sass, a project in which Hampton was not currently involved, had since switched preferences for the Dart compiler version, as it would be easier to support by the Node community in the future. It was too difficult to find a C++ developer to maintain the old dependencies.

The answer came down from atop the Rails team: A decided no.

Why? We do not want a situation where SASS compiling — an essential and core part of our daily workflow— is dependent on Javascript. This was a show-stopper for DHH and the future of Rails.

Let’s back up.

The History of Javascript Dependencies in Ruby on Rails

History of Sprockets (also known as “The Asset Pipeline)

Sprockets is a Ruby library for compiling and serving web assets. Sprockets allows us to organize an application’s JavaScript files into smaller, more manageable chunks that can be distributed over a number of directories and files. It provides structure and practices on how to include assets in our projects.

Sprockets, also known as ‘the asset pipeline,’ was first introduced in 2011 as part of Rails 3.1.

Then, in 2014, Sprockets was extracted from Rails into its own Gem with the release of Rails 4.0 (a gem called sprockets-rails). To be clear, you were still encouraged to use Sprockets, but Rails 4 now shipped with jQuery + CoffeeScript as the default and Sprockets as opt-in using the sprocket-rails gem. It did not come installed by default with a new Rails app.

Using directives at the start of each JavaScript file, Sprockets can determine which files a JavaScript file depends on. When it comes to deploying your application, Sprockets then uses these directives to turn your multiple JavaScript files into a single file for better performance.

Sprockets looks like this

/app/assets/javascripts/application.js

// This is a manifest file that'll be compiled into including all the files listed below.
// Add new JavaScript/Coffee code in separate files in this directory and they'll automatically
// be included in the compiled file accessible from http://example.com/assets/application.js
// It's not advisable to add code directly here, but if you do, it'll appear at the bottom of the
// the compiled file.
//
//= require jquery
//= require jquery_ujs
//= require_tree .

Sprockets required us to define a manifest file for each part of our app.

Introduce “ES6” aka ECMAScript 2015

In the dark times of Javascript, everything in the Javascript application was smashed together in global namespaces — often called jQuery soup.

You would use jQuery to pick at the DOM via targeting. Things lived in global namespaces. Parts of the codebase had no separation, composability, or dependency management. Like I said, they were dark times.

Several early pioneers took computer science ideas from other languages and made attempts at making Javascript into a well-encapsulated language. These implementations to make JS into a better-structured language were called things like Object literal pattern, IFFE (“iffy”), CommonJS, and AMD (Asynchronous Module Definition).

Then, ES6 Modules become the de-facto way to write modern JS: Defining encapsulated functionality within a module, exporting it explicitly, and then importing the functionality in the places where you want to use it.

You can get a quick dive into ES Modules, once referred to as ES6 Modules, here.

What’s important about the innovation of ES Modules is it means that different parts of our Javascript codebase could be cleanly structured to load different libraries or another part of our app.

The only problem was, the old paradigms of loading Javascript in Rails (at this point the year is about 2016) — rely on a manifest-based list of what Javascript libraries you will need. What most apps did was segregate parts of their apps into sections — a frontend, for example, and an admin interface. Each section of the app would have a specific named manifest file, and “only” the Javascript dependencies needed for that part of the app were loaded.

This worked, to a point and in theory. In practice, many developers just loaded huge asset pipeline files and didn’t notice how much JS they were loading into their browsers.

Enter Webpack & Webpacker

In 2019, with Rails 6, the Rails core defaults adopted a tool from the JS world called webpack. Webpack is a Node tool for transpiling Javascript. You needed to transpile your modern Javascript (React JSX and Typescript) because many browsers didn’t support the modern syntaxes you were using at that time.

A Ruby extension named webpacker provided the glue between Ruby and webpack.

Webpacker and webpack solved an important problem but introduced new ones. Like Sprockets, Webpack created bundles that used thumbprint digests to expire them.

With webpack we divided our Javascript into multiple bundles. What was also important about Webpack is that it transpiled our Javascript from modern syntax to backward-compatible syntax. During the transpiling, Webpack mapped our Import/Export statements to the places where they were defined.

Although Webpack fit nicely with the Javascript Import/Export modules, it can have a steep learning curve for Rails developers, creates dependency hell, and makes build times extremely slow.

The Overpacking Problem

As discussed in this excellent post many apps suffered (and still do today) from an overpack problem.

The Speed, or Lack Thereof

One of the significant hurdles for Rails developers adopting Webpack is how slow it is to make changes. A good tool called web-dev-server allows you to rebuild CSS quickly while developing, but still running this extra overhead seemed like another thing to worry about to the Rails core team.

Also, having webpack in the build pipelines turns out to be slow. There then becomes much work to deal with this slowness in your pipeline.

When Internet Explorer still existed and before browsers widely supported HTTP2 and ImportMap, this was all necessary. However, now that IE is dead and the browser supports HTTP2, the story has changed.

Enter ImportMap-Rails

DHH’s explained his reasons for switching away from Babel and transpiling with Node in a blog post from Aug 12, 2021. As discussed above, digest-based manifest files require expiring the entire manifest file for every change, creating a slow development workflow.

Two key advancements in browser technology:

1. The universal adoption of ES Modules across all modern browsers obviates the need for transpiling with bundler. Of course, if you still are targeting old browsers you would still need a bundler. But with support ES Modules in all browsers Babel transpiling is no longer needed.
2. In the dark days of HTTP’s 1st standard, every time you made a web request you had to do something time consuming called negotiating an SSL connection. Each one of these negotations add a small overhead to your load time. That’s why we compiled Javascript into one big pack: So that your browser could make one single request and get the Javascript it needs at once. HTTP2 changed all of that. Now, the browser server can keep the connection alive and the client can continue requsting resources without paying the SSL overhead penalty.

This giant leap forward in browser technology is only because of very recent advancements.

ImportMaps relies on this feature to load Javascript dynamically at runtime based on what imports are mapped to specific keywords.

ImportMap is a part of the HTTP spec. importmap-rails is a gem that now ships with Rails 7 to switch the Javascript paradigm away from Bundler, Webpack, and Node.

They are supported natively in Chrome and Edge browsers and work via a shim for Firefox and Safari. Fortunately, the Rails core team has packed the shim into the importmap-rails gem which comes by default with new Rails 7 apps.

Enter cssbundling-rails

As well, 2022 saw the entry of a new player in CSS bundling options for Rails: cssbundling-rails. This augments Sprockets and makes the following changes to your Rails app.

1. app/assets/builds is now where application.js and applicatoin.css files live

2. these hold your bundled output as artifacts that are not checked into source control

To install it, you have one of two options:

1. Start your new Rails 7 project with --css=bootstrap (as in rails new MyGreatApp --css=bootstrap)
2. Don’t start with --css=bootstrap and instead Add gem 'cssbundling-rails' to your Gemfile and then install it with rails css:install:bootstrap

Or, you can install any of Tailwind, Bulma, PostCSS, or SASS either.

If you use the cssbundling-rails installer, the installer adds this directory to .gitignore by default.

Goodbye Node & Node Packages

Will Rails ever bring Webpack back by default? Probably not. Do you need to listen to the Rails core team and eject Node yourself? Absolutely not.

For one thing, building a React frontend packaged inside of your Rails app — still a great deployment option — still will require transpiling for the time being. Be sure to read my guide Rails 7 : Do I need ImportMap-rails to understand if you need or want Importmap-Rails or JSBundling.