When Rails Ejected Node and the History of Compiled JS in Rails

I was with Hampton Lintorn-Catlin when he asked the Rails core team about whether or not SASS, a hugely popular 14-year old CSS preprocessor engine that Hampton was co-inventor of, if the future of Rails & SASS together would include a permanent switch to adopt the Javascript-based (NPM) compiler (newer + slower). The C++ based alternative (older but faster), was based on an implementation being switched away from in favor of another based on the Dart compiler.

The rationale, Hampton argued, was that the core maintainers of lib-sass, a project in which Hampton was not currently involved, had since switched preferences for the Dart compiler version, as it would be easier to support by the Node community in the future. It was too difficult to find a C++ developer to maintain the old dependencies.

The answer came down from atop the Rails team: A decided no.

Why? We do not want a situation where SASS compiling — an essential and core part of our daily workflow— is dependent on Javascript. This was a show-stopper for DHH and the future of Rails.

Let’s back up.

The History of Javascript Dependencies in Ruby on Rails

History of Sprockets aka The Asset Pipeline

Sprockets is a Ruby library for compiling and serving web assets. Sprockets allowes us to organize an application’s JavaScript files into smaller more manageable chunks that could be distributed over a number of directories and files. It provides structure and practices on how to include assets in our projects.

Sprockets, also known as ‘the asset pipeline,’ was first introduced in 2011 as part of Rails 3.1.

Then, in 2014, Sprockets was extracted out of Rails into its own Gem with the release of Rails 4.0 (a gem called sprockets-rails). To be clear, you were still encouraged to use Sprockets, but Rails 4 now shipped with jQuery + CoffeeScript as the default and Sprockets as opt-in using the sprocket-rails gem. It did not come installed by default with a new Rails app.

Using directives at the start of each JavaScript file, Sprockets can determine which files a JavaScript file depends on. When it comes to deploying your application, Sprockets then uses these directives to turn your multiple JavaScript files into a single file for better performance.

Sprockets looks like this

/app/assets/javascripts/application.js

// This is a manifest file that'll be compiled into including all the files listed below.
// Add new JavaScript/Coffee code in separate files in this directory and they'll automatically
// be included in the compiled file accessible from http://example.com/assets/application.js
// It's not advisable to add code directly here, but if you do, it'll appear at the bottom of the
// the compiled file.
//
//= require jquery
//= require jquery_ujs
//= require_tree .

Sprockets required us to define a manifest file for each part of our app.

Introduce “ES6” aka ECMAScript 2015

In the dark times of Javascript, everything in Javascript application was smashed together in global namespaces — often called jQuery soup.

You would use jQuery to pick at the DOM via targeting. Things lived in global namespaces. Parts of the codebase had no separation, composability, or dependency management. Like I said, they were dark times.

Several early pioneers took computer science ideas from other languages and made attempts at making Javascript into a well-encapsulated language. These implementations to make JS into a better-structured language were called things like Object literal pattern, IFFE (“iffy”), CommonJS, and AMD (Asynchronous Module Definition).

Then, ES6 Modules become the de-facto way to write modern JS: Defining encapsulated functionality within a module, exporting it explicitly, and then importing the functionality in the places where you want to use it.

You can get a quick dive into ES Modules, once referred to as ES6 Modules, here.

What’s important about the innovation of ES Modules is it meant that different parts of our Javascript codebase could be cleanly structured to load different libraries or another part of our app.

The only problem was, the old paradigms of loading Javascript in Rails (at this point the year is about 2016) — rely on a manifest-based list of what Javascript libraries you will need. What most apps did was segregate parts of their apps into sections — a frontend, for example, and an admin interface. Each section of the app would have a specific named manifest file, and “only” the Javascript dependencies needed for that part of the app were loaded.

This worked, to a point and in theory. In practice, many people just loaded huge asset pipeline files and didn’t notice how much JS they were loading into their browser.

Enter Webpack & Webpacker

In 2019, with Rails 6, the Rails core defaults adopted something called webpack: a Node tool for transpiling Javascript. You needed to transpile your modern Javascript because many browsers didn’t, at that time, support the modern syntaxes you were using.

A Ruby extension named webpacker provided the glue between Ruby and webpack.

Webpacker and webpack solved an important problem but introduced new ones. Like Sprockets, Webpack created bundles that used thumbprint digests to expire them.

With webpack we divided our Javascript into multiple bundles. What was also important about Webpack is that it transiled our Javascript from modern syntax to backward-compatible syntax. During the transpilation, Webpack took care of mapping our Import/Export statements to the places where they are defined.

Although it fits nicely in with the Javascript Import/Export modules, it can have a steep learning curve for Rails developers.

The Overpacking Problem

As discussed in this excellent post many apps suffered (and still do today) from an overpack problem.

The Speed, or Lack Thereof

One of the significant hurdles for Rails developers adopting Webpack is how slow it is to make changes. A good tool called web-dev-server allows you to rebuild CSS quickly while developing, but still running this extra overhead seemed like another thing to have to worry about to the Rails core team.

Also, having webpack in the build pipelines turns out to be slow. There then becomes a lot of work to deal with this slowness in your pipeline.

When Internet Explorer still existed and before browsers widely supported HTTP2 and ImportMap, this was all necessary. However, now that IE is dead and the browser support HTTP2, the story has changed.

Enter ImportMap-Rails

DHH’s explained his reasons for switching away from Babel and transpiling with Node in a blog post from Aug 12, 2021. As discussed above, digest-based manifest files required expiring the entire manifest file for every change, creating a slow development workflow.

Two key advancements in browser technology:

1. The universal adoption of ES Modules across all modern browsers obviates the need for transpiling with bundler. Of course, if you still are targeting old browsers you would still need a bundler. But with support ES Modules in all browsers Babel transpiling is no longer needed.
2. In the dark days of HTTP’s 1st standard, every time you made a web request you had to do something time consuming called negotiating an SSL connection. Each one of these negotations add a small overhead to your load time. That’s why we compiled Javascript into one big pack: So that your browser could make one single request and get the Javascript it needs at once. HTTP2 changed all of that. Now, the browser server can keep the connection alive and the client can continue requsting resources without paying the SSL overhead penalty.

This giant leap forward in browser technology is only because of very recent advancements.

ImportMaps rely on this feature to load Javascript dynamically at runtime based on what imports are mapped to specific keywords.

ImportMap is a part of the HTTP spec. importmap-rails is a gem that now ships with Rails 7 to switch the Javascript paradigm away from Bundler, Webpack, and Node.

They are supported natively in Chrome and Edge browsers and work via a shim for Firefox and Safari. Fortunately, the Rails core team has packed the shim into the importmap-rails gem which comes by default with new Rails 7 apps.

Enter cssbundling-rails

As well, 2022 saw the entry of a new player in CSS bundling options for Rails: cssbundling-rails. This is an augmentation to Sprockets and makes the following changes to your Rails app.

1. app/assets/builds is now where application.js and applicatoin.css files live

2. these hold your bundled output as artifacts that are not checked into source control

To install it, you have one of two options:

1. Start your new Rails 7 project with --css=bootstrap (as in rails new MyGreatApp --css=bootstrap)
2. Don’t start with --css=bootstrap and instead Add gem 'cssbundling-rails' to your Gemfile and then install it with rails css:install:bootstrap

Or, you can install any of Tailwind, Bulma, PostCSS, or SASS either.

If you use the cssbundling-rails installer, the installer adds this directory to .gitignore by default.

Goodbye Node & Node Packages

Will Rails ever bring Webpack back by default? Probably not. Do you need to listen to the Rails core team and eject Node yourself? Absolutely not.

For one thing, building a React frontend packaged inside of your Rails app — still a great deployment option — still will require transpiling for the time being. Be sure to read my guide Rails 7 : Do I need ImportMap-rails to understand if you need or want Importmap-Rails or JSBundling.