Migrate From Webpacker to Jsbundling-Rails With Esbuild

We’ve come a long way as full-stack Rails engineers. We began with everything neatly bundled in one framework, handling both backend and frontend together. The asset pipeline, with Sprockets opens a new window , kept this ecosystem running smoothly for us. Then, as front-end complexity grew, we turned to Webpacker opens a new window to pick up where Sprockets left off.

Now, it’s time for the next step in our journey: moving from Webpacker to esbuild. In this post, we’ll explore how to make that transition and why it’s worth considering.

Journey to JS the Rails 5 Way
jsbundling-rails with esbuild
Migrating to jsbundling-rails with esbuild
Some gotchas
Considerations before migrating to esbuild
Conclusion
Further reading and references

Journey to JS the Rails 5 Way

The frontend evolved at a breakneck pace, and soon browser technologies couldn’t keep up. To bridge this gap, tools like npm/yarn, Webpack, and Babel entered the scene, offering modern solutions for bundling and compiling JavaScript beyond what the asset pipeline could handle.

Around this time, some developers jumped ship from full-stack Rails and opted for a Rails API paired with a single-page application (SPA) frontend. But the progressive Rails community had a different vision, introducing Webpacker. This gave us a way to stay loyal to the “majestic monolith” by integrating Webpack directly into Rails.

With Rails 5.2, Webpacker became an alternative JavaScript compiler, and by Rails 6, it had fully replaced Sprockets as the default choice for JavaScript. The Rails way now involved using Webpack for JavaScript while leaving everything else to Sprockets in the asset pipeline.

While Webpacker made Webpack easier to configure in Rails, it came with its own set of issues. Fast-forward a few years, and things have changed a lot since 2017. Newer alternatives have emerged as browser technology improved, making the complexity and friction of Webpacker less appealing.

Rails 7 introduces a new default for JavaScript, following the Rails doctrine opens a new window of flexibility and offering alternatives when the default doesn’t suit. Now Rails uses import maps opens a new window as the default way to include JavaScript.

But if we need to transpile or compile JavaScript - say we’re working with .jsx, .ts, or .vue files - import maps won’t meet our needs (at least, not yet). So, are we stuck on the Webpacker train indefinitely? Thankfully, the answer is a resounding no! In the rest of this article, we’ll explore how jsbundling-rails opens a new window with esbuild opens a new window can be our next “get-off” point.

jsbundling-rails with esbuild

The jsbundling-rails gem gives us everything we need to configure and use a JavaScript bundler of our choice. Essentially, jsbundling-rails sets up a few rake tasks that create an entry point and define the final build path.

In theory, we’re not tied to any one bundler: jsbundling-rails works as long as we stick to the expected entry point and output the bundled files to app/assets/builds.

One great choice here is esbuild, a highly performant JavaScript bundler. Esbuild’s core version doesn’t have every feature Webpacker does, and some features will likely never be added, as explained on its roadmap opens a new window .

For a Rails app with just a sprinkle of JavaScript, esbuild’s core functionality is more than enough to get the job done.

Migrating to jsbundling-rails with esbuild

Install jsbundling-rails

First we add the gem to our Gemfile.

+ gem 'jsbundling-rails'

Then we bundle install and run the following in the terminal:

./bin/rails javascript:install:esbuild

The installation script provides the default esbuild configuration which includes:

Updates to the .gitignore file
A procfile for running multiple processes with foreman
A app/assets/builds directory
Updates to the manifest.js
A app/javascript/application.js entry point file
A javascript include tag
A bin/dev script
Updates to the package.json dependencies

We need to add the build script to our package.json if it was not added by the installation script. So we add the following to the package.json file.

"scripts": { "build": "esbuild app/javascript/*.* --bundle --sourcemap --outdir=app/assets/builds"  }

This default build script will make it possible to compile our js assets by running yarn build.

Remove Webpack from any development scripts

Next, we need to remove Webpack references from any development scripts, including those in the Procfile and our bin directory. The installation script we ran earlier added yarn build --watch to our development Procfile, which is now the only build script needed to compile JavaScript.

Remove Webpacker tags

The installation script also added a JavaScript include tag to our application.html.erb file. This tag points to the new JavaScript entry point, javascript/application.js, which allows the build script to be included in our application.

Next, we’ll need to search the project for any instances of javascript_pack_tag, which Webpacker required, and remove them. These are no longer needed. Except for any layout file. In the layout files we need to replace the javascript_pack_tag with the new javascript_include_tag.

<%# Before %>
<%= javascript_pack_tag 'application' %>

<%# After %>
<%= javascript_include_tag "application", defer: true %>

Move entrypoint

Our new entry point is now at app/javascript/application.js. We’ll copy the contents from the old Webpacker entry point at app/javascript/packs/application.js to this new location.

It is important to convert require statements into import statements with relative paths. Paying close attention as we update the relative paths imported into the new application.js file.

// this
require("controllers")
require("@rails/ujs").start();


// becomes
import "./controllers"
import Rails from “@rails/ujs”
Rails.starts()

We can delete app/javascript/packs once everything has been moved to the app/javascript directory.

Remove Webpacker

Remove the following files, but remember to first add any required configuration to your esbuild build script. Take a look at adding an esbuild config file for insight into creating more complex esbuild configurations.

./bin/webpack
./bin/webpack-dev-server
./config/initializers/webpacker.rb
./config/webpacker.yml
./config/webpack/development.js
./config/webpack/environment.js
./config/webpack/production.js
./config/webpack/test.js

Remove the Webpacker gem in our Gemfile and bundle install afterwards.

- gem 'webpacker'

Finally we can also remove the Webpacker packages from our package.json.

yarn remove @rails/webpacker webpack-dev-server

Some gotchas

Including gem provided javascript

Including gem-provided JavaScript is a common practice in Rails applications. Gems like jquery-ujs opens a new window often provide JavaScript that adds functionality when specific data-* attributes opens a new window are used on elements. This setup works smoothly with Sprockets and the asset pipeline, but how do we include gem-provided JavaScript when using jsbundling-rails with esbuild? It’s straightforward; just follow these steps:

Create a New Entry Point: In the app/javascript directory, create a new file for these assets. Let’s call it sprockets.js.
Update manifest.js: Add the new entry point to manifest.js so it’s included in the asset pipeline.

// app/assets/config/manifest.js

//= link sprockets.js

Require the Gem-Provided JavaScript: In sprockets.js, require the JavaScript files from your gem(s).

// app/javascript/sprockets.js

//= require jquery
//= require jquery_ujs

Update javascript_include_tag: Finally, include the new entry point in your HTML by updating your javascript_include_tag methods.

<%= javascript_include_tag "application", "sprockets", defer: true %>

This way, you can easily include gem-provided JavaScript files even with jsbundling-rails and esbuild.

Including ERB in javascript files

In the past, even with Webpacker, we could include ERB in our JavaScript files. However, esbuild doesn’t support ERB in JavaScript files. Unlike Webpack, esbuild lacks a plugin system, which means we can’t use the erb-loader plugin that Webpacker provides.

To work around this, we can create a new entry point file that includes the ERB code and then import this entry point into our main JavaScript file. This approach is similar to the steps for including gem-provided JavaScript. Here’s how to set it up:

Create a New Entry Point: In the app/javascript directory, create a new entry point file for ERB, for example, sprockets.erb.js.
Update manifest.js: Add this new entry point to manifest.js so it’s included in the asset pipeline.
```
// app/assets/config/manifest.js
//= link sprockets.erb.js
```
Tell Sprockets to Precompile: In config/initializers/assets.rb, configure Sprockets to compile this new entry point file.
```
# config/initializers/assets.rb
Rails.application.config.assets.precompile += %w( sprockets.erb.js )
```
Add it to Your HTML: Update your javascript_include_tag methods to include this new entry point.

<%= javascript_include_tag "application", "sprockets", defer: true %>

Add ERB Code: Finally, add your ERB code to sprockets.erb.js as needed.

Optionally add a esbuild.config.mjs

The esbuild API can be accessed from the command line, in Javascript or in Go. The script we added to our package.json shows how to access esbuild from the command line. In some cases it might be more convenient to use Javascript or Go. We will briefly demonstrate how to use Javascript.

Our build script defined earlier in our package.json file included all build options on a single line. For more complex projects an external build script might be preferable.

We can convert the build script we saw earlier. It doesn’t really matter what we call this build script but it might make sense to call it esbuild.config.mjs. As per the documentation this code must be saved in a file with the .mjs extension because it uses the import keyword. This seems like a naming convention used by other javascript config files.

// esbuild.config.mjs

import * as esbuild from 'esbuild'

await esbuild.build({
  entryPoints: ['app/javascript/application.js'],
  bundle: true,
  sourcemap: true,
  outdir: 'app/assets/builds',
})

Notice that we don’t use the app/javascript/*.* entry point. This is because glob expansion is done by our shell and not by esbuild. We would need to include a library such as glob opens a new window to expand the glob pattern first before passing the paths to esbuild.

Now we can update our package.json build script to ”build”: “node esbuild.config.mjs”. And everything should work just like it did before.

You may want to review the esbuild API opens a new window if you have a more complex Webpack configuration that you wish to migrate to esbuild. Keep in mind that some Webpack features can only be accomplished with one of the esbuild plugins opens a new window .

Using jQuery

With Webpacker you might have made jQuery available globally by doing something like this:

environment.plugins.prepend(
  'Provide',
  new webpack.ProvidePlugin({
    $: 'jquery/src/jquery',
    jQuery: 'jquery/src/jquery'
  })
)

With esbuild we need to take a different approach to achieve the same result. We cannot simply import jquery like this:

// app/javascript/application.js
import “jquery”
window.jQuery = jquery
window.$ = jquery

The problem is that import statements get hoisted. So any import statement that requires jquery will be hoisted to execute before the window object assignments. This means that imported scripts that depend on jquery will execute before jquery is available.

Instead we create a new file, let us call it jquery.js.

// app/javascript/jquery.js
import jquery from “jquery”
window.jQuery = jquery
window.$ = jquery

And we import this file into our main entry point, application.js like so:

// app/javascript/application.js
import “./jquery”

Now all imports after the jquery import will have access to the jquery window object.

Using the global object

With Webpack we could assign a global variable to the global object. Webpack automatically converted this to the window object opens a new window .

If we need to maintain backward compatibility with the global object in Webpack, then we need to add the --define:global=window to our build command. However if backward compatibility is not required then we can very easily search and replace all instances of global with window. You could also modify the config file to include the define option.

// esbuild.config.mjs

import * as esbuild from 'esbuild'
import {sassPlugin} from 'esbuild-sass-plugin'

await esbuild.build({
  entryPoints: ['app/javascript/application.js'],
  bundle: true,
  sourcemap: true,
  outdir: 'app/assets/builds',
  define: {
    "global": 'window',
  }
})

Live reloading

As we have mentioned above esbuild does not support hot module reloading (HMR) opens a new window , it is not even on the roadmap. After being spoiled with HMR via Webpack for a while, this might be a developer luxury we don’t want to go without.

We may not be able to do HMR, but we can add configuration that enables live reloading. Live reloading essentially triggers a page reload whenever a watched file is updated.

For the best developer experience we would need to add Chokidar opens a new window from npm. Chokidar is a file watching library that allows us to watch for changes to any set of files we wish. Combined with the lightning speed of esbuild we can watch and trigger rebuilds with minimal delay.

First we might want to add Chokidar by running the following command in the terminal.

yarn add -D chokidar

Then we would add a file for our esbuild config. As we noted earlier this file’s name is not important. We could call it esbuild.config.mjs if it makes sense.

#!/usr/bin/env node

import esbuild from "esbuild";
import { createServer } from "http";
import chokidar from 'chokidar';

const clients = [];
const watching = process.argv.includes("--watch");
const watchedDirectories = [
  "./app/javascript/**/*.js",
  "./app/views/**/*.html.erb",
  "./app/assets/stylesheets/*.css",
];
const bannerJs = watching
  ? ' (() => new EventSource("http://localhost:8082").onmessage = () => location.reload())();'
  : "";

const config = {
  entryPoints: ['app/javascript/application.js'],
  bundle: true,
  sourcemap: true,
  outdir: path.join(process.cwd(), "app/assets/builds"),
  absWorkingDir: path.join(process.cwd(), "app/javascript"),
  banner: { js: bannerJs },
};

if (watching) {
  createServer((req, res) => {
    return clients.push(
      res.writeHead(200, {
        "Content-Type": "text/event-stream",
        "Cache-Control": "no-cache",
        "Access-Control-Allow-Origin": "*",
        Connection: "keep-alive",
      }),
    );
  }).listen(8082);

  (async () => {
     const result = await esbuild.build(config);
     chokidar.watch(watchedDirectories).on("all", (event, path) => {
       if (path.includes("javascript")) {
         console.log(`rebuilding ${path}`);
         result.rebuild();
       }
       clients.forEach((res) => res.write("data: update\n\n"));
       clients.length = 0;
     });
  })();
} else {
  esbuild.build(config).catch(() => process.exit(1));
}

We don’t want to dive too deeply into the esbuild configuration api, instead we will do a quick overview of this config file.

First we import the required libraries and we set a few variables.

The config contains the build options we pass to esbuild. The heart of this reloading technique is contained in the banner option. Here we specify that we would like to prepend a piece of Javascript to the build artifact. When the --watch flag is passed in, the additional javascript reloads the page whenever it receives a message from localhost:8082.

Next we have a conditional section. We check and if we are watching then we create a local web server using node’s http module. This server can send requests to the browser to trigger a page refresh.

Next we have an asynchronous self-invoking function that defines chokidar’s behavior. It watches for changes in the specified watch directories. It rebuilds when javascript files are updated. And finally it sends a message to the browser.

Now we need to update our build script in our package.json so that it looks like this:

  "scripts": {
    "build": "node esbuild-dev.config.mjs"
  }

Finally we want to make sure that our Procfile.dev passes in the watch option. The js section of this file must include the --watch option like this.

js: yarn build –-watch

We can now start the development server with ./bin/dev. And to make sure it works we can make a change to any of our javascript, css or erb files. We can watch more files by adding the appropriate directory to the watchDirectories variable.

Please take a look at this github issue for more information on adding live reload to the esbuild serve opens a new window .

Considerations before migrating to esbuild

Switching away from Webpacker isn’t required. There’s an actively maintained gem called Shakapacker, which is essentially a rebranded Webpacker. Internally, it still uses the name webpacker, making it easy to trial Shakapacker and see if it’s a good fit for your app.

jsbundling-rails works in conjunction with the asset pipeline, whether that’s Sprockets or Propshaft. So, we need to ensure that either Sprockets or Propshaft is included in our Gemfile.

One thing to keep in mind is that esbuild is currently in what’s considered a “late beta” phase, not yet at version 1.0.0. While a lot of effort goes into maintaining stability and backward compatibility, this may be a consideration if your application requires production-ready tools only.

Conclusion

There are several benefits to replacing Webpacker with jsbundling-rails and esbuild. Chief among them is speed: esbuild is significantly faster than Webpacker, especially in smaller applications. The chart below illustrates just how much quicker esbuild can be.

This should set you up for a strong finish! Let me know if you want to add more insights into the considerations or elaborate on the speed benefits.

Graph showing an example where esbuild builds 7 times faster than Webpack

Another reason we might want to migrate to esbuild is because it might be easier to use and maintain. Webpack is known for its flexibility. Esbuild is less flexible than Webpack by design. This is right up the Rails developer’s alley where convention over configuration has always been a valued characteristic of the tools we use.

The Esbuild community is growing and there are many community provided plugins and starter configuration scripts already available. Esbuild is also used in many other developer tools such as Amazon CDK opens a new window and Phoenix opens a new window .

Are you considering upgrading your Rails applications so you can also simplify the javascript part of your application? We are experts at upgrading Rails applications. Reach out opens a new window so we can help you evaluate the benefits of migrating to jsbundling-rails.

Journey to JS the Rails 5 Way

jsbundling-rails with esbuild

Migrating to jsbundling-rails with esbuild

Install jsbundling-rails

Remove Webpack from any development scripts

Remove Webpacker tags

Move entrypoint

Remove Webpacker

Some gotchas

Including gem provided javascript

Including ERB in javascript files

Optionally add a esbuild.config.mjs

Using jQuery

Using the global object

Live reloading

Considerations before migrating to esbuild

Conclusion

Further reading and references

Francois Buys

Listing all tags

Best Practices

Case Study

Compatibility

Devops

Javascript

Performance

RailsConf

Ruby

Rubyconf

Security

Technical Debt

Upgrades

You might be interested in

Migrate from webpacker to jsbundling-rails with esbuild

Journey to JS the Rails 5 Way

jsbundling-rails with esbuild

Migrating to jsbundling-rails with esbuild

Install jsbundling-rails

Remove Webpack from any development scripts

Remove Webpacker tags

Move entrypoint

Remove Webpacker

Some gotchas

Including gem provided javascript

Including ERB in javascript files

Optionally add a esbuild.config.mjs

Using jQuery

Using the global object

Live reloading

Considerations before migrating to esbuild

Conclusion

Further reading and references

Francois Buys

Share this post

Listing all tags

You might be interested in