Upgrade Rails 101 Workshop

This is a companion page to the Rails upgrade workshop at RailsConf 2019. Here you will find a series of hands-on exercises to get started.

The main goal for this workshop is to define a roadmap to upgrade a Rails application. For all exercises we will use a sample application.

1. Pre-Requisites

Next Step

Here are the pre-requisites to participate in the upgrade rails workshop:

A laptop with Internet connection
A ready-to-go development environment.
Git, Docker, and `docker-compose`
Ruby 2.3.8 or higher
Successfully set up our sample Rails application: Instructions here

2. Sample Outdated Rails App

Previous Step Next Step Back to Top

We will work with this sample application: e-petitions -- an open source application to track petitions in the UK.

If you don't have Docker, please install it. Instructions here: Docker Installation

To make sure it is installed:

docker --version
        Docker version 18.09.2, build 6247962

If you don't have `docker-compose`, please install it. Instructions here: Docker Compose Installation

To make sure it is installed:

docker-compose --version
        docker-compose version 1.24.1, build unknown

Once you have installed `docker-compose` and `Docker`, you can clone the repository:

git clone https://github.com/fastruby/e-petitions
        cd e-petitions
        git fetch origin docker:docker
        git checkout docker

Follow these steps to install the application in your local environment:

docker-compose up -d db cache
        docker-compose build
        docker-compose run app ./bin/docker-setup

To make sure that everything worked fine, let's run a few specs...

3. Run the Test Suite

Previous Step Next Step Back to Top

At this point your services should be ready to go. If they are not, please raise your hand so that we can sort it out.

Let's run a part of the test suite like this:

docker-compose run app rspec spec/models

Test execution will take about 4 minutes.

4. Check Test Coverage

Previous Step Next Step Back to Top

Our sample application already has `simple_cov` as a dependency:

group :development, :test do
          gem 'simplecov'
          gem 'brakeman', '~> 4.5.1', require: false
          gem 'bundler-audit', require: false
          gem 'rspec-rails'
          gem 'jasmine-rails'
          gem 'pry'
        end

Let's open `spec_helper.rb` to see how `SimpleCov` gets loaded:

# spec/spec_helper.rb

        if ENV['COVERAGE'] == 'true'
          require 'simplecov'
          SimpleCov.start 'rails'
        end

You will need to pass an environment variable to `docker-compose`. You can do it like this:

docker-compose run -e COVERAGE=true app rspec spec/models/

Depending on your computer, this will take between 4 and 8 minutes. Once it's done, let's open the generated report in `coverage/index.html`. If you're on a Mac, you can do this:

open coverage/index.html

If we go to the Models tab we will see that models are very well covered (more than 80% test coverage!)

5. Bundle Outdated

Previous Step Next Step Back to Top

The next step to create the upgrade roadmap is to get an idea of how outdated our application really is by running this command:

docker-compose run app bundle outdated
        Starting e-petitions_cache_1 ... done
        Starting e-petitions_db_1    ... done
        Fetching gem metadata from https://rubygems.org/..........
        Fetching gem metadata from https://rubygems.org/.
        Resolving dependencies..........

        Outdated gems included in the bundle:
          * actionmailer (newest 5.2.3, installed 4.2.11.1)
          * actionpack (newest 5.2.3, installed 4.2.11.1)
          * actionview (newest 5.2.3, installed 4.2.11.1)
          * activejob (newest 5.2.3, installed 4.2.11.1)
          ...

While `bundle outdated` is quite useful, it doesn't give us details relative to our own `Gemfile`. We want to know the answers to these questions: How outdated is our application? Is it 50% or 1% out of date?

Fortunately there is another Ruby gem we can use: `next_rails`. Let's add it to our Gemfile:

group :development, :test do
          gem 'next_rails'
          # ...
        end

Then we will need to rebuild:

docker-compose down --volumes
        docker-compose build
        docker-compose run app ./bin/docker-setup

Let's check it out! It should give us a good idea about the shape of our dependencies:

docker-compose run app bundle_report outdated
        Starting e-petitions_db_1    ... done
        Starting e-petitions_cache_1 ... done
        cucumber-rails 1.4.0: released almost 6 years ago (latest version, 1.7.0, released 3 months ago)
        rack-test 0.6.3: released over 4 years ago (latest version, 1.1.0, released about 1 year ago)
        cucumber-wire 0.0.1: released over 3 years ago (latest version, 1.0.0, released 10 months ago)
        dalli 2.7.6: released over 3 years ago (latest version, 2.7.10, released 4 months ago)
        cucumber-core 1.5.0: released about 3 years ago (latest version, 4.0.0, released 10 months ago)
        cucumber 2.4.0: released about 3 years ago (latest version, 3.1.2, released about 1 year ago)
        arel 6.0.4: released over 2 years ago (latest version, 9.0.0, released over 1 year ago)
        pg 0.20.0: released over 2 years ago (latest version, 1.1.4, released 7 months ago)
        gherkin 4.1.3: released about 2 years ago (latest version, 6.0.17, released 4 months ago)
        textacular 5.0.1: released about 2 years ago (latest version, 5.1.0, released about 1 year ago)
        authlogic 3.6.1: released almost 2 years ago (latest version, 5.0.2, released 3 months ago)
        factory_bot 4.8.2: released almost 2 years ago (latest version, 5.0.2, released 5 months ago)
        factory_bot_rails 4.8.2: released almost 2 years ago (latest version, 5.0.2, released 3 months ago)
        database_cleaner 1.6.2: released over 1 year ago (latest version, 1.7.0, released over 1 year ago)
        rails-dom-testing 1.0.9: released over 1 year ago (latest version, 2.0.3, released about 2 years ago)
        daemons 1.2.6: released over 1 year ago (latest version, 1.3.1, released 7 months ago)
        tzinfo 1.2.5: released over 1 year ago (latest version, 2.0.0, released 7 months ago)
        i18n 0.9.5: released over 1 year ago (latest version, 1.6.0, released 5 months ago)
        paperclip 5.3.0: released over 1 year ago (latest version, 6.1.0, released 12 months ago)
        rack 1.6.11: released 9 months ago (latest version, 2.0.7, released 4 months ago)
        capybara 3.13.2: released 6 months ago (latest version, 3.26.0, released 12 days ago)
        childprocess 1.0.1: released 6 months ago (latest version, 2.0.0, released 16 days ago)
        shoulda-matchers 4.0.1: released 5 months ago (latest version, 4.1.1, released 12 days ago)
        concurrent-ruby 1.1.5: released 5 months ago (latest version, 0.7.2, released over 4 years ago)
        activesupport 4.2.11.1: released 5 months ago (latest version, 5.2.3, released 4 months ago)
        actionview 4.2.11.1: released 5 months ago (latest version, 5.2.3, released 4 months ago)
        actionpack 4.2.11.1: released 5 months ago (latest version, 5.2.3, released 4 months ago)
        activejob 4.2.11.1: released 5 months ago (latest version, 5.2.3, released 4 months ago)
        actionmailer 4.2.11.1: released 5 months ago (latest version, 5.2.3, released 4 months ago)
        activemodel 4.2.11.1: released 5 months ago (latest version, 5.2.3, released 4 months ago)
        activerecord 4.2.11.1: released 5 months ago (latest version, 5.2.3, released 4 months ago)
        railties 4.2.11.1: released 5 months ago (latest version, 5.2.3, released 4 months ago)
        rails 4.2.11.1: released 5 months ago (latest version, 5.2.3, released 4 months ago)
        bundler 1.17.3: released 4 months ago (latest version, 2.0.2, released about 1 month ago)
        webdrivers 3.8.1: released 3 months ago (latest version, 4.1.1, released 9 days ago)
        brakeman 4.5.1: released 3 months ago (latest version, 4.6.1, released 3 days ago)
        public_suffix 3.1.1: released about 1 month ago (latest version, 4.0.0, released about 1 month ago)
        aws-sdk-core 2.11.320: released 3 days ago (latest version, 3.61.1, released 2 days ago)
        aws-sdk-resources 2.11.320: released 3 days ago (latest version, 3.50.0, released 16 days ago)
        aws-sdk 2.11.320: released 3 days ago (latest version, 3.0.1, released almost 2 years ago)

        0 gems are sourced from git
        40 of the 139 gems are out-of-date (29%)

We now know that the dependencies are 29% out of date.

6. Bundle Update

Previous Step Next Step Back to Top

Now we are going to change _fixed dependencies_ into _pessimistic dependencies_ so that we get the latest patch releases into our app. Fortunately there is only one dependency that is fixed:

gem 'dalli', '2.7.6'

We should try to change it to be like this:

gem 'dalli', '~> 2.7.10'

After this change, you want to rebuild once again:


        docker-compose down --volumes
        docker-compose build
        docker-compose run app ./bin/docker-setup
        docker-compose run app rspec spec/models

Did it work? If that didn't work, you should write an item in your TODO list or create a user story in your upgrade project board. Why didn't it work? What failure did you get? Let's write up an item for our TODO list.

After that, make sure you roll back the change and rebuild your app image.


        docker-compose down --volumes
        docker-compose build
        docker-compose run app ./bin/docker-setup

7. Dual Boot: Setup

Previous Step Next Step Back to Top

We are going to use a helper tool for dual booting: `next_rails`. You should have it in your environment already.

docker-compose run app next --init
        Created Gemfile.next (a symlink to your Gemfile). Your Gemfile has been modified to support dual-booting!

        There's just one more step: modify your Gemfile to use a newer version of Rails using the `next?` helper method.

        For example, here's how to go from 4.2.x to 5.0:

        if next?
          gem "rails", "4.2.11.1"
        else
          gem "rails", "~> 5.0.1"
        end

`next --init` created a symlink called `Gemfile.next` and added a helper method to the top of our `Gemfile`.

Now we can open our `Gemfile` and verify that we have a `next?` method defined in it. In the odd chance it didn't work, we can try doing it manually like this:

docker-compose run app ln -s Gemfile Gemfile.next

Then we can add this method to the top of your `Gemfile`:

def next?
          File.basename(__FILE__) == Gemfile.next
        end

Now we have all we need to start tweaking our `Gemfile` for the next version of Rails:

if next?
          gem 'rails', '~> 5.0.1' # our target version
        else
          gem 'rails', '4.2.11.1' # our current version
        end

Before we start bundling two versions of our `Gemfile`, we can do some prep work. `next_rails` provides an interesting command that we could run to find known compatibility issues:

docker-compose run app bundle_report compatibility --rails-version=5.0.0
        Starting e-petitions_db_1    ... done
        Starting e-petitions_cache_1 ... done
        => Incompatible with Rails 5.0.0 (with new versions that are compatible):
        These gems will need to be upgraded before upgrading to Rails 5.0.0.

        rails-dom-testing 1.0.9 - upgrade to 2.0.3

        => Incompatible with Rails 5.0.0 (with no new compatible versions):
        These gems will need to be removed or replaced before upgrading to Rails 5.0.0.

        test_after_commit 1.2.2 - new version, 1.2.2, is not compatible with Rails 5.0.0

        2 gems incompatible with Rails 5.0.0

Now we know there will be two gems that will cause problems. We can add these gems to our TODO list.

It's time to `bundle install` using the next version. We can use the `next` command like this:

docker-compose run app next bundle install

If that doesn't work, you can try with:

docker-compose run -e BUNDLE_GEMFILE=Gemfile.next app bundle install

If that worked, great! If you run into unexpected issues or don't want to go down the rabbit hole, raise your hand and I can come and help.

Now that we have _two_ Gemfiles we will need to tweak our `Dockerfile`. Make sure that the last part of the `Dockerfile` looks like this:

COPY Gemfile Gemfile.lock ./
        COPY Gemfile.next Gemfile.next.lock ./

        RUN gem install bundler -v=1.17.3

        RUN bundle install
        RUN next bundle install

        COPY . .

With those extra lines we make sure that rebuilding our Docker image installs dependencies for both Gemfiles. Now let's rebuild them once again:

docker-compose down --volumes
        docker-compose build
        docker-compose run app ./bin/docker-setup

8. Dual Boot: spec/models

Previous Step Next Step Back to Top

Now that we have bundled our project with the next version of Rails, we can start running the test suite.

Now we can get ready to run into a bunch of errors:

docker-compose run app next rspec spec/models

We will probably find a problem with `after_commit` that will look like this:

Bundler::GemRequireError:
        There was an error while trying to load the gem 'test_after_commit'.
        Gem Load Error is: after_commit testing is baked into rails 5, you no longer need test_after_commit gem

The good news is that we have everything we need to fix this problem for both Rails 4.2.x and Rails 5.0+. We can tweak our `Gemfile` to look like this:

group :test do
          # ...
          gem 'test_after_commit' unless next?
        end

That will skip the `test_after_commit` dependency when using Rails 5.0. We will need to make sure that no references to `TestAfterCommit` are present in our codebase:

# spec/support/after_commits.rb

        if defined?(TestAfterCommit)
          TestAfterCommit.enabled = false

          RSpec.configure do |config|
            config.around(:each) do |example|
              if example.metadata.key?(:with_commits)
                TestAfterCommit.with_commits(example.metadata[:with_commits]) do
                  example.run
                end
              else
                example.run
              end
            end
          end
        end

Once we fixed that problem we are probably going to find another one related to `hide_action`:

An error occurred while loading ./spec/models/archived/rejection_spec.rb.
            Failure/Error: require File.expand_path('../../config/environment', __FILE__)

            NoMethodError:
              undefined method `hide_action' for Delayed::Web::ApplicationController:Class
            # ./config/initializers/delayed_web.rb:10:in `block in '
            # ./config/initializers/delayed_web.rb:7:in `class_eval'
            # ./config/initializers/delayed_web.rb:7:in `'

A good way to solve that would be to make sure that `hide_action` is replaced with a `protected` statement. Something like this: https://github.com/alphagov/e-petitions/commit/fac05d457b8fe4fec6b981674eea5973bace1022

Now we should be finally ready to run all model specs.

docker-compose run app next rspec spec/models

As we process the output (sample) from the test suite, we should consider a couple of things:

1. Every deprecation warning that we find should become a new story in our roadmap. For example:


        DEPRECATION WARNING: Passing `ActiveRecord::Base` objects to `sanitize_sql_hash_for_assignment`
        (or methods which call it, such as `update_all`) is deprecated. Please pass the
        id directly, instead. (called from block in invalidate! at /usr/src/app/app/models/signature.rb:605)

2. Every failure we find should become a new story. When creating the story we should write down the potential root cause. For example:


        rake aborted!
        Bundler::GemRequireError: There was an error while trying to load the gem 'test_after_commit'.
        Gem Load Error is: after_commit testing is baked into rails 5, you no longer need test_after_commit gem
        Backtrace for gem load error is:
        /Users/etagwerker/.rvm/gems/ruby-2.4.5@petitions/gems/test_after_commit-1.1.0/lib/test_after_commit.rb:4:in `'
        /Users/etagwerker/.rvm/rubies/ruby-2.4.5/lib/ruby/site_ruby/2.4.0/bundler/runtime.rb:81:in `require'
        /Users/etagwerker/.rvm/rubies/ruby-2.4.5/lib/ruby/site_ruby/2.4.0/bundler/runtime.rb:81:in `block (2 levels) in require'
        /Users/etagwerker/.rvm/rubies/ruby-2.4.5/lib/ruby/site_ruby/2.4.0/bundler/runtime.rb:76:in `each'

Some of the failures we encounter might have a simple solution, some might take us hours or days to fix. That's why we won't fix all the issues we find today.

To find the root cause of a test failure, we will need to check the changes between Rails versions. We can find the official guides over here: https://edgeguides.rubyonrails.org/upgrading_ruby_on_rails.html

We can find the unofficial guides over here:

Another useful resource is RailsDiff. We can see what changed between Rails 4.2 and Rails 5.0.

9. Dual Boot: spec/lib

Previous Step Next Step Back to Top

Now that we have tracked all the issues with our models, we can move on to the lib specs. We can run them like this:

docker-compose run app next rspec spec/lib

We should get an output like this one: sample.log. The specs pass and all the issues we see are related to changes in the Rails configuration. Let's create an item in our TODO list for each of them. If you find a couple occurrences of something like this:

DEPRECATION WARNING: Passing strings or symbols to the middleware builder is deprecated, please change
        them to actual class references.  For example:

          "CloudFrontRemoteIp" => CloudFrontRemoteIp

         (called from  at /usr/src/app/config/environment.rb:5)
        DEPRECATION WARNING: Passing strings or symbols to the middleware builder is deprecated, please change
        them to actual class references.  For example:

          "QuietLogger" => QuietLogger

         (called from  at /usr/src/app/config/environment.rb:5)
        DEPRECATION WARNING: Passing strings or symbols to the middleware builder is deprecated, please change
        them to actual class references.  For example:

          "ActionDispatch::RemoteIp" => ActionDispatch::RemoteIp

         (called from  at /usr/src/app/config/environment.rb:5)

We should create one item in our TODO list, not three.

10. Dual Boot: spec/controllers

Previous Step Next Step Back to Top

Now that we have tracked all the issues with our libraries, we can move on to the controller specs. We can run them like this:

docker-compose run app next rspec spec/controllers

We should get an output like this one: sample.log. The specs pass and all the issues we see are related to changes in the Rails configuration. Let's create an item in our TODO list for each of them. We will find a couple occurrences of something like this:

Failure/Error: expect(assigns[:signature]).to be_persisted

        NoMethodError:
          assigns has been extracted to a gem. To continue using it,
                  add `gem 'rails-controller-testing'` to your Gemfile.
        # /usr/local/bundle/gems/actionpack-5.0.7.2/lib/action_dispatch/testing/test_process.rb:27:in `assigns'
        # ./spec/controllers/sponsors_controller_spec.rb:392:in `block (6 levels) in '
        # ./spec/controllers/sponsors_controller_spec.rb:368:in `block (6 levels) in '
        # /usr/local/bundle/gems/activesupport-5.0.7.2/lib/active_support/testing/time_helpers.rb:110:in `travel_to'
        # ./spec/controllers/sponsors_controller_spec.rb:368:in `block (5 levels) in '
        # ./spec/support/database_cleaner.rb:10:in `block (3 levels) in '
        # /usr/local/bundle/gems/database_cleaner-1.6.2/lib/database_cleaner/generic/base.rb:16:in `cleaning'
        # /usr/local/bundle/gems/database_cleaner-1.6.2/lib/database_cleaner/base.rb:98:in `cleaning'
        # /usr/local/bundle/gems/database_cleaner-1.6.2/lib/database_cleaner/configuration.rb:86:in `block (2 levels) in cleaning'
        # /usr/local/bundle/gems/database_cleaner-1.6.2/lib/database_cleaner/configuration.rb:87:in `cleaning'
        # ./spec/support/database_cleaner.rb:9:in `block (2 levels) in '

We can quickly fix this by adding `rails-controller-testing` to our Gemfile:

group :test do
          gem 'rails-controller-testing' if next?
          # ...
        end

In many cases (like this one) some errors will hide other errors. We won't be able to see the real errors until we patch our project to run with the next version of Rails. You will have to rebuild your `app` image:

docker-compose down --volumes
        docker-compose build
        docker-compose run app ./bin/docker-setup

Your controllers spec output will be polluted with a lot of these deprecation warnings:

DEPRECATION WARNING: Using positional arguments in functional tests has been deprecated,
        in favor of keyword arguments, and will be removed in Rails 5.1.

        Deprecated style:
        get :show, { id: 1 }, nil, { notice: "This is a flash message" }

        New keyword style:
        get :show, params: { id: 1 }, flash: { notice: "This is a flash message" },
          session: nil # Can safely be omitted.
         (called from block (4 levels) in  at /usr/src/app/spec/controllers/admin/debate_outcomes_controller_spec.rb:29)
        .DEPRECATION WARNING: Using positional arguments in functional tests has been deprecated,
        in favor of keyword arguments, and will be removed in Rails 5.1.

We can create a single item in our TODO list to deal with this before making the jump to Rails 5.1. The good news is that we can fix this problem with Rubocop! :)

We will need to add `rubocop` and `rubocop-rails` as two new dependencies:

# Gemfile
        # ...
        group :development, :test do
          # ...
          gem 'rubocop'
          gem 'rubocop-rails'
        end

We will need some basic configuration that specifies what is our target Rails version:

# .rubocop.yml

        AllCops:
          # What version of Rails is the inspected code using?  If a value is specified
          # for TargetRailsVersion then it is used.  Acceptable values are specificed
          # as a float (i.e. 5.1); the patch version of Rails should not be included.
          # If TargetRailsVersion is not set, RuboCop will parse the Gemfile.lock or
          # gems.locked file to find the version of Rails that has been bound to the
          # application.  If neither of those files exist, RuboCop will use Rails 5.0
          # as the default.
          TargetRailsVersion: 5.0
          # When specifying style guide URLs, any paths and/or fragments will be
          # evaluated relative to the base URL.
          StyleGuideBaseURL: https://rails.rubystyle.guide

And then we can just run `rubocop` with the `--auto-correct` flag:

rubocop --only Rails/HttpPositionalArguments --require rubocop-rails spec/controllers  --auto-correct
        Inspecting 47 files
        .CCCCCCCCCCC.CCCCCCCCCCCCCCCCCCCCCCC.CC.CCC.CCC

        Offenses:

        spec/controllers/admin/admin_users_controller_spec.rb:87:9: C: [Corrected] Rails/HttpPositionalArguments: Use keyword arguments instead of positional arguments for http call: post.
                post :create, :admin_user => admin_user_attributes
                ^^^^
        spec/controllers/admin/admin_users_controller_spec.rb:138:9: C: [Corrected] Rails/HttpPositionalArguments: Use keyword arguments instead of positional arguments for http call: get.
                get :edit, :id => edit_user.to_param
                ...
                ...
                ...

Now you can try running your spec controllers once again:

docker-compose run app next rspec spec/controllers

11. Stay Current

Previous Step Next Step Back to Top

Deprecation warnings in the latest versions of Rails have been quite helpful in guiding us to the next version of Rails. We should treat all new deprecation warnings as exceptions:

# config/environments/test.rb

        Rails.application.configure do
          # ...

          # Raise on deprecation notices
          config.active_support.deprecation = :raise

          # ...
        end

That way we can turn _noise_ into many _signals_. Deprecation warnings can easily be ignored when they're just a message in `test.log`. They can't be easily ignored if they break your test suite. That's why I recommend you do this.

We should add `bundler-audit` as a dependency and make sure to run a check every time you run your test suite. We can do this by tweaking your `Rakefile`:

# Rakefile

        require File.expand_path('../config/application', __FILE__)
        require 'rake'

        Rails.application.load_tasks

        task default: %i[
          bundle:audit brakeman:check
          spec spec:javascripts cucumber
        ]

Now everytime you run `bundle exec rake`, it will not only run your test suite but also run `bundle:audit` which does this:

namespace :bundle do
          desc "Audit bundle for any known vulnerabilities"
            task :audit do
              unless system "bundle-audit check --update"
                exit 1
              end
            end
          end
        end

Finally you can keep up with the latest version by using the Rails `master` branch:

# Gemfile

        source 'https://rubygems.org'

        git_source(:github) do |repo_name|
          repo_name = "#{repo_name}/#{repo_name}" unless repo_name.include?("/")
          "https://github.com/#{repo_name}.git"
        end

        if next?
          gem 'rails', github: 'rails/rails'
        else
          gem 'rails', '~> 6.0.0.rc1'
        end

You can find instructions on how to set this up in your CI server over here: https://fastruby.io/blog/upgrade-rails/dual-boot/dual-boot-with-rails-6-0-beta.html

At this point you should have a solid list of items in your roadmap. You can start addressing them one by one with many, tiny pull requests.

12. Questions?

Previous Step Next Step Back to Top

If you have any questions, feel free to raise your hand and ask. If you had to leave the workshop, you are welcome to reach out to me in the _hallway track_ or via Twitter: @etagwerker

13. Thank you!

Previous Step Back to Top

You are wonderful! Thanks for participating in my workshop. I hope you can apply all of this in your next Rails upgrade project! :)