How and Why to Measure Dependency Freshness in your Ruby Application
Whether you are working in a legacy Ruby application, or a brand new application, measuring your dependency freshness score can be a positive indicator to understand whether you are staying current or gradually falling out of date.
Dependency freshness is defined as the difference between the currently used version of a dependency and the version that the system would ideally use.
In this article, I will discuss a couple of ways to keep track of how outdated or how fresh your dependencies really are.
Context
One of the core services that we offer our clients is a slow and steady approach to pay off their technical debt .
Tech debt sometimes shows up as a set of dependencies that haven’t been upgraded in years. Other times it shows up as a set of really complicated files, which are constantly changing and lacking a stable test suite.
As consultants, we want to constantly show the value that we provide, so showing the progress we have made upgrading outdated dependencies is a really important responsibility in our day to day.
How to Measure Dependency Freshness
We have considered two different ways of assessing the progress we make upgrading dependencies:
- Measuring what percentage of your dependencies are out of date
- Measuring how many years behind you are in terms of dependencies (Libyears )
For this article, I will be using one of our open source projects, Skunk. Skunk is a command line tool that helps you detect which files are constantly changing, really complicated, and lack tests .
I will upgrade its minitest development dependency from 5.8.5 to 5.20.0. Then
I will show what progress looks like using the two different approaches.
Measuring Out-of-date Percentage
For this section, I’m going to use another open source tool that we maintain:
next_rails
Despite its name, it works with Ruby and Rails projects. Basically any project
that has a Gemfile.
When I run the bundler_report outdated command this is what I get:
$ bundle_report outdated
minitest 5.8.5: released Sep 26, 2016 (latest version, 5.20.0, released Sep 6, 2023)
simplecov-console 0.5.0: released May 24, 2019 (latest version, 0.9.1, released Feb 1, 2021)
codecov 0.1.21: released Aug 26, 2020 (latest version, 0.6.0, released Aug 18, 2021)
webmock 3.10.0: released Nov 12, 2020 (latest version, 3.19.1, released Aug 29, 2023)
vcr 6.1.0: released Mar 13, 2022 (latest version, 6.2.0, released Jun 26, 2023)
bundler 2.4.7: released Feb 15, 2023 (latest version, 2.4.22, released Nov 9, 2023)
regexp_parser 2.8.2: released Oct 10, 2023 (latest version, 2.8.3, released Dec 4, 2023)
rdoc 6.6.0: released Nov 6, 2023 (latest version, 6.6.1, released Dec 5, 2023)
irb 1.9.1: released Nov 21, 2023 (latest version, 1.10.1, released Dec 5, 2023)
json 2.7.0: released Dec 1, 2023 (latest version, 2.7.1, released Dec 5, 2023)
0 gems are sourced from git
10 of the 60 gems are out-of-date (17%)
In other words, my project’s dependencies are 83% up to date (or fresh).
Why Not bundle outdated?
Bundler includes a command called outdated which lists your dependencies in
a concise table showing your application’s current version and the latest
version available in Rubygems.
I love bundler and I’m thankful to its maintainers and contributors, but the
output of bundle outdated is not that useful for quickly understanding the
state of dependencies in a project. This is what it looks like:
$ bundle outdated
Fetching gem metadata from https://rubygems.org/........
Resolving dependencies...
Gem Current Latest Requested Groups
codecov 0.1.21 0.6.0 ~> 0.1.16 development
irb 1.9.1 1.10.1
json 2.7.0 2.7.1
minitest 5.8.5 5.20.0 ~> 5.8.4 development
rdoc 6.6.0 6.6.1
regexp_parser 2.8.2 2.8.3
simplecov-console 0.5.0 0.9.1 = 0.5.0 development
vcr 6.1.0 6.2.0 ~> 6.1.0 default
webmock 3.10.0 3.19.1 ~> 3.10.0 development
Sure, one by one you can go ahead and compare current and latest versions per dependency, but I usually work with legacy Rails applications that have 300+ dependencies, so it is quite time consuming to get insights from the output.
Measuring Libyears: How Many Years Behind are my Dependencies?
Libyears is a measure of the age of your dependencies. The age is calculated by the current version of your application and release date, the latest version of the dependency and release date, and the delta between those two release dates.
(Source: github.com/nasirhjafri/libyear )
For example: If I’m using Minitest v5.8.5 in my project and the latest stable release is v5.20.0, what’s the age of that dependency? It’s the number of days between the release of v5.8.5 and v5.20.0.
If you look at the release dates, then it turns out that my minitest
dependency is 6.9 years old (2023-09-06 minus 2016-09-26)
If you do that for every single dependency in your Gemfile, you get a total
amount that represents the libyears of your dependencies.
Fortunately Jared Beck created and maintains an open source tool called
libyear-bundler which you can use to get this number in a few seconds:
https://github.com/jaredbeck/libyear-bundler
$ libyear-bundler Gemfile
codecov 0.1.21 2020-08-26 0.6.0 2021-08-18 1.0
irb 1.9.1 2023-11-21 1.10.1 2023-12-05 0.0
json 2.7.0 2023-12-01 2.7.1 2023-12-05 0.0
minitest 5.8.5 2016-09-26 5.20.0 2023-09-06 6.9
rdoc 6.6.0 2023-11-06 6.6.1 2023-12-05 0.1
regexp_parser 2.8.2 2023-10-10 2.8.3 2023-12-04 0.2
simplecov-console 0.5.0 2019-05-24 0.9.1 2021-02-01 1.7
vcr 6.1.0 2022-03-13 6.2.0 2023-06-26 1.3
webmock 3.10.0 2020-11-12 3.19.1 2023-08-29 2.8
ruby 3.1.3 3.2.2 0.0
System is 14.0 libyears behind
This doesn’t mean that my project is 14 years behind. I’ve been updating it over the years, but I haven’t updated some of its dependencies and I’m missing the opportunity to upgrade them.
If you don’t care for this new metric, libyear-bundler also provides a
summary of the releases and versions and how behind your project is:
$ libyear-bundler --all
codecov 0.1.21 2020-08-26 0.6.0 2021-08-18 25 [0, 5, 0] 1.0
irb 1.9.1 2023-11-21 1.10.1 2023-12-05 2 [0, 1, 0] 0.0
json 2.7.0 2023-12-01 2.7.1 2023-12-05 2 [0, 0, 1] 0.0
minitest 5.8.5 2016-09-26 5.20.0 2023-09-06 30 [0, 12, 0] 6.9
rdoc 6.6.0 2023-11-06 6.6.1 2023-12-05 1 [0, 0, 1] 0.1
regexp_parser 2.8.2 2023-10-10 2.8.3 2023-12-04 1 [0, 0, 1] 0.2
simplecov-console 0.5.0 2019-05-24 0.9.1 2021-02-01 7 [0, 4, 0] 1.7
vcr 6.1.0 2022-03-13 6.2.0 2023-06-26 1 [0, 1, 0] 1.3
webmock 3.10.0 2020-11-12 3.19.1 2023-08-29 19 [0, 9, 0] 2.8
ruby 3.1.3 3.2.2 4 [0, 1, 0] 0.0
System is 14.0 libyears behind
Total releases behind: 92
Major, minor, patch versions behind: 0, 33, 3
Showing Progress
Now I’m going to go ahead and upgrade the dependency that is most out of date:
minitest
Sometimes doing this can be as easy as bumping the version in my gemspec:
spec.add_development_dependency "debug"
- spec.add_development_dependency "minitest", "~> 5.8.4"
+ spec.add_development_dependency "minitest", "~> 5.20.0"
spec.add_development_dependency "minitest-around", "~> 0.5.0"
After doing that, I can run my test suite and make sure that everything works. CI reports that everything works so I can ship this change to production. 🚀
Luckily I didn’t have to change anything in my test suite. Everything just works with Minitest v5.20.
Now that I have shipped that upgrade: How can I show the progress I’ve made to a non-technical stakeholder?
If we consider the two different tools, we can summarize it as making your dependencies X% more up to date.
Using Out-of-date Percentage
I can use next_rails’s bundle_report command. Here is the output after
the upgrade:
$ bundle_report outdated
simplecov-console 0.5.0: released May 24, 2019 (latest version, 0.9.1, released Feb 1, 2021)
codecov 0.1.21: released Aug 26, 2020 (latest version, 0.6.0, released Aug 18, 2021)
webmock 3.10.0: released Nov 12, 2020 (latest version, 3.19.1, released Aug 29, 2023)
vcr 6.1.0: released Mar 13, 2022 (latest version, 6.2.0, released Jun 26, 2023)
bundler 2.4.7: released Feb 15, 2023 (latest version, 2.4.22, released Nov 9, 2023)
regexp_parser 2.8.2: released Oct 10, 2023 (latest version, 2.8.3, released Dec 4, 2023)
rdoc 6.6.0: released Nov 6, 2023 (latest version, 6.6.1, released Dec 5, 2023)
irb 1.9.1: released Nov 21, 2023 (latest version, 1.10.1, released Dec 5, 2023)
json 2.7.0: released Dec 1, 2023 (latest version, 2.7.1, released Dec 5, 2023)
0 gems are sourced from git
9 of the 60 gems are out-of-date (15%)
Now we could say that the project’s dependencies are 2% more up to date. My pull request took the out-of-date percentage from 17% to 15%. A smaller number is definitely better!
Using Libyears
Here is the output after the upgrade:
$ libyear-bundler --all
codecov 0.1.21 2020-08-26 0.6.0 2021-08-18 25 [0, 5, 0] 1.0
irb 1.9.1 2023-11-21 1.10.1 2023-12-05 2 [0, 1, 0] 0.0
json 2.7.0 2023-12-01 2.7.1 2023-12-05 2 [0, 0, 1] 0.0
rdoc 6.6.0 2023-11-06 6.6.1 2023-12-05 1 [0, 0, 1] 0.1
regexp_parser 2.8.2 2023-10-10 2.8.3 2023-12-04 1 [0, 0, 1] 0.2
simplecov-console 0.5.0 2019-05-24 0.9.1 2021-02-01 7 [0, 4, 0] 1.7
vcr 6.1.0 2022-03-13 6.2.0 2023-06-26 1 [0, 1, 0] 1.3
webmock 3.10.0 2020-11-12 3.19.1 2023-08-29 19 [0, 9, 0] 2.8
ruby 3.1.3 3.2.2 4 [0, 1, 0] 0.0
System is 7.0 libyears behind
Total releases behind: 62
Major, minor, patch versions behind: 0, 21, 3
Now we could say that the project’s dependencies are 50% better. My pull request took the libyears metric from 14 to 7 years. That’s 7 libyears closer to up to date!
Why should I care?
While automated dependency management tools like Dependabot can help, they won’t do much for major or minor releases without a concerted effort by your engineering team to constantly upgrade dependencies.
Systems using outdated dependencies are four times as likely to have security issues compared to systems that are up-to-date. There is a correlation between dependency freshness and dependencies with a reported vulnerability:
(Source: Measuring Dependency Freshness in Software Systems )
Conclusion
Both metrics can be useful for assessing the state of your dependencies and communicating progress. Personally, I like the additional information provided by libyears more than the out-of-date percentage, but I also see the value in combining both metrics to truly understand the state of my dependencies.
Having a libyears value of 0 years and having a set of dependencies that is 100% up-to-date does not guarantee that your dependencies are free of vulnerabilities. It does guarantee that all known vulnerabilities and CVEs in your dependencies have been addressed, which takes you one step closer to running a secure set of dependencies in production.
If you are interested in other ways to secure your application, you should read this article: 4 Essential Security Tools To Level Up Your Rails Security
How does your current project do in terms of libyears and out-of-date percentage? Do you keep track of these metrics? I strongly recommend it!
Interested in paying off 1% of your tech debt every month? Let’s talk!