Upgrading Airbnb from Rails 2.3 to Rails 3.0
By Ross Allen
One of the major nerd goals for Airbnb in 2011 was upgrading to Rails 3. Our production instances made the final switch in the week leading up to Thanksgiving, but it didn’t happen all at once.
Breaking the Upgrade into 3 Steps
We added the required pieces throughout the past year, and, looking back, breaking the upgrade into three major steps was easier to manage than trying to cram it into a single deploy. The first two steps are independent from actually upgrading to the Rails 3 gem, and any Rails 2.3 app out there should do them even if there’s no intention of upgrading to 3.0.
Step 1: Add Bundler
This was our first step into Rails 3 land, and we did it at the end of 2010. Our app was running Rails 2.3.8 at the time, and the production instances were a mess. Each instance had its own set of gems and its own versions of those gems. Naturally, none of the instances matched.
One of our engineers picked the latest version of each gem initialized by config.gem in production to create our first Gemfile. He installed Bundler using their Rails 2.3 guide and added any missing gems when trying to start the app. He used the recommended ~> notation for gem versions at first, but that upgraded our Paperclip gem to 2.3.3. That version had performance problems in our environment, so the engineer specified exact versions for our gems instead, the versions we had fully tested.
He then ran each of our Rake tasks and added any gems those tasks needed to run. That was the last step before our Gemfile was ready to deploy in production.
Adding Bundler at the end of 2010 was a one-engineer task that took about a week working on-and-off. If Bundler had been added the same time as the rest of the Rails 3 upgrade, almost a full year later, it would have been a much bigger ordeal.
After installing Bundler, Airbnb ran without any thoughts of Rails 3 for a while. Our production environment was far cleaner, each production instance was running the same set of gems, and our deploy process was simpler.
Step 2: Install the rails_xss plugin
Templates in Rails 3 auto-escape all strings, which is a major change from 2.3 and breaks a lot of assumptions. Auto-escaping HTML is a good thing, but many of our templates assumed Ruby strings would be interpreted as HTML.
Skip ahead from our Bundler install about 6 months to Spring 2011. To bring the Rails 3 behavior into our app, Tobi Knaup, one of our engineers, chose to install the rails_xss plugin. The plugin escapes strings by default like Rails 3. He knew this was likely to break a ton, if not all, of our templates, so he setup a Rails XSS Hacksession to get every engineer’s help in the installation.
Tobi listed every controller in our app at the time and divided them among the engineers. The engineers went through the templates used by each controller and made sure string output in templates was marked as html_safe or rendered with raw where appropriate.
Most of the XSS work was done in a single day with all of the engineers’ help, but there was cleanup on pages that weren’t used very often that continued for several weeks after installing the plugin.
With rails_xss installed, Airbnb was more secure and closer to Rails 3.
Step 3 (the big one): Add the Rails 3 Gem
This is the well-known upgrade step that includes updating the ‘rails’ gem to 3.x. Thankfully our Gemfile was already mature, and our templates expected most strings to be HTML-escaped when we got to this step around Thanksgiving 2011.
We knew the real upgrade required major code changes, so four engineers started the upgrade on a Saturday afternoon when there were minimal changes being committed. We asked other engineers to hold off on changes over the weekend to prevent any merge nightmares.
We kept the Rails upgrade branch separate with its own Gemset in case we needed a local working version of master before the upgrade was done. The most basic version of our workflow looked like this:
- Create a new rails3 branch
- Clone the rails3 branch into a separate directory from the usual working directory (instead of simply switching branches)
- Create a new gemset for the rails3 branch’s directory
- Install the rails_upgrade plugin
- Follow the steps from the rails_upgrade README, skipping the step that generates a Gemfile since we were already using Bundler
- Try running rails console and rails server
- Fix any raised errors
- Return to step 6 if there were errors in step 7
- Done! Sort of…
Things that caused headaches during the upgrade:
- The auto-generated routes file from the rails_upgrade plugin was wrong. The routes for our internal API (the API that powers our iPhone app) were busted. The problems weren’t isolated to those routes, but those were the most notable. While we did use the auto-generated file as a starting point, we spent a lot of time tracing problems back to the broken routes file.
- Timezones that worked in Rails 2.3 raised errors in 3.0. We didn’t change the timezones we had stored, but various ones started raising errors. Benjamin Oakes’s post about Rails Timezones explains the problem and how to start fixing it.
- The lib directory is no longer auto-loaded. This was not called out prominently anywhere, but a quick search turns up several ways to address it. We added our lib directory to the autoload paths in application.rb
- config.autoload_paths += %W( #{Rails.root}/lib )
The app was running on Rails 3 by the end of the weekend, and specs were passing, but there were plenty of pages that hadn’t been tested.
Monday morning all of the engineers took ownership of features they had recently worked on and thoroughly tested them similar to how we approached the Rails XSS install. By the middle of the week we merged the rails3 branch into master and deployed it to our production instances.
What made the upgrade successful
If everything goes smoothly, the Rails 3 upgrade should be transparent to users. That can make the upgrade a hard sell to anyone outside engineering considering the effort that goes into it. Installing Bundler was a small project done by a single engineer, but installing the XSS plugin and doing the actual Rails 3 upgrade took considerable hours that had to be planned.
We coordinated with our product team for each of the big steps. They were planned between major product updates to make sure the codebase wasn’t changing while we were upgrading.
Check out all of our open source projects over at airbnb.io and follow us on Twitter: @AirbnbEng + @AirbnbData
Originally published at nerds.airbnb.com on January 04, 2012.
