Upgrading GWW from Rails 2, RSpec 1 and prototype to Rails 3, RSpec 2 and jQuery

Background reading

The most helpful blog posts on this topic that I found included Simone Carletti’s excellent post on preparing for the upgrade, Jeremy McAnally’s post on upgrading, Ryan Bigg’s Rails 3 upgrade log and Kurt Snyder’s summary of changes between RSpec 1 and RSpec 2. Thanks, all.

Before the upgrade

I made sure GWW’s test suite was thorough. Testing routes is critical, since routes.rb changes completely. For an app that uses ERB, tests of page content (GWW uses RSpec-Rails controller specs with, in RSpec 1-speak, integrate_views) are also especially valuable, since the move to Erubis breaks lots of things.

If you don’t already write tests the way GWW’s are written, you probably wouldn’t want to change for the sake of the upgrade alone, but it’s worth mentioning: GWW tests controllers and views in controller specs with (in RSpec 1 terminology) integrate_views. Although these tests had to change (see below), they had to change much less than if they’d been separate tests of controllers and views. That was not at all the point of writing tests that way (it was that I think the interface between skinny controllers and views is an uninteresting implementation detail, not test-worthy behavior), but it was a nice bonus. Also, GWW uses shoulda matchers in its routing specs; as well as being more succinct and readable than the RSpec matchers they replace, they work on Rails 3 unchanged.

I upgraded some components of GWW to what Rails 3 uses, testing and deploying each independently, before upgrading Rails, to make the actual upgrade as small and short as possible:

I upgraded to Rails 2.3.11. This shook out some things that would otherwise have come up when upgrading to Rails 3.
I upgraded the mysql gem to mysql2. I’ve lost track of whether it was a recent Rails 2.3.something or Rails 3 that required this.
I migrated to Bundler. This is a good idea whether you’re planning to upgrade to Rails 3 or not.
Immediately before upgrading to Rails 3, I upgraded GWW production to Phusion Passenger 3.0.5.

Some things that I didn’t do before the actual upgrade, but should have:

Migrate to Erubis and the new default of escaping HTML in strings interpolated into templates.
Migrate from RSpec’s have_tag and have_text matchers to Webrat’s have_selector and contains matchers or some other RSpec 2-compatible equivalent.

I don’t know of a simpler upgrade path from prototype to jQuery than rewriting all of the Javascript in mid-upgrade, which is what I ended up doing.

The upgrade

Install the rails_upgrade plugin

script/plugin install git://github.com/rails/rails_upgrade.git, then rake rails:upgrade:check, which said to do some of the things below and some other things that weren’t relevant or could wait. As mentioned above I didn’t worry about the deprecations until I ran into them.

Create application.rb

rake rails:upgrade:configuration > someplace-out-of-the-way/application.rb

Create the new `routes.rb`

rake rails:upgrade:routes > someplace-out-of-the-way/routes.rb
The generated routes.rb had several classes of errors which I fixed manually:

It sometimes said :via => get when it meant :via => :get
It ignored :only options to resources routes. Fortunately that syntax is unchanged in Rails 3 and I could just copy them over.
It ignored :conditions options on with_options blocks. I added more :via => whatever as needed.

The generated routes.rb wasn’t well factored, either, but fixing that could wait.

Back up customized files

rake rails:upgrade:backup

Install Rails 3

I edited GWW’s Gemfile to use Rails 3.0.5, rspec 2.5.0, shoulda-matchers 1.0.0.beta2 (it had previously been using shoulda 2.11.3), webrat 0.7.3 and will_paginate 3.0.pre2 and ran bundle install.

Generate a Rails 3 application on top of the existing Rails 2 application

rails new .

I let it overwrite anything it wanted, backing it up manually first if it hadn’t already been backed up by rake rails:upgrade:backup.

Sort out the damage

I went through all of the files that were modified or not in version control, merging changes from the app’s Rails 2 version and then removing the backup.
I moved the application.rb and routes.rb created above to config.
The directory I was in didn’t have the name that I wanted to use for the module containing my Rack application object, so I searched for all of the places where rails_upgrade or rails new . had used that directory name for the module name and corrected them.
I removed Rails 2’s config/initializers/new_rails_defaults.rb, since the defaults that it sets are now actually defaults in Rails 3.
I removed config/preinitializer.rb, which integrated Bundler with Rails 2 and is obsoleted by Rails 3’s built-in Bundler integration.
The only script in the script directory in Rails 3 is rails; I removed all of the others, leftovers from Rails 2.
Finally, I removed the rails_upgrade plugin, since its job was done.

Start the Rails console

rails c reported some errors, mostly uninteresting. One issue that caused a few minutes of head-scratching at this stage: A bug makes Rails say “Missing helper file” when it should say “required file not found” when loading a helper module with an incorrect require. Having figured out the error message, I didn’t try to figure out why requires that worked under Rails 2 failed under Rails 3; I just fixed them.

After a few rounds of minor fixes, rails c started without errors. I added a temporary fake production database config to config/database.yml and went another few rounds with rails c production, after which it too started without errors.

Configure RSpec 2 and RSpec-Rails 2

I ran rails g rspec:install and moved customizations from the old spec/spec_helper.rb to the new one. Per the instructions, I

made sure that rspec-rails was in GWW’s Gemfile’s :development group (which it already was for a reason not related to Rails 3 or RSpec 2)
removed lib/tasks/rspec.rake, spec/spec.opts and spec/rcov.opts

copied the definition of the spec:rcov task from /opt/local/lib/ruby/gems/1.8/gems/rspec-rails-2.5.0/lib/rspec/rails/tasks/rspec.rake to a new task file, lib/tasks/rcov.rake, and modified it to unload the old spec:rcov task and to exclude some additional files from coverage:

require 'rspec/core'
require 'rspec/core/rake_task'

Rake.application.instance_variable_get('@tasks').delete 'spec:rcov'

namespace :spec do
  desc "Run all specs with rcov"
  RSpec::Core::RakeTask.new(:rcov => 'db:test:prepare') do |t|
    t.rcov = true
    t.pattern = "./spec/**/*_spec.rb"
    t.rcov_opts = '--exclude /gems/,/Library/,/usr/,lib/tasks,.bundle,config,/lib/rspec/,/lib/rspec-,spec/,buildAgent/plugins'
  end
end

Run the specs, one layer at a time, and fix issues

rake spec:lib reported errors related to a use of ActiveSupport::TestCase.use_transactional_fixtures. It wasn’t essential (GWW’s specs other than model specs turn off transactions just to reduce noise in the RSpec log) so I commented it out and rake spec:lib passed.

rake spec:routing failed, complaining of “undefined method `route'“. It seems that current versions of shoulda don’t integrate quite correctly with RSpec; the solution was to add config.include Shoulda::Matchers::ActionController to spec/spec_helper.rb. That fixed most of the routing tests. The remaining handful of failing routing tests were due to errors in the new routes.rb. (Was I glad I’d tested every route!) After fixing those rake spec::routing passed.

rake spec:models failed with the error “undefined method `add_limit!'” in methods using will_paginate’s paginate_by_sql. That was easily fixed by upgrading will_paginate to 3.0.pre2. (I lied above; I didn’t figure out that I needed that upgrade until this point.) That was all it took to get rake spec::models to pass.

rake spec:helpers took only a little more effort:

Helper specs that set request parameters through the magic params variable had to be changed to use the new magic controller.params variable.
Different helpers with methods with the same name suddenly found themselves in conflict. GWW had only a few such methods, so the easy solution was to rename some of them to eliminate ambiguities.
Custom matchers and other modules that were included in spec/spec_helper.rb with the :type option failed to load when :type was an array of spec types. Breaking each such include into one line per spec type fixed that. For example,

config.include Photos, :type => [ :helper, :controller ]

became

config.include Photos, :type => :helper config.include Photos, :type => :controller

Making rake spec:controllers pass took by far the most effort of any step so far. First, the easy parts:

RSpec 1’s integrate_views is render_views in RSpec 2, a quick search-and-replace.
Each partial called with :object (which defines a local with the name of the partial) no longer saw that local. I’m not sure whether this was an intentional change in Rails 3 or not, but it was easy to name the local explicitly.

And then the big part, which was to config.include Webrat::HaveTagMatcher and rewrite all of the specs that used RSpec’s have_tag matcher to use Webrat’s have_selector and contains matchers. I referred to this nice cheat sheet by Wincent Colaiuta.

The updated specs found one more problem: Since Rails 3/Erubis now escapes all expressions in templates by default, expressions which returned HTML needed to be wrapped in raw().

With the controller specs rewritten, rake spec:controllers and rake:spec passed. Weirdly, rake spec:rcov failed; although it had not been necessary for rake spec:models to pass, rake spec:rcov needed config.include Shoulda::Matchers::ActiveRecord in spec/spec_helper.rb.

Manual testing

Now that all of the specs passed it was finally time to rails s GWW and look at it in a browser. It came out from under the bandages in great shape, with only a few blemishes that needed correcting:

A few template expressions that didn’t want to be escaped and hadn’t been caught in tests needed to be wrapped in raw().
The ID of the form created by the button_to helper had changed from button-to with a hyphen to button_to with an underscore, so some CSS and Javascript had to be updated.
A popup that issued an AJAX request no longer worked. Rails complained that it didn’t know how to respond to a request for application/json. Amusingly, the action emitted HTML, not JSON; a copy-and-paste error had made the requesting Javascript Accept: application/json. Rails 2 never noticed, but Rails 3 did, and refused. The fix was just to remove the bogus parameter.
Some of GWW’s HTML needs to be exact about its newlines, since it’s pasted into Flickr discussions where newlines are significant. Erubis ignores whitespace around <% %>. Where GWW’s ERB templates intentionally used <% %>, not <% -%>, to insert a newline, I needed to add a blank line.
The other side of the coin: Putting <% -%> at the end of a line in an ERB template suppresses the newline, but that doesn’t work in Erubis. Fortunately <%= -%> does work, so I just needed to replace each <% -%> with <%= -%>.

Just one more thing …

The very last thing that had broken was autocompletion in two text fields. Autocompletion had been done by a venerable plugin which was last updated in 2007 and didn’t work with Rails 3. There seemed to be no replacement which used prototype; the leading autocomplete plugin uses jQuery. Well, the Rails team is backing jQuery these days (and who isn’t?) and I’d have migrated soon enough anyway. I installed jquery-rails and rails3-jquery-autocomplete, installed jquery-rails per the instructions (with --ui), picked a jQuery UI CSS file from jqueryui.com, used the new autocompleter on the autocompleted fields, and rewrote all of GWW’s custom Javascript to use jQuery instead of prototype.

After 2 1/2 days of work, production GWW is on Rails 3. There’s still some followup to do: refactor routes.rb, make use of Arel, take all of the now-redundant h‘s out of the templates, etc. And then back to features!

Dave Schweisguth in a Bottle