Our First Node.js App: Backbone on the Client and Server
By Spike Brehm
Here at Airbnb, we’ve been looking curiously at Node.js for a long time now. We’ve used it for odds and ends, such as the build process for some of our libraries, but we hadn’t built anything production-scale. Until now.
The Problem
There’s a disconnect in the way we build rich web apps these days. In order to provide a snappy, fluid UI, more and more of the application logic is moving to the client. In some cases, this promotes a nice, clean separation of concerns, with the server merely being a set of JSON endpoints and a place to host static files. This is the approach we’ve taken recently, using Backbone.js + Rails.
But all too often, it’s not so clean; application logic is somewhat arbitrarily split between client and server, or in some cases needs to be duplicated on both sides. Think date and currency formatting. You tend to have a Ruby (Python, Java, PHP) library that you’ve been using for awhile, but all of a sudden you have to replicate this logic in a JavaScript library. The same is true for the view layer; some of your markup lives in Mustache or Handlebars templates for the client, while other needs to live in ERB or Haml so they can be rendered on the server for SEO or other reasons.
If you’ve seen my tech talk or last blog post, then all this should sound familiar. Our thesis, four months ago when I gave this talk, was that, in theory, if we have a JavaScript runtime on the server, we should be able to pull most of this application logic back down to the server in a way that can be shared with the client. Then, as a developer you just focus on writing application code. Your great new product can run on both sides of the wire, serving up real HTML on first pageload, but then kicking off a client-side JavaScript app. In other words, the Holy Grail.
This Holy Grail approach is something we had dreamt about for a long time, but not having any experience with Node.js, we didn’t quite know where to start. Luckily, we hired our first engineer who has experience running Node.js at large scale in production; meet Barbara Raitz, who comes to us from LinkedIn, where she built out the API that powers their mobile apps using Node.js. Now that we had some in-house Node expertise, Barbara and I set out to create our solution for the Holy Grail.
Our Solution
I’m proud to announce that we’ve launched our first Holy Grail app into production! We’ve successfully pulled Backbone onto the server using Node.js, tied together with a library we’re calling Rendr. We haven’t open-sourced Rendr quite yet, but you can expect it in the coming months, once we’ve had a chance to build a few more apps with it and decouple it from our use case a bit.
So, the app: we re-launched our Mobile Web site using this new stack, replacing the Backbone.js + Rails stack that it used previously. You may have been using it for over a month without even knowing. It looks exactly the same as the app it replaced, however initial pageload feels drastically quicker because we serve up real HTML instead of waiting for the client to download JavaScript before rendering. Plus, it is fully crawlable by search engines.
The performance gains are an awesome side effect of this design. In testing, we’re using a metric we call “time to content”, inspired by Twitter’s time to first tweet. It measures the time it takes for the user to see real content on the screen. Let’s take our search results page, for example. Under the old design, before any search results could be rendered in the client, first all of the external JavaScript files had to download, evaluate, and execute. Then, the Backbone router would look at the URL to determine which page to render, and thus which data to fetch. Then, our app would make a request to the API for search results. Finally, once the API request returned with our data, we could render the page of search results. Keep in mind all of this has most likely happened over a mobile connection, which tends to have very high latency. All of these steps add up to a “time to content” that can be more 10 seconds in extreme cases.
Compare this with serving the full search results HTML from the server. Over a mobile connection, it may take 2 seconds to download the initial page of HTML, but it can be immediately rendered. Even as the rest of the JavaScript application is being downloaded, the user can interact with the page. It feels 5x faster.
Gimme the Deets!
We built upon tools we already know and love: Backbone.js and Handlebars.js. We ended up with a hybrid of Rails, Backbone, and Node conventions. For example, the app’s directory structure will look familiar to Rails users (minus collections and models):
| app
|-- collections
|-- controllers
|-- helpers
|-- models
|-- templates
|-- viewsIn a typical Backbone app, “controller” logic — fetching the appropriate data and instantiating view(s) for a particular page — lives in methods on your instance of Backbone.Router. As your app grows, however, the router quickly becomes bloated with all of these route handlers. This is why we’ve created real controller objects, which group related actions into more manageable chunks. This abstraction also allows us to more easily add before and after filters if we want to. Here’s what a users controller might look like:
# app/controllers/users_controller.coffeemodule.exports =
index: (params, callback) ->
spec =
collection: {collection: 'Users', params: params}
@app.fetch spec, (err, results) ->
callback(err, results)show: (params, callback) ->
spec =
model: {model: 'User', params: params}
@app.fetch spec, (err, results) ->
callback(err, results)
First off, you'll notice we're using CoffeeScript. It's a bit controversial, I know, but we're fans. Secondly, notice the module.exports = at the top. That's a tell-tale sign of a CommonJS module. CommonJS is what Node.js uses to require modules, and we're able to reuse the same syntax in the client using Stitch, which was written by the Sam Stephenson, the same fellow who also wrote Sprockets for Rails.
Now, keep in mind that this controller code gets executed on both the client and the server. For example, if a user lands on "/users/1234", and a route exists that routes that to users#show (more on that below), then the show action will be invoked.
The @app.fetch you see is our way of encapsulating resource fetching. In this case, a User model with id 1234 will be fetched from the API, instantiated, and passed to the view. Why not just instantiate the User model yourself in the controller? You could do that, but the fetcher provides a layer of indirection which allows us to do a bunch of fancy caching in both the client and the server-side. I'll leave out the details for now, but that could be the subject of a future blog post!
Routing Requests
We need to be able to match a certain URL to a controller/action pair both on the client-side and the server-side. Inspired by Rails, we have a routes file that specifies these routes and any additional route parameters.
# app/routes.coffeemodule.exports = (match) ->
match 'users/:id', 'users#show'
match 'users', 'users#index'match 'listings/:id', 'listings#show'
match 'search', 'listings#search', maxAge: 900
Notice the optional parameter maxAge on the listings#search route; this is used to set caching headers. You can add any arbitrary parameters here and access them in the router. We also plan to make this more advanced as needed, such as adding parameter requirements.
Our routes file format is heavily inspired by a really interesting project called ChaplinJS. Chaplin is an application framework built on top of Backbone. Chaplin is under active development and is quite well maintained; definitely worth a look if you want some more structure in your Backbone app.
These routes are parsed by a router, which delegates incoming requests to particular controllers. We have a ClientRouter, which delegates to Backbone.History and translates pushState events to controller actions, and a ServerRouter, which does the same for Express requests; both extend BaseRouter to share common logic.
We serve the app using Express, the de-facto web server for Node.js.
Views
In Rendr, your views extend our BaseView, which in turn extends Backbone.View, adding a number of methods that allow it to easily render on both the server and the client. Here's an example ListingsIndexView for you:
# app/views/listing/index.coffeeBaseView = require('rendr/base/view')
_ = require('underscore')module.exports = class ListingsIndexView extends BaseView
# Straight outta Backbone.View
tagName: 'section'
className: 'listing clearfix'events:
'click .book_it': 'clickBookIt'clickBookIt: ->
...# Custom Rendr methods
postRender: ->
@createSlideshow()getTemplateData: ->
_.extend super,
bullets: _.first(@model.get('picture_urls'), 50)createSlideshow: ->
...
First you'll notice that we require BaseView with require('rendr/base/view'). This CommonJS module path is standard in Node.js for requiring files within NPM packages, and using a trick in our Stitch packaging, we can use the same path in the client.
In addition to the typical methods and properties from Backbone.View, we've added some custom ones to hook into the view's lifecycle. Notice the #postRender() method; this gets called only in the client-side, right after rendering. This is where you would put any code that needs to touch the DOM, such as initializing jQuery plugins like slideshows or sliders.
Each view has a Handlebars template associated with it, and a View#getHtml() method. On the server, we simply call #getHtml() and return the resulting string. On the client, View#render() calls #getHtml() and updates the innerHTML of its element. We also have a #getTemplateData() method, which by default returns @model.toJSON(). You can override it to act as a view-model, adding any view-specific properties to pass to the template for rendering.
We decided on an entirely string-based templating approach, preferring not to depend upon a DOM on the server. One result of this is that it becomes necessary to push all HTML manipulation either into the Handlebars template or into a custom #getHtml() method. In other words, you cannot rely on any DOM manipulation code to construct your markup; if you have a habit of appending child views, loading spinners, or any other markup in #render(), get used to pushing that down to the template. Now, here's the tricky part: when we generate a page of HTML from a hierarchy of views on the server, we also have to ensure that once it reaches the client, all of the views' event handlers are properly bound to the DOM, and we have living, breathing view instances that can respond to user interactions. Our approach was to decorate the generated HTML with data-attributes, which specify which view class it represents. Here's an example from one of our listing pages:
<section class="listing" data-view="listings/index" data-model_id="687210" data-model_name="listing">
<div class="slideshow">
...
</div><section class="details tab_panel" data-view="listings/details" data-model_id="687210" data-model_name="listing">
<ul>
...
</ul>
</section><section class="user" data-view="listings/user" data-model_id="2088962" data-model_name="user" data-lazy="true">
...
</section></section>
On pageload, we find all DOM elements with the data-view="some_view_class" attribute, instantiate view objects for each, and reconstruct the view hierarchy. That way, events are properly bound, and we preserve any parent-child relationships between our view objects so they can listen to and emit events on each other.
Sounds easy enough, right? Well, it ended up being more difficult to solve than we expected. One of the problems we ran into is that all data needed to instantiate a view needs to be extractable from the DOM. You'll notice in the above code snippet that we also have the data-attributes data-model_name and data-model_id. This allows us to pass in the correct model or collection into any view, fetching it from the client-side model cache (remember @app.fetch from our controller? it handles this for us). This is the pattern we came up with in order to ship; a better approach would be to assign a unique id to each view instance on the server, and have some sort of mapping from view id to view data which we could read from in the client, rather than looking up from the DOM.
There's another way to ensure views are properly bound to the DOM, which we decided against. It involves discarding the server-generated HTML upon pageload, regenerating the view hierarchy in the client, and swapping the view DOM tree into the page. The downside is the potential for weird UX interactions if, for example, a user starts to interact with elements on the page that then get destroyed and replaced.
NestedView
We really like enforcing encapsulation within our views. We also wanted an easy way to declaratively nest subviews within any view. We came up with a nifty way of achieving both using Handlebars helpers. Check out this slightly contrived example.
Given these views:
BaseView = require('rendr/base/view')class IndexView extends BaseViewclass UserDetailView extends BaseViewand these templates, app/templates/index.hbs:<h1>Hello, {{user.name}}.</h1>
{{view "users/detail" context=user}}`app/templates/user_detail_view.hbs:<li>Name: {{name}}</li>
<li>Email: {{email}}</li>and this data:{
"user": {
"name": "Spike",
"email": "spike@example.info"
}
}calling View#getHtml() on the parent returns this HTML:<div data-view="index_view" data-cid="view1">
<h1>Hello, Spike.</h1>
<ul data-view="user_detail_view" data-cid="view2">
<li>Name: Spike</li>
<li>Email: spike@example.info</li>
</ul>
</div>
So, you can arbitrarily nest views using a simple Handlebars helper, and simply call #getHtml() on the top-most view to get the entire hierarchy's HTML. Nifty, eh?
We found this useful enough as a standalone library that we've pulled it out into NestedView. Check it out on Github.
Models & Collections
Now onto models and collections.
Rendr has a BaseModel and BaseCollection, which extend Backbone.Model and Backbone.Collection. Here's the most basic User model:
# app/models/userBaseModel = require('rendr/base/model')module.exports = class User extends BaseModel
url: '/users/:id'
Pretty thin, right? Of course, you can add whatever custom methods you want to your models.
The important part is the url property. Rendr expects to get all of its data from a JSON API over HTTP. This could be on the same server, but in our case, it's a preexisting API that powers a number of other apps. The url specified above points to the path on this API server.
Calling Backbone's CRUD methods on the model or collection (#save(), #fetch(), #destroy(), and now #update() in the latest Backbone) will dispatch a request to this API. We override Backbone.sync() such that when these methods are called from the server-side, Rendr sends an HTTP request directly to the API server. When called from the client, Rendr will prepend /api to the model/collection URL, proxying the request to the API through the Rendr server. This allows us to do any additional formatting of the request or response, and to centralize the API request and response handling logic, at the expense of some network time.
Sprechen Sie Deutsch?
Internationalization (I18n) is incredibly important to us here at Airbnb. We support 30+ languages, and have localized web sites all around the world. We've been performing I18n in JavaScript for some time, but the need arose to make a more-robust library that can run in CommonJS environments as well. That's led to Polyglot.js, our open source JavaScript I18n library. The extra special sauce is our pluralization logic. Here's an excerpt from the docs:
In English (and German, Spanish, Italian, and a few others) there are only two plural forms: singular and not-singular.
However, some languages get a bit more complicated. In Czech, there are three separate forms: 1, 2 through 4, and 5 and up. Russian is even crazier.
Polyglot.js abstracts all that away from you, and just requires you to provide translations for whichever locales you're interested in.
What's Next?
There's still a lot of work to be done. Before we release Rendr, we need to build a few more applications with it and modularize the code a bit more. Luckily we have a growing need for rich JavaScript apps that are fast and SEO-friendly here at Airbnb. Keep an eye out here and follow @AirbnbNerds and @spikebrehm for more updates.
I'll be speaking more about Rendr and Airbnb's experiences building rich JavaScript apps at HTML5DevConf on April 1-2 in San Francisco. It's shaping up to be a great conference. Sign up if you haven't already!
Check out all of our open source projects over at airbnb.io and follow us on Twitter: @AirbnbEng + @AirbnbData
Originally published at nerds.airbnb.com on January 30, 2013.
