Airbnb Engineering http://nerds.airbnb.com Nerds Mon, 24 Nov 2014 06:03:25 +0000 en-US hourly 1 http://wordpress.org/?v=3.8.5 Meet the Nerds: Carol Leung http://nerds.airbnb.com/meet-the-nerds-carol-leung/ http://nerds.airbnb.com/meet-the-nerds-carol-leung/#comments Mon, 24 Nov 2014 06:03:25 +0000 http://nerds.airbnb.com/?p=176594578 Welcome to our second installment in our series that peeks behind the curtain of Airbnb’s engineering team. Today, say hello to Carol Leung, an Engineer on our mobile team working on our Andoid app. Carol shares with us her experiences from founding her own photography start-up in Hong Kong to making the Airbnb mobile app […]

The post Meet the Nerds: Carol Leung appeared first on Airbnb Engineering.

]]>
Welcome to our second installment in our series that peeks behind the curtain of Airbnb’s engineering team. Today, say hello to Carol Leung, an Engineer on our mobile team working on our Andoid app. Carol shares with us her experiences from founding her own photography start-up in Hong Kong to making the Airbnb mobile app seamless to use.

How did you get started in Computer Science?

My high school started offering a computer science class in my sophomore year. I was curious and took the intro and follow-up CS classes. I was fascinated by how applicable CS is to so many things in our daily life. I was inspired to build a program that simplified the process of matching students with their interests for our annual “Diversity Day.” The experience of being able to build a solution to a real problem set me on the path to become a software engineer.

What was your path to Airbnb?

I love travel and immersing myself in different cultures. I have lived and worked as an engineer in different parts of the world: building algorithmic trading systems at Goldman Sachs Tokyo, co-founding my own digital photography software startup in Hong Kong, leading a mobile development team at XtremeLabs (now Pivotal) in Toronto, and building an iOS and Android app from the ground up for a small social network startup after I moved back to San Francisco. My love of travel (and software development) lead me to Airbnb. I love that I can use my passion for traveling to help shape a product that enriches other people’s journeys.

What’s the most interesting technical challenge you’ve worked on since joining?

Guests have traditionally felt more comfortable browsing and searching for listings on the web than on mobile. It’s my mission to change that perception by creating an outstanding guest experience on mobile. The challenge is creating a seamless, cohesive user experience across a variety of complex features in the app: Wish Lists, optimized search and filtering, embedded maps in the listing feed, listing navigation flows in map search. My teammates and I strive to hide that complexity from users and make the experience intuitive and “just work.”

What do you want to work on next?

I think we can go beyond helping guests find their ideal place. We can empower them to plan, build and experience the trips of their dreams. I want to build a mobile experience that would allow guests to enjoy a delightful journey from when they first start thinking about their trip. Searching, planning, and booking should be frictionless, delightful, and personalizable. We want users to be able to easily collect and compare listings they browse across different devices, make collaboration with friends and families on group trips fun, and surface recommendations based on users’ previous searches and location to inspire more meaningful trips.

141118_CarolLeung_NerdBlog0059

What is your favorite core value, and how do you live it?

“Every frame matters!” As a mobile engineer, I look at every UI element on every screen with a critical lens. I constantly ask myself: is this iconography clear? Does this animation feel natural? Are the widgets user-friendly? Does this navigation and transition make sense? Does this screen provide enough context while being pleasant to use? These questions often lead to great discussions with my teammates: engineers, designers, and product managers. “Every frame matters,” is also a starting point when we create technical designs and review code. This detailed-oriented attitude allows us to constantly improve code quality and structure of the app. It also yields world class products.

What’s your favorite Airbnb experience?

While I’ve enjoyed Airbnb in large cities like New York, Barcelona and Tokyo, my favorite was a lovely, cozy listing in Sonoma with my boyfriend and his family. The host thoughtfully wrote down great recommendations of local restaurants and vineyards on their cute, colorful kitchen chalkboard, and left us a fruit basket and a complimentary bottle of wine as welcome gift. I still remember the delicious smell from their spacious, beautiful garden growing fresh tomatoes and herbs. We had a blast relaxing and playing croquet and horseshoes in their backyard at night, a perfect way to end our wine country trip.

The post Meet the Nerds: Carol Leung appeared first on Airbnb Engineering.

]]>
http://nerds.airbnb.com/meet-the-nerds-carol-leung/feed/ 0
Maintaining Quality at Scale http://nerds.airbnb.com/maintaining-quality-scale/ http://nerds.airbnb.com/maintaining-quality-scale/#comments Tue, 18 Nov 2014 17:04:43 +0000 http://nerds.airbnb.com/?p=176594528 When I joined Airbnb in the fall of 2012 things were a little chaotic to say the least. At some point in the preceding year the growth of the company had kicked into high gear, or hyper-growth, as some call it. Along with the increase in site traffic and transaction volume, the engineering team also […]

The post Maintaining Quality at Scale appeared first on Airbnb Engineering.

]]>
When I joined Airbnb in the fall of 2012 things were a little chaotic to say the least. At some point in the preceding year the growth of the company had kicked into high gear, or hyper-growth, as some call it. Along with the increase in site traffic and transaction volume, the engineering team also expanded very rapidly. When I arrived we were a team of about 40, compared to 16 one year before (and over 130 today). Looking back now, I think we were in the midst of a make-or-break period in Airbnb’s history.

Growing a website and engineering team as quickly as Airbnb presents both technical and cultural challenges, and in many cases those challenges are intertwined. In the old days, we were struggling with many issues, but broadly those issues were related to scaling a monolithic Ruby on Rails application (which wasn’t built with scaling in mind) and scaling the engineering team in a way that facilitated the former.

Today, I think it’s fair to say that we’ve overcome the existential challenges we faced back then. This is not to say we don’t have numerous new challenges and hard work ahead of us, but we survived the kick into hyper-growth without tearing apart. There are a lot of different things we can look back on, but in this post I’d like to focus on how we have transitioned to writing code in a more maintainable (and thus scalable way).

I work on the Payments Team at Airbnb. For us, code quality, robustness, and maintainability are of the utmost importance. With the amount of transactions flowing through our system these days, we literally can’t afford even small mistakes. Over the past year, we’ve put a lot of work into figuring out how our team can continue to iterate quickly while still producing quality code and avoiding technical debt. The three things that I think have been most helpful at this level are forming and adhering to specific code style and conventions, practicing a comprehensive and collaborative peer review process, and testing.

Code Conformity

Code conformity relates both to syntactic/textual style and to applying conventions and best practices when writing code. It has been said many times before, but if you have a codebase where you can open up a file and tell which person on your team wrote it, that’s a very bad sign. Formulating a standard way of writing code and getting everyone on board has been highly beneficial for us.

Computer code is generally consumed by two classes of entities, machines and humans. Machines generally don’t care about what your code looks like (assuming it compiles), but humans on your team probably do. Having a wide variety of styles and approaches to similar problems in your codebase will create unnecessary cognitive load for everyone, and eat up a lot of time as people attempt to grok new terrain. On the other hand, if everyone on the team is writing code in a more-or-less identical manner, everyone on the team should be able to grok, debug, and/or maintain one another’s code with close to the same level of effort (once they get through the initial ramp-up). Put another way, it’s important to favor the human reader, not the machine, when crafting your code, and it’s even more important to be consistent. This is not to say you should chose an O(N) solution over an O(1) solution for readability or simplicity’s sake, but there are many steps you can take to facilitate your fellow humans through code conformity.

We’ve achieved better code conformity in two ways. The first is the use of style guides (of which we’ve created several of note). Our most detailed guides cover JavaScript and Ruby, however we also have internal guides for RSpec, API design, service design, etc. Although these guides grew from our specific experience, some of them are now public and utilized by other individuals and organizations (who then often contribute back). Depending on the type of application you’re working on, they may or may not be useful for you, but if you work with JavaScript or Ruby, I’d encourage you to check them out, if for no other reason than inspiration.

Some style rules we’ve settled on are completely arbitrary. Take for example indentation characters (tabs vs. spaces), or which line the opening brace goes on. It’s hard to argue for one over the other in cases like these. The important thing is settling on something and then having everyone stick to it. On the other hand, some seemingly simple rules can have vast ramifications.

Consider line length. We like to keep lines short and sweet. Shorter lines are not only more friendly to other engineers’ eyes (and editors), but they also tend to result in cleaner, simpler statements (especially when paired with descriptive variable names). Methods that consist of a series of short, simple statements are easier for other team members to comprehend and modify. Diffs (in version control tools such as Git) also end up showing more clearly what behavior was changed between commits. And, since we’re also writing testable code (see “Testing” below) which promotes small, concise methods, you start to develop a very clean, modular, and easy to understand codebase.

Consider this example:

usernames = Users.where(:status => 1, :deleted => false).map{ |u| u.first_name.downcase }.reject{ |n| n == ‘john’ }.map{ |n| n.titleize }

Imagine we wanted to tweak the behavior of this line but switching from downcase to upcase. The diff would end up looking something like this:

- usernames = Users.where(:status => 1, :deleted => false).map{ |u| u.first_name.downcase }.reject{ |n| n == ‘john’ }.map{ |n| n.titleize }
+ usernames = Users.where(:status => 1, :deleted => false).map{ |u| u.first_name.upcase }.reject{ |n| n == ‘JOHN’ }.map{ |n| n.titleize }

It’s hard to tell which part changed without parsing the line in your head. Had the code been written more like this:

users = Users.where(:status => 1, :deleted => false)
usernames = users.
map{ |user| user.first_name.downcase }.
reject{ |name| name == ‘john’ }.
map{ |name| name.titleize }

The diff would also be much cleaner, showing exactly what behavior changed:

users = Users.where(:status => 1, :deleted => false)
usernames = users.
- map{ |user| user.first_name.downcase }.
- reject{ |name| name == ‘john’ }.
+ map{ |user| user.first_name.upcase }.
+ reject{ |name| name == ‘JOHN’ }.
map{ |name| name.titleize }

The funny thing is that those of us who started promoting this faced a good deal of pushback in the beginning. Like testing, however, we were persistent and we’ve gotten to a place where this has become second nature. To see some extreme examples of this mentality, I suggest checking out the Joint Strike Fighter C++ coding guide or the JPL C Coding guide. Obviously these standards are overkill for most consumer web applications, but always keep in mind what type of application and goals you have when settling on rules. It’s about picking the right balance.

As mentioned before, we’ve also begun to create style guides for higher-level concepts such as API and service design. Although prescribing how to build things like this can be a bit more of a gray area, it’s been incredibly useful to consolidate the collective knowledge of the team into single resources that are easy to digest. When coupled with a peer review process and a collective insistence on conforming, we’ve eliminated a lot of pain points and needlessly repeating conversations.

Peer Review

Another way we have been able to achieve more consistent code is by having a comprehensive and collaborative code review process. Code reviews come with many benefits. The obvious case that comes to mind is a colleague catching a horrible bug before it goes out and brings the site down. But there are many more subtle benefits. Over the past year or so, we’ve instituted code reviews for every change and we like to see at least one other person to sign off on each diff before it goes out.

Having just one other person sign off, however, is the bare minimum. We encourage everyone who has context to get involved, and most non-trivial pull requests have at least one or two substantive conversations pop up. Sometimes these can get pretty deep and some of us end up learning something we didn’t know about (credit card processing, database internals, and cryptography come to mind). If there are big insights we are also sure to record those in the relevant guide/documentation. Most importantly, there is no authority in code reviews based on seniority or title. Everyone’s input is valid and yet best practices always tend to prevail and new team members tend to learn quickly.

It’s important to remember that just because a style guide exists doesn’t mean that it will always be followed (or even read) or that it addresses every case. Peer reviews are what really help people get on the same page. They are also an effective way to learn and teach the art of programming itself, which can’t be as easily captured in pre-written material. In addition, having an environment where people learn a lot on the job helps keep them engaged and boosts the quality of their work.

As an illustration of something that might not belong in a style guide because it falls more into the wisdom category, take this example.

us_users = User.active.select{ |u| ‘US’ == u.country }
puts “There are #{us_users.length} US users today”

vs.

us_users = User.active.where(:country => ‘US’)
puts “There are #{us_users.count} US users today”

We used to run into a lot of examples like the first one in our codebase. While they both output the same thing, the first example is far more expensive, and at scale could actually cause a catastrophe. Adding select to the User.active proxy object means that Rails will go to MySQL, fetch every active user, instantiate them, put them into an array, then iterate through that array and select the users with country equals ‘US’. All this just to get the count.

In the second example, we start with the same proxy object, Users.active, but then we use where to filter on that object. The first line does not trigger any DB queries! On the next line, when we ask for the count instead, Rails knows to just do a SELECT COUNT(*) query, and not bother fetching any rows or instantiating any models.

Knowing the language and framework well is very important, along with spreading that knowledge to everyone involved. We have been diligent at Airbnb about fixing these types of common mistakes and promoting the best practice of pushing work to the DB when it makes sense. Take another example. How much did we pay out today?

Bad:

amount = Payout.
where(:successful => true).
where(‘DATE(timestamp) = ?’, Date.today).
inject(0) { |sum, p| sum += p.amount }

Good:

amount = Payout.
where(:successful => true).
where(‘DATE(timestamp) = ?’, Date.today).
sum(:amount)

The first example could be worse, we do at least narrow the scope to a single day, but it still causes a large number of payout objects to get fetched and instantiated, just to sum a numeric field. In the second example, we have MySQL do the summing during the lookup and just return the number we care about. This comes at essentially no extra cost to MySQL yet we avoid both unnecessary network traffic and a potentially vast amount of processing in Ruby.

So one benefit to having a healthy review process is to formulate and teach conventions and best practices that will benefit everyone. These seemingly trivial examples make our site more responsive (and can even prevent it from going down). Things like this also might have the side effect of striking up conversations about database internals or financial reporting. The thing about this kind of knowledge is that it’s shared and it would be hard to consolidate it into a single resource (and expect everyone to read that resource). It gets introduced by new team members, or veterans who have been bitten, and then passed down through the generations. It can only live on, however, if there is an active, healthy dialog surrounding the production of code. Personally, I feel as though the rate at which I’m learning in this environment is much greater than in previous positions I’ve held where code reviews and quality weren’t taken as seriously.

Testing

I’m not going to go too deep into testing in this post because my colleague Lou recently published an excellent post of his own on this blog. What I will say though, and I don’t think I’m saying anything new, is that I believe testing is extremely important to writing quality, maintainable code. However, it’s not just about raw coverage. It’s about creating culture in which testing becomes second nature to everyone on the team. When writing tests becomes second nature, writing testable code becomes second nature. At this point you really start to see dividends.

Conclusion

So concluding this little overview, I’ll summarize the three main topics again. Code conformity. On a professional team of any size, it’s important for everyone to write code the same way. This helps spread good practices and makes maintaining other peoples’ code much easier. Style guides are a great starting point. Even seemingly simple style choices can impact code quality, robustness, and ease of refactoring. Active, healthy review process. Above maybe all else, it’s important to foster an active and healthy review process. At least one person besides the author should look at each diff, and suggestions should be gladly given and welcomed by team members of any level. This helps spread wisdom. It helps newer team members learn and develop good habits. It helps more senior members teach and stay honest. Ultimately it can keep your application from failing spectacularly. Testing. It’s important to have a strong testing ethic, and not to compromise or take shortcuts. The more we test, the more second nature it becomes, and we might eventually even learn to love it. It doesn’t have to start by decree, but can grow organically in a few spots until it begins sprouting up everywhere.

The post Maintaining Quality at Scale appeared first on Airbnb Engineering.

]]>
http://nerds.airbnb.com/maintaining-quality-scale/feed/ 2
Meet the Nerds: Scott Raymond http://nerds.airbnb.com/meet-nerds-scott-raymond/ http://nerds.airbnb.com/meet-nerds-scott-raymond/#comments Wed, 12 Nov 2014 00:26:15 +0000 http://nerds.airbnb.com/?p=176594518 We’re starting up (or restarting depending on how you look at it) a series that looks at life inside the Airbnb engineering team and introduces you to some of our engineers. In this first installment, Engineering Manager, Scott Raymond talks about his early days in computer science, transitioning from founding Gowalla to working on the […]

The post Meet the Nerds: Scott Raymond appeared first on Airbnb Engineering.

]]>

We’re starting up (or restarting depending on how you look at it) a series that looks at life inside the Airbnb engineering team and introduces you to some of our engineers. In this first installment, Engineering Manager, Scott Raymond talks about his early days in computer science, transitioning from founding Gowalla to working on the mobile apps at Airbnb, and of course his favorite Airbnb experience.

How did you get started in Computer Science?

My first rudimentary childhood programming was in BASIC, on a Commodore VIC-20, a hand-me-down from my uncle. The only storage was on cassette tape. A little later, I got access to a Mac SE/30 with HyperCard, and things really got rolling, making games and graphics programs. Eventually we got a modem, which allowed me to connect with the wider world of programming. In college, I actually studied Linguistics instead of comp sci — and every once in a while, I get the chance to apply that training in my work.

What was your path to Airbnb?

In 2007, I co-founded a company called Gowalla, whose mission was to to get people out exploring the world. The app was like a digital passport that you could fill with stamps — unique illustrations from thousands of places around the world. We were acquired by Facebook. After some time, I got to know a few folks here at Airbnb, and became enchanted by the product. It’s been really cool to pick up on some of the themes from Gowalla, especially connecting people with unique places.

What’s the most interesting technical challenge you’ve worked on since joining?

We are constantly looking for ways to improve the core experience (like performance and stability), running experiments to make the marketplace efficient, and building new features to make travel better. The hard part is doing all three at once. We spent a lot of time this year working on tools and processes that allow us to iterate really quickly, without letting quality slip. For one example, see my co-worker Zane’s recent post about our project to redesign Airbnb for iOS.

What do you want to work on next?

Right now, we’re thinking a lot about how people move between devices. They might use their tablet to browse listings, and their computer to book, and their phone to communicate with their host. I think we can make the seams in that experience much smoother. If we’re successful, the logistics of travel fade into the background, so that people can focus on the joy of travelling.

What is your favorite core value, and how do you live it?

Easy: “Be a Host”. I actually think about this all the time, even when planning how to code a feature in our app. There are all kinds of things that can go wrong with technology, especially when you’re travelling — you might be on a flaky network, run out of space on your phone, whatever. Like a great Airbnb host, our app can try to anticipate needs and deal with snafus graciously.

What’s your favorite Airbnb experience?

I recently took a great trip with my wife and 3-year-old daughter to Paris and Amsterdam. Our listing in Amsterdam was this delightful place on a canal, with the steepest staircase I’ve ever seen. We woke up in the morning and the street outside had transformed into a massive bustling Saturday market.

 

 

The post Meet the Nerds: Scott Raymond appeared first on Airbnb Engineering.

]]>
http://nerds.airbnb.com/meet-nerds-scott-raymond/feed/ 2
Taking Flight – Women in Tech Series http://nerds.airbnb.com/taking-flight-women-tech-series/ http://nerds.airbnb.com/taking-flight-women-tech-series/#comments Fri, 07 Nov 2014 22:53:54 +0000 http://nerds.airbnb.com/?p=176594504 This past June we hosted our first Taking Flight event. The evening centered around women in engineering and the presentations and discussions were wide ranging, looking at everything from “Infrastructure, demystified” to “How to build your network”. The highlight of the night was the breakout sessions where people were able to ask each other questions […]

The post Taking Flight – Women in Tech Series appeared first on Airbnb Engineering.

]]>
This past June we hosted our first Taking Flight event. The evening centered around women in engineering and the presentations and discussions were wide ranging, looking at everything from “Infrastructure, demystified” to “How to build your network”. The highlight of the night was the breakout sessions where people were able to ask each other questions and engage in a small group.

We left inspired by the camaraderie that was built from the gathering. We decided to make Taking Flight a more regular series and we are excited to share our additional installments. Last night, we hosted “Small Talks, Big Data” which featured lightening talks from women in Data Science (including our very own Lisa Qian).

Our next event will be held on Monday, November 17 with Niniane Wang, CEO of Evertoon (former CTO, Minted) and Varsha Rao, Head of Global Operations, Airbnb. They both will be discussing their experiences within the tech industry. We hope you can join us!

The post Taking Flight – Women in Tech Series appeared first on Airbnb Engineering.

]]>
http://nerds.airbnb.com/taking-flight-women-tech-series/feed/ 0
Interning with the Airbnb Payments Team http://nerds.airbnb.com/interning-airbnb-payments-team/ http://nerds.airbnb.com/interning-airbnb-payments-team/#comments Thu, 25 Sep 2014 15:43:42 +0000 http://nerds.airbnb.com/?p=176594486 My name is Rasmus Rygaard, and I’m a graduate student at Stanford University about to wrap up my masters in computer science. I have spent the last 12 weeks interning as a software engineer at Airbnb for the second summer in a row. This year, I joined the Payments team to help build out our […]

The post Interning with the Airbnb Payments Team appeared first on Airbnb Engineering.

]]>
My name is Rasmus Rygaard, and I’m a graduate student at Stanford University about to wrap up my masters in computer science. I have spent the last 12 weeks interning as a software engineer at Airbnb for the second summer in a row. This year, I joined the Payments team to help build out our global payments platform. It has been a fantastic experience, and I have been amazed by the kind of responsibility I have gotten as an intern. During my internship, I sped up our financial pipeline by more than an order of magnitude, isolated and upgraded the code that lets guests pay with credit cards, and set up payouts through local bank transfers that will save our hosts millions of dollars in fees.

Speeding up the Financial Pipeline

My main project for the summer was improving our financial pipeline. Airbnb is fortunate to have plenty of graphs that go up and to the right, but when those graphs track application performance, we would rather have them stay flat. To help our Finance team understand our business, the Payments team built a service to generate and serve key business metrics. That service, however, was not scaling with our business. The Finance team depends on having fresh data available at all times, but our infrastructure was struggling to deliver results fast enough. My task was clear yet open-ended: Speed up the pipeline.

I considered several approaches to speeding up the pipeline, but I ended up adapting our existing MySQL queries to be runnable in Presto. Presto, a distributed in-memory query engine built at Facebook, is heavily used for ad-hoc analytics queries at Airbnb. Presto’s in-memory computation provides significantly better performance than Hive, and for a task that required significant financial number-crunching, it seemed like the obvious choice. In addition to translating the queries, I built a pipeline that will help us seamlessly run complex queries in Presto even for data that lives in MySQL. The system is written in Ruby, which lets us leverage Presto’s computational power from our existing Rails application. As a result, the functionality that we have moved to our new infrastructure now runs 10 to 15 times faster.

Upgrading and Isolating Credit Card Code

My second project was to reorganize the way we process credit cards when you book a trip on Airbnb. I upgraded the JavaScript code that handles checkout logic so our system no longer needs to be exposed to sensitive credit card information. Now, we exchange the credit card information for a token in the browser when a user books a listing. This token works as a replacement for the credit card information with our credit card processing partner so we can complete the transaction exactly like before.

This was a high-impact project that had to be treated like surgery: We were replacing code on our perhaps most important page, and our guests should not experience any bumps along the way, nor should we cause any downtime. In addition to upgrading the code, I extracted this functionality into a separate module that we now share between our Rails and Node.js apps for web and mobile web. This required decomposing the existing code, but as a side-effect, we now have much more modularized code around capturing and processing transactions that we can test more aggressively.

Skærmbillede 2014-09-03 kl. 15.06.22

Paying Hosts in Local Currencies

Wherever possible, we want to pay our hosts in their local currency. A convenient way of doing that is through bank transfers that are faster and less expensive than international wire transfers. My final project for the summer was to work with a third party provider to start supporting local bank transfers in a number of markets where the existing methods for payouts are slow or expensive. I was the point person for our engineering side of the integration, working on the technical integration with our provider and on product decisions with our own Global Payments team.

We rely on several partners to support our platform, so the experience of partnering with another provider was not new to the team. I, however, had never worked this closely on a third party integration before. Still, my team trusted me to see through a project that would end up delivering payouts to hosts in countries around the world. Although work that requires syncing with an external company can be difficult to plan, the experience of balancing interests, requirements, and timelines between us and our partner gave me real world experience that no other class project or internship could provide. Our hosts in these markets can now look forward to saving millions of dollars in processing fees that would otherwise be charged by the providers of the existing payout methods.

My 12 weeks at Airbnb have been an amazing learning experience. As an intern, I have worked on projects that had significant impact on our hosts, guests, and core business. I have seen the global scale of our payments platform and how a small team of talented engineers make sure that money flows safely from guests to hosts on Airbnb. I’ve been blown away by the visible, high-impact projects that the company trusts interns with, and it’s clear that interns are treated just like full-time employees.

The post Interning with the Airbnb Payments Team appeared first on Airbnb Engineering.

]]>
http://nerds.airbnb.com/interning-airbnb-payments-team/feed/ 6
Inside the Airbnb iOS Brand Evolution http://nerds.airbnb.com/inside-airbnb-ios-brand-evolution/ http://nerds.airbnb.com/inside-airbnb-ios-brand-evolution/#comments Tue, 16 Sep 2014 15:55:20 +0000 http://nerds.airbnb.com/?p=176594459 Earlier this year we embarked on a six month project to refresh the mobile app, to coincide with the launch of our updated brand. The Airbnb product teams continued to ship new features for our mobile apps even during the months it took to implement the new brand. Logistically, this is the equivalent of repainting […]

The post Inside the Airbnb iOS Brand Evolution appeared first on Airbnb Engineering.

]]>
Earlier this year we embarked on a six month project to refresh the mobile app, to coincide with the launch of our updated brand. The Airbnb product teams continued to ship new features for our mobile apps even during the months it took to implement the new brand. Logistically, this is the equivalent of repainting a plane while it is in flight — without any of the passengers knowing it. When it landed, we also wanted it to be a surprise.

When we embarked upon the brand evolution we chose to use small and nimble engineering teams to prevent blocking each other. When the spec for a project is changing it is easy to have too many cooks in the kitchen as engineers begin to move faster than the shifting designs. Until the last month the team consisted of one engineer on each mobile application and three on the website. Part of the logic behind this decision was also an acknowledgement that design is an iterative process: no PSD file survives first contact with the user. A small engineering team is better able to adapt, with fewer places for communication to breakdown.

Brand Evolution Workshop

The design team had already been hard at work for months creating the new brand, but they brought the engineers into the room early to start helping to scope the project. At this point, we were considering:

  1. Design language: what would our “toolkit” look like (buttons, switches, etc.?)
  2. How could we improve the usability of the application without moving beyond the scope of a reskin?
  3. How could we do all this without disrupting other teams?

Even thinking within the confines of reskinning the application using a small team, it became apparent that there were some quick usability wins to be had. For example, we changed the background color of our side-drawer (Airnav) to be dark, which greatly improved readability. For contrast, here’s the before and after shots of our navigation system:

AirNav

We made the tabs on the Reservation screen more obviously buttons, which improved the ability of users to discover functionality. It was tempting to say “yes” to every great idea dreamed up in these design/engineering meetings, but the constraint of small engineering teams meant that we were hyper aware of our limited resources.

Toolkit

Above: one of the PSDs in our toolkit, defining modal and user picture styles. Those with a sharp eye will note that this does not exactly match the final version, which was one of the challenges we knew we would be facing during the process.

Build Engineering

We had two options for how to structure our workflow. We could have constrained the branding code to a long-lived Git branch, or we could have kept merging to master and split the XCode targets. We decided to pursue the latter approach for a number of reasons. Most notably, we decided that even though there was only one engineer responsible for the brand evolution itself, team members would be responsible for ensuring that new features were compatible with the brand evolution.

Build

After duplicating our target and swapping out the app icon, we now were faced with a new set of challenges. First, we needed to make sure no assets were leaked by accidental inclusion in the distribution target. We meticulously examined every .ipa which was built for leaks, and even went so far to use code words as symbol names in the code to prevent leakage through reverse engineering the binary.

Next, we needed to decide how to branch the code. On the surface this was simple: we just added a preprocessor macro to the new target called STYLE_PRETZEL. Then, we could use #if defined(STYLE_PRETZEL) in our code and rest assured that anything in the block would not be compiled into the existing deployment target. There were definitely drawbacks to this method, such as:

  • Code complexity: the codebase was quickly riddled with branching logic
  • No header changes: due to the multi-project nature of our XCode workspace, we could not be guaranteed that the macro would be available in a header file, so we could not use the #if defined in .h files
  • (Almost) no new classes: for both of the above reasons, it was generally best to not add any new classes, but rather to work with existing classes

In some ways, these restrictions actually helped us: it forced us to think about the minimum viable code change necessary to accomplish the task. In other words: if you’re writing lots of code, you’ve probably moved beyond reskinning.

Progressive Implementation

Our internal styling library, Nitrogen, allowed us to swap out colors, fonts, and basic component (“toolkit”) styles. This wholesale replacement was then followed by multiple progressive refinement passes across all the screens. We liked to think of it like the progressive rendering of images downloaded over a slow internet connection:

After the toolkit came the “first-cut”, at which point the app definitely looked new, but was still nowhere near Airbnb’s standard of design perfection; we essentially eyeballed the changes:

 First Cut

Next was “redlining.” The production team took the PSDs provided and used software to call out the exact fonts, colors, margins, etc.:

Redlining

With nested views, subclassed controllers, and other pitfalls it can sometimes be hard to know that the design is actually implemented to-spec. To check our work, we used Reveal to compare the correct values to the actual implementation:

reveal

All Hands on Deck

With about a month left until the big unveiling, we deleted the old XCode target and removed the preprocessor macro and #if defined statements, locking us into deploying the next version of the app with the brand evolution. Not only were all mobile team members now working with the reskinned app, but it was also distributed to employees of the company to start testing.

One of our mantras at Airbnb is that “every frame matters,” and now was the time to prove it. Designers began going through the app with a fine-tooth comb, and engineers’ queues filled with bug reports. It cannot be stressed enough how important these last few weeks were. This is the point at which we were moving text by one pixel here and one pixel there, yet all these pixels added up to the difference between a good app and a clean app.

Lessons Learned

The biggest oversight we made was to not think about localization earlier. Airbnb is an exceptionally global brand, and it is paramount that the app look stunning no matter what language it is shown in. This speaks to a greater need to “design for the dynamic.” We as engineers need to do a better job communicating with the design team about how they imagine the implementation changing. Not just for different sizes of text, but for animations, screen sizes, etc.

The early decision we made to split the XCode targets rather than creating a Git branch proved to be a good one. The result was that the rest of the team was at least superficially familiar with the new code, and when we called for All Hands On Deck there was minimum friction.

Conclusion

Overall, the process went exceptionally smoothly. Teams across the company continued to work on their own projects until just a few weeks before launch. On the big day, all of the teams pressed the “launch” button at the same time, and the world saw the new brand across all channels.

The post Inside the Airbnb iOS Brand Evolution appeared first on Airbnb Engineering.

]]>
http://nerds.airbnb.com/inside-airbnb-ios-brand-evolution/feed/ 5
How to Drive Growth: Lessons from Facebook, Dropbox, and More http://nerds.airbnb.com/drive-growth-lessons-facebook-dropbox/ http://nerds.airbnb.com/drive-growth-lessons-facebook-dropbox/#comments Tue, 01 Jul 2014 16:00:03 +0000 http://nerds.airbnb.com/?p=176594450 You’re inundated with tactics about how to grow your company. This talk will help you refine a framework for how to think about growth, and arm you with some tools to help hack through the bullshit. We’ll fly through a 50,000 foot view of the product development process used at Dropbox and Facebook to drive […]

The post How to Drive Growth: Lessons from Facebook, Dropbox, and More appeared first on Airbnb Engineering.

]]>
You’re inundated with tactics about how to grow your company. This talk will help you refine a framework for how to think about growth, and arm you with some tools to help hack through the bullshit.

We’ll fly through a 50,000 foot view of the product development process used at Dropbox and Facebook to drive growth.

We’ll dive into the nitty gritty: dozens of real examples of tests that worked and didn’t work.

Ivan Kirigin is an engineer and founder at YesGraph, a young startup focused on referral recruiting. Previous he lead growth at Dropbox, helping drive 12X growth in 2 years, and helped build Facebook Credits, and co-founded Tipjoy YC W08.

The post How to Drive Growth: Lessons from Facebook, Dropbox, and More appeared first on Airbnb Engineering.

]]>
http://nerds.airbnb.com/drive-growth-lessons-facebook-dropbox/feed/ 1
How to Search http://nerds.airbnb.com/search/ http://nerds.airbnb.com/search/#comments Sat, 28 Jun 2014 18:45:54 +0000 http://nerds.airbnb.com/?p=176594446 I have worked on search projects for around 7 years now in 3 companies and have had the privilege of working with smart search engineers from many other companies.Solving search problems together, we’ve found there are many ways to skin a cat. And we’ve formed opinions on the best ways among them. In this talk, […]

The post How to Search appeared first on Airbnb Engineering.

]]>
I have worked on search projects for around 7 years now in 3 companies and have had the privilege of working with smart search engineers from many other companies.Solving search problems together, we’ve found there are many ways to skin a cat. And we’ve formed opinions on the best ways among them.

In this talk, I’ll share some of these experiences and cover different approaches to deliver the best search results for our user. I’ll cover infrastructure (indexing strategies, sharding, efficient retrieval, etc.) as well as relevance (query rewriting, scoring, etc.). Finally, I’ll spend some time of frontend/product issues and ways to measure search quality.

Sriram Sankar is a Principal Staff Engineer at LinkedIn, where he is leading the development of their next-generation search infrastructure and search quality frameworks. Before that, he led Facebook’s search quality efforts for Graph Search, and was a key contributor to Unicorn, the index powering Graph Search. He previously worked at Google on search quality and ads infrastructure and has held senior technical roles at VMware, WebGain, and Sun. He is the author of JavaCC, one of the leading parser generators for Java. Sriram obtained his PhD from Stanford University, and a BS from the Indian Institute of Technology Kanpur.

The post How to Search appeared first on Airbnb Engineering.

]]>
http://nerds.airbnb.com/search/feed/ 1
Referrals at Airbnb: Driving Sustainable & Scalable Growth http://nerds.airbnb.com/referrals-airbnb-driving-sustainable-scalable-growth/ http://nerds.airbnb.com/referrals-airbnb-driving-sustainable-scalable-growth/#comments Sat, 28 Jun 2014 18:43:13 +0000 http://nerds.airbnb.com/?p=176594442 Word of mouth is the largest source of growth for Airbnb, in part because Airbnb experiences are so personal. People use Airbnb to unlock incredible experiences–anything from weekend getaways with friends, cultural exchanges, and once in a lifetime events like honeymoons. The Growth team builds products that helps users tell their stories. Our most successful […]

The post Referrals at Airbnb: Driving Sustainable & Scalable Growth appeared first on Airbnb Engineering.

]]>
Word of mouth is the largest source of growth for Airbnb, in part because Airbnb experiences are so personal. People use Airbnb to unlock incredible experiences–anything from weekend getaways with friends, cultural exchanges, and once in a lifetime events like honeymoons.

The Growth team builds products that helps users tell their stories. Our most successful program is Referrals which accounts for up to 30% of growth in certain markets. We launched Referrals in January on all three of our platforms: our website, our Android app, and our iOS app allowing users both to send and redeem referrals.

In this talk we will dive into the details of why we decided to build out Referrals, how we built it and the effect it have had on our growth trajectory. We’ll walk you through in-depth learnings and tips that we’ve acquired along the way. Finally, we’ll share what we’re working on now.

Jimmy Tang is an engineer on the Airbnb Growth team and built referrals for iOS. He works on all things growth from product, tools, and tracking. Previously, he cofounded Yozio, a platform for organic mobile growth.

Gustaf Alstromer is a product manager on the Airbnb Growth team and helped launched Referrals. Gustaf was previously Head of Growth at Voxer before he joined Airbnb in the end of 2012.

The post Referrals at Airbnb: Driving Sustainable & Scalable Growth appeared first on Airbnb Engineering.

]]>
http://nerds.airbnb.com/referrals-airbnb-driving-sustainable-scalable-growth/feed/ 0
Architecting a Machine Learning System for Risk http://nerds.airbnb.com/architecting-machine-learning-system-risk/ http://nerds.airbnb.com/architecting-machine-learning-system-risk/#comments Mon, 16 Jun 2014 16:30:45 +0000 http://nerds.airbnb.com/?p=176594368 Online risk mitigation At Airbnb, we want to build the world’s most trusted community.  Guests trust Airbnb to connect them with world-class hosts for unique and memorable travel experiences. Airbnb hosts trust that guests will treat their home with the same care and respect that they would their own.  The Airbnb review system helps users find community members who earn […]

The post Architecting a Machine Learning System for Risk appeared first on Airbnb Engineering.

]]>
Online risk mitigation

At Airbnb, we want to build the world’s most trusted community.  Guests trust Airbnb to connect them with world-class hosts for unique and memorable travel experiences. Airbnb hosts trust that guests will treat their home with the same care and respect that they would their own.  The Airbnb review system helps users find community members who earn this trust through positive interactions with others, and the ecosystem as a whole prospers.

The overwhelming majority of web users act in good faith, but unfortunately, there exists a small number of bad actors who attempt to profit by defrauding websites and their communities.  The trust and safety team at Airbnb works across many disciplines to help protect our users from these bad actors, ideally before they have the opportunity to impart negativity on the community.

There are many different kinds of risk that online businesses may have to protect against, with varying exposure depending on the particular business. For example, email providers devote significant resources to protecting users from spam, whereas payments companies deal more with credit card chargebacks.

We can mitigate the potential for bad actors to carry out different types of attacks in different ways.

1) Product changes

Many risks can be mitigated through user-facing changes to the product that require additional verification from the user. For example, requiring email confirmation, or implementing 2FA to combat account takeovers, as many banks have done.

2) Anomaly detection

Scripted attacks are often associated with a noticeable increase in some measurable metric over a short period of time. For example, a sudden 1000% increase in reservations in a particular city could be a result of excellent marketing, or fraud.

3) Simple heuristics or a machine learning model based on a number of different variables

Fraudulent actors often exhibit repetitive patterns.  As we recognize these patterns, we can apply heuristics to predict when they are about to occur again, and help stop them.  For complex, evolving fraud vectors, heuristics eventually become too complicated and therefore unwieldy.  In such cases, we turn to machine learning, which will be the focus of this blog post.

For a more detailed look at other aspects of online risk management, check out Ohad Samet’s great ebook.

Machine Learning Architecture

Different risk vectors can require different architectures. For example, some risk vectors are not time critical, but require computationally intensive techniques to detect. An offline architecture is best suited for this kind of detection. For the purposes of this post, we are focusing on risks requiring realtime or near-realtime action. From a broad perspective, a machine-learning pipeline for these kinds of risk must balance two important goals:

  1. The framework must be fast and robust.  That is, we should experience essentially zero downtime and the model scoring framework should provide instant feedback.  Bad actors can take advantage of a slow or buggy framework by scripting many simultaneous attacks, or by launching a steady stream of relatively naive attacks, knowing that eventually an unplanned outage will provide an opening.  Our framework must make decisions in near real-time, and our choice of a model should never be limited by the speed of scoring or deriving features.
  1. The framework must be agile Since fraud vectors constantly morph, new models and features must be tested and pushed into production quickly.  The model-building pipeline must be flexible to allow data scientists and engineers to remain unconstrained in terms of how they solve problems.

These may seem like competing goals, since optimizing for realtime calculations during a web transaction creates a focus on speed and reliability, whereas optimizing for model building and iteration creates more of a focus on flexibility. At Airbnb, engineering and data teams have worked closely together to develop a framework that accommodates both goals: a fast, robust scoring framework with an agile model-building pipeline.

Feature Extraction and Scoring Framework

In keeping with our service-oriented architecture, we built a separate fraud prediction service to handle deriving all the features for a particular model. When a critical event occurs in our system, e.g., a reservation is created, we query the fraud prediction service for this event. This service can then calculate all the features for the “reservation creation” model, and send these features to our Openscoring service, which is described in more detail below. The Openscoring service returns a score and a decision based on a threshold we’ve set, and the fraud prediction service can then use this information to take action (i.e., put the reservation on hold).

The fraud prediction service has to be fast, to ensure that we are taking action on suspicious events in near realtime. Like many of our backend services for which performance is critical, it is built in java, and we parallelize the database queries necessary for feature generation.  However, we also want the freedom to occasionally do some heavy computation in deriving features, so we run it asynchronously so that we are never blocking for reservations, etc. This asynchronous model works for many situations where a few seconds of delay in fraud detection has no negative effect. It’s worth noting, however, that there are cases where you may want to react in realtime to block transactions, in which case a synchronous query and precomputed features may be necessary.  This service is built in a very modular way, and exposes an internal restful API, making adding new events and models easy.

Openscoring

Openscoring is a Java service that provides a JSON REST interface to the Java Predictive Model Markup Language (PMML) evaluator JPMML.  Both JPMML and Openscoring are open source projects released under the Apache 2.0 license and authored by Villu Ruusmann (edit – the most recent version is licensed the under AGPL 3.0) .  The JPMML backend of Openscoring consumes PMML, an xml markup language that encodes several common types of machine learning models, including tree models, logit models, SVMs and neural networks.  We have streamlined Openscoring for a production environment by adding several features, including kafka logging and statsd monitoring.  Andy Kramolisch has modified Openscoring to permit using several models simultaneously.

As described below, there are several considerations that we weighed carefully before moving forward with Openscoring:

Advantages

  • Openscoring is opensource - this has allowed us to customize Openscoring to suit our specific needs.
  • Supports random forests - we tested a few different learning methods and found that random forests had an appropriate precision-recall for our purposes.
  • Fast and robust - after load testing our setup, we found that most responses took under 10ms.
  • Multiple models - after adding our customizations, Openscoring allows us to run many models simultaneously.
  • PMML format - PMML allows analysts and engineers to use any compatible machine learning package (R, Python, Java, etc.) they are most comfortable with to build models.  The same PMML file can be used with pattern to perform large-scale distributed model evaluation in batch via cascading.

Disadvantages

  • PMML doesn’t support some types of models - PMML only supports relatively standard ML models; therefore, we can’t productionize bleeding-edge models or implement significant modifications to standard models.
  • No native support for online learning - models cannot train themselves on-the-fly. A secondary mechanism needs to be in place to automatically account for new ground truth.
  • Rudimentary error handling -  PMML is difficult to debug.  Editing the xml file by hand is a risky endeavor.  This task is best left to the software packages, so long as they support the requisite features.  JPMML is known to return relatively cryptic error messages when the PMML is invalid.

After considering all of these factors, we decided that Openscoring best satisfied our two-pronged goal of having a fast and robust, yet flexible machine learning framework.

Model Building Pipeline

hackpad.com_86HnsLzAU5J_p
A schematic of our model-building pipeline using PMML is illustrated above.  The first step involves deriving features from the data stored on the site.  Since the combination of features that gives the optimal signal is constantly changing, we store the features in a json format, which allows us to generalize the process of loading and transforming features, based on their names and types.  We then transform the raw features through bucketing or binning values, and replacing missing values with reasonable estimates to improve signal.  We also remove features that are shown to be statistically unimportant from our dataset.  While we omit most of the details regarding how we perform these transformations for brevity here, it is important to recognize that these steps take a significant amount of time and care.  We then use our transformed features to train and cross-validate the model using our favorite PMML-compatible machine learning library, and upload the PMML model to Openscoring.  The final model is tested and then used for decision-making if it becomes the best performer.

The model-training step can be performed in any language with a library that outputs PMML.  One commonly used and well-supported library is the R PMML package.  As illustrated below, generating a PMML with R requires very little code.

This R script has the advantage of simplicity, and a script similar to this is a great way to start building PMMLs and to get a first model into production.  In the long run, however, a setup like this has some disadvantages.  First, our script requires that we perform feature transformation as a pre-processing step, and therefore we have add these transformation instructions to the PMML by editing it afterwards.  The R PMML package supports many PMML transformations and data manipulations, but it is far from universal.   We deploy the model as a separate step — post model-training — and so we have to manually test it for validity, which can be a time-consuming process.  Yet another disadvantage of R is that the implementation of the PMML exporter is somewhat slow for a random forest model with many features and many trees.  However, we’ve found that simply re-writing the export function in C++ decreases run time by a factor of 10,000, from a few days to a few seconds. We can get around the drawbacks of R while maintaining its advantages by building a pipeline based on Python and scikit-learn.  Scikit-learn is a Python package that supports many standard machine learning models, and includes helpful utilities for validating models and performing feature transformations.  We find that Python is a more natural language than R for ad-hoc data manipulation and feature extraction.  We automate the process of feature extraction based on a set of rules encoded in the names and types of variables in the features json; thus, new features can be incorporated into the model pipeline with no changes to the existing code.  Deployment and testing can also be performed automatically in Python by using its standard network libraries to interface with Openscoring.  Standard model performance tests (precision recall, ROC curves, etc.) are carried out using sklearn’s built-in capabilities.  Sklearn does not support PMML export out of the box, so have written an in-house exporter for particular sklearn classifiers.  When the PMML file is uploaded to Openscoring, it is automatically tested for correspondence with the scikit-learn model it represents.   Because feature-transformation, model building, model validation, deployment and testing are all carried out in a single script, a data scientist or engineer is able to quickly iterate on a model based on new features or more recent data, and then rapidly deploy the new model into production.

Takeaways: ground truth > features > algorithm choice

Although this blog post has focused mostly on our architecture and model building pipeline, the truth is that much of our time has been spent elsewhere.  Our process was very successful for some models, but for others we encountered poor precision-recall.  Initially we considered whether we were experiencing a bias or a variance problem, and tried using more data and more features.  However, after finding no improvement, we started digging deeper into the data, and found that the problem was that our ground truth was not accurate.

Consider chargebacks as an example. A chargeback can be “Not As Described (NAD)” or “Fraud” (this is a simplification), and grouping both types of chargebacks together for a single model would be a bad idea because legitimate users can file NAD chargebacks. This is an easy problem to resolve, and not one we actually had (agents categorize chargebacks as part of our workflow); however, there are other types of attacks where distinguishing legitimate activity from illegitimate is more subtle, and necessitated the creation of new data stores and logging pipelines.

Most people who’ve worked in machine learning will find this obvious, but it’s worth re-stressing:
If your ground truth is inaccurate, you’ve already set an upper limit to how good your precision and recall can be. If your ground truth is grossly inaccurate, that upper limit is pretty low. 

Towards this end, sometimes you don’t know what data you’re going to need until you’ve seen a new attack, especially if you haven’t worked in the risk space before, or have worked in the risk space but only in a different sector. So the best advice we can offer in this case is to log everything. Throw it all in HDFS, whether you need it now or not. In the future, you can always use this data to backfill new data stores if you find it useful. This can be invaluable in responding to a new attack vector.

Future Outlook

Although our current ML pipeline uses scikit-learn and Openscoring, our system is constantly evolving.  Our current setup is a function of the stage of the company and the amount of resources, both in terms of personnel and data, that are currently available.  Smaller companies may only have a few ML models in production and a small number of analysts, and can take time to manually curate data and train the model in many non-standardized steps.  Larger companies might have many, many models and require a high degree of automation, and get a sizable boost from online training.  A unique challenge of working at a hyper-growth company is that landscape fundamentally changes year-over-year, and pipelines need to adjust to account for this.

As our data and logging pipelines improve, investing in improved learning algorithms will become more worthwhile, and we will likely shift to testing new algorithms, incorporating online learning, and expanding on our model building framework to support larger data sets.  Additionally, some of the most important opportunities to improve our models are based on insights into our unique data, feature selection, and other aspects our risk systems that we are not able to share publicly.  We would like to acknowledge the other engineers and analysts who have contributed to these critical aspects of this project.  We work in a dynamic, highly-collaborative environment, and this project is an example of how engineers and data scientists at Airbnb work together to arrive at a solution that meets a diverse set of needs.  If you’re interested in learning more, contact us about our data science and engineering teams!

 

*Illustration above by Shyam Sundar Srinivasan

The post Architecting a Machine Learning System for Risk appeared first on Airbnb Engineering.

]]>
http://nerds.airbnb.com/architecting-machine-learning-system-risk/feed/ 19