Optimizing web servers for high throughput and low latency


This is an expanded version of my talk at NginxConf 2017 on September 6, 2017. As an SRE on the Dropbox Traffic Team, I’m responsible for our Edge network: its reliability, performance, and efficiency. The Dropbox edge network is an nginx-based proxy tier designed to handle both latency-sensitive metadata transactions and high-throughput data transfers. In a system that is handling tens of gigabits per second while simultaneously processing tens of thousands latency-sensitive transactions, there are efficiency/performance optimizations throughout the proxy stack, from drivers and interrupts, through TCP/IP and kernel, to library, and application level tunings.

Read more

Accelerating Iteration Velocity on Dropbox’s Desktop Client, Part 2

In our previous blog post on investing in the Desktop Client platform at Dropbox, we discussed the challenges of trying to innovate and iterate on a product while maintaining high platform quality and low overhead. In 2016, Dropbox quadrupled the cadence at which we shipped the Desktop Client, releasing a new a major version every 2 weeks rather than every 8 weeks by investing in foundational improvements. These efforts tended to illustrate one or both of the following themes:

  1. Reduce KTLO work: “Keeping The Lights On,” or KTLO, includes manual tasks such as setting configuration parameters,

Read more

Accelerating Iteration Velocity on Dropbox’s Desktop Client, Part 1

Motivation

Imagine you’re an engineer working on a new product feature that is going to have a high impact on the end user, like the Dropbox Badge. You want to get quick validation on the functionality and utility of the feature. Each individual change you make might be relatively simple, like a tweak to the CSS changing the size of a font, or more substantial, like enabling the Badge on a new file type. You could set up user studies, but these are relatively expensive and slow, and are a statistically small sample size. Ideally,

Read more