How we optimized Magic Pocket for cold storage

Ever since we launched Magic Pocket, our in-house multi-exabyte storage system, we’ve been continuously looking for opportunities to improve efficiency, while maintaining our high standards for reliability. Last year, we pushed the limits of storage density by being the first major tech company to adopt SMR storage. In this post, we’ll discuss another advance in storage technology at Dropbox: a new cold storage tier that’s optimized for less frequently accessed data. This storage runs on the same SMR disks as our more active data, and through the same internal network.

The Lifetime of a file

The access characteristics of a file at Dropbox varies heavily over time.

Read more

Using machine learning to predict what file you need next

As we laid out in our blog post introducing DBXi, Dropbox is building features to help users stay focused on what matters. Searching through your content can be tedious, so we built content suggestions to make it easier to find the files you need, when you need them.

We’ve built this feature using modern machine learning (ML) techniques, but the process to get here started with a simple question: how do people find their files? What kinds of behavior patterns are most common? We hypothesized the following two categories would be most prevalent:

  • Recent files: The files you need are often the ones you’ve been using most recently.

Read more

Embracing papercuts

Team growth requires giving people room to make mistakes. Figuring out which mistakes are just “papercuts” and which are critical is one of the most difficult challenges in engineering leadership.

We’ve all seen “helicopter parents,” hovering over their kids to catch them at the slightest inclination they might fall. We swear we’d never do that, that we’d give our kids room to grow and learn from mistakes. Then we become tech leads and turn into the worst kind of “helicopter leaders.”

I was certainly guilty of micromanagement. It started with code reviews, commenting on every minor issue I could find. 

Read more

Towards better vendor security assessments

Addressing vendor security is a significant and inescapable problem for any modern company. Like many other companies, Dropbox has external third-party integrations with our products, and we also use vendors for internal services, from HR workflows to sales, marketing, and IT. In many ways, vendors play a critical part in Dropbox’s overall security posture and thus require appropriate scrutiny from our security team based on the risk posed by the vendor and feasible mitigations.

Today, we’re sharing the results of an experiment to improve vendor security assessments—directly codifying reasonable security requirements into our vendor contracts. We’re also sharing our model security legal terms and making them freely available for anyone to use and modify.

Read more

On working with designers

Editor’s note: On January, 18, 2019 the Dropbox Design blog featured a post from Product Designer, Jenny Wen, on working with engineers. This post covers the topic from an engineer’s perspective.

One of my favorite things as an engineer building Paper is how closely I get to work with designers. It’s an important partnership. When we share the same goals, work closely together, and understand what is important to each other we can create things that we would never be able to accomplish on our own. The opposite is also true. When we don’t align early,

Read more

Don’t lead by example

This is the first in a series of posts that Dropbox Principal Engineer James Cowling has published on his personal Medium blog about technical leadership. Being a strong tech lead is very different from being a strong engineer and we thought the readers of our tech blog would find his experiences relevant and interesting.

Back when I was a first-time tech lead at Dropbox I had the misfortune of juggling two intimidating responsibilities at the same time:

  1. Build a multi-exabyte distributed storage system and migrate our data off Amazon S3,

Read more