Inside the Magic Pocket

We’ve received a lot of positive feedback since announcing Magic Pocket, our in-house multi-exabyte storage system. We’re going to follow that announcement with a series of technical blog posts that offer a look behind the scenes at interesting aspects of the system, including our protection mechanisms, operational tooling, and innovations on the boundary between hardware and software. But first, we’ll need some context: in this post, we’ll give a high level architectural overview of Magic Pocket and the criteria it was designed to meet.

Read more

Scaling to exabytes and beyond

Years ago, we called Dropbox a “Magic Pocket” because it was designed to keep all your files in one convenient place. Dropbox has evolved from that simple beginning to become one of the most powerful and ubiquitous collaboration platforms in the world. And when our scale required building our own dedicated storage infrastructure, we named the project “Magic Pocket.” Two and a half years later, we’re excited to announce that we’re now storing and serving over 90% of our users’ data on our custom-built infrastructure.

Read more

Open Sourcing Pytest Tools

At Dropbox, we made the switch from testing with unittest to pytest. We love the features, fixtures, plugins, and customizability of pytest. To further improve our experience, we built a couple of tools (pytest-flakefinder, unittest2pytest) for working with pytest and released them as open source.

We developed the pytest-flakefinder plugin to help with a common problem, flaky tests. Tests that involve multiple threads, or that depend on certain ordering can often fail at a fairly low rate. A few flaky tests aren’t a big deal,

Read more

What do you mean ‘we need more time’??

Project Schedule Estimation in Software Development

In tech, we spend little time talking about the softer skills like communication, project management, and prioritization. These are the skills that elevate someone from a good programmer to a great software engineer. Today, I’m going to focus on one aspect of project management that we’re famously bad at — the art of estimating a project schedule.

If there’s any doubt that this is a necessary skill, just consider that dreaded but frequently-asked question “How long will it take?” Even if you’re uber-Agile and don’t believe in far-off project deadlines,

Read more

Inside LAN Sync

Dropbox LAN Sync is a feature that allows you to download files from other computers on your network, saving time and bandwidth compared to downloading them from Dropbox servers. As the number of companies and offices using Dropbox has increased, the use cases for LAN Sync have grown, and the feature was recently rewritten and improved. Here’s a look inside how it works.

Read more

Open Sourcing Zulip – a Dropbox Hack Week Project

This year’s Dropbox Hack Week saw some incredible projects take shape – from the talented team that visited Baltimore to research food deserts, to a project to recreate the fictional Pied Piper algorithm from HBO’s Silicon Valley. One of the most special elements of Hack Week, though, is that often times we’re able to share these exciting projects openly with our users and our community.

At Dropbox, we love and depend on numerous excellent open source projects, and we consider contributing back to the open source community to be vitally important. Popular open source projects that Dropbox has released include the zxcvbn password strength estimator,

Read more