Our journey to type checking 4 million lines of Python

Dropbox is a big user of Python. It’s our most widely used language both for backend services and the desktop client app (we are also heavy users of Go, TypeScript, and Rust). At our scale—millions of lines of Python—the dynamic typing in Python made code needlessly hard to understand and started to seriously impact productivity. To mitigate this, we have been gradually migrating our code to static type checking using mypy, likely the most popular standalone type checker for Python. (Mypy is an open source project, and the core team is employed by Dropbox.)

Dropbox has been one of the first companies to adopt Python static type checking at this scale.

Read more

Incrementally migrating over one million lines of code from Python 2 to Python 3

The Dropbox desktop client is relied on by millions of users across the world to save their most important files and keep them in sync across their devices. Weighing in at over 1 million lines of Python logic, we had a massive surface area for potential issues in our migration from Python 2 to Python 3. In this process, we knew that we had to be worthy of the trust that users place in Dropbox and keep their information safe.

Over the last few months, we’ve explored why and how we rolled out our Python 3 migration,

Read more

Crash reporting in desktop Python applications

One of the greatest challenges associated with maintaining a complex desktop application like Dropbox is that with hundreds of millions of installs, even the smallest bugs can end up affecting a very large number of users. Bugs inevitably will strike, and while most of them allow the application to recover, some cause the application to terminate. These terminations, or “crashes,” are highly disruptive events: when Dropbox stops, synchronization stops. To ensure uninterrupted sync for our users we automatically detect and report all crashes and take steps to restart our application when they occur.

In 2016, faced with our impending transition to Python 3,

Read more

How we rolled out one of the largest Python 3 migrations ever

Dropbox is one of the most popular desktop applications in the world: You can install it today on Windows, macOS, and some flavors of Linux. What you may not know is that much of the application is written using Python. In fact, Drew’s very first lines of code for Dropbox were written in Python for Windows using venerable libraries such as pywin32.

Though we’ve relied on Python 2 for many years (most recently, we used Python 2.7), we began moving to Python 3 back in 2015. This transition is now complete: If you’re using Dropbox today,

Read more

Introducing Pyston: an upcoming, JIT-based Python implementation

Hello everyone, I’m very excited to announce Pyston, a new open-source implementation of Python, currently under development at Dropbox.  The goal of the project is to produce a high-performance Python implementation that can push Python into domains dominated by traditional systems languages like C++.

Here at Dropbox, we love Python and try to use it for as much as we can.  As we scale and the problems we tackle grow, though, we’re starting to find that hitting our performance targets can sometimes become prohibitively difficult when staying on Python.  Sometimes, it can be less work to do a rewrite in another language.  

Read more

Plop: Low-overhead profiling for Python

It’s almost time for another Hack Week at Dropbox, and with that in mind I’d like to present one of the projects from our last Hack Week.

A profiler is an indispensable tool for optimizing programs.  Without a profiler, it’s hard to tell which parts of the code are consuming enough time to be worth looking at.  Python comes with a profiler called cProfile, but enabling it slows things down so much that it’s usually only used in development or simulated scenarios, which may differ from real-world usage.

At our last hack week, I set out to build a profiler that would be usable on live servers without impacting our users.  

Read more