Edgestore is the metadata store that powers many internal and external Dropbox services and products. We first talked about Edgestore in late 2013 and needless to say, much has happened since.
In this post, we give a high-level overview of the motivation behind Edgestore, its architecture, salient features and how it’s being used at Dropbox. We’ll be doing a deep-dive on various aspects of Edgestore in subsequent posts.
Like so many startups, Dropbox started with vanilla MySQL databases for our metadata needs. As we rapidly added both users and features, we soon ended up with multiple, independent databases; some databases grew so large that we had to split them into multiple shards. And before long, unsurprisingly, we started hitting challenges of such an architecture, such as:
In late 2012, we began building a system that would address these challenges. In addition, we wanted the following characteristics to meet future needs:
For brevity, we’re glossing over many technical details here. Please check back for our follow-up posts where we’ll deep-dive into the special topics listed under “Coming Up” below!
Thanks to: Alex Degtiar, Adil Hafeez, Bogdan Munteanu, Chris Roberson, Daniel Tahara, Kerry Xing, Maxim Bublis, Mehant Baid, Michelle Chesley, Mihnea Giurgea, Rajat Goel, Renjish Abraham, Samir Goel, Tom Manville and Zviad Metreveli