How to Write a Better Scribe

Like many companies, Dropbox uses scribe to aggregate log data into our analytics pipeline. After a recent scribe pipeline outage, we decided to rewrite scribe with the goals of reducing operational overhead, reducing data loss, and adding enhancements that are missing from the original scribe. This blog post describes some of the design choices we made for the rewrite.

Scribe Basics

This section describes the scribe pipeline with respect to how it is setup at Dropbox (we suspect most companies deploy/use scribe in similar fashion).  Feel free to skip this section if you are already familiar with scribe.

Read more