Within a matter of hours of each day beginning, we ingest tens of billions of records from online sources to HDFS, aggregated across 50+ offline data flows, and visualize 40+ metrics across our business lines to support a dashboard providing company leaders with timely and critical business insights. Dubbed the Daily Executive Dashboard (DED), this dashboard contains critical growth, engagement, and bookings metrics that indicate the health of LinkedIn’s business. In this blog post, we’ll spotlight the challenges to DED’s delivery and performance posed by the growing complexity of computing and aggregating these metrics, and how this dashboard transformed our approach to data pipelines across the company.
