Detecting and diagnosing test regressions is critical to maintaining stable, high-quality code. Speed is critical: if new changes are committed while tests are failing, then it is much harder to diagnose which test failures were caused by which changes. To find out what code change caused a regression, a developer needs to examine the test's history. However, existing test history visualizations are slow, employ poor visual encodings, and are hard to use.
This CSE 512 project investigates how better visualizations can aid in diagnosing test regressions in the context of WebKit, a large open source project. We present a redesigned dashboard prototype that uses small multiples and compact visual encodings to succinctly present the recent history of multiple tests and configurations. Users can filter visible timelines according to criteria such as test name, file path, platform, expected result, or date range. Timelines are interactive, and more detailed views about the particular test or build run are shown on demand.
Try it online here.