Biography

Alexey is a Software Engineer at Uber in Amsterdam, focusing on empowering Uber engineers to build high quality software with reliable guarding rails, frictionless workflow and minimal time waiting on computers. He holds a degree in Computer Science from the RWTH Aachen and previously worked at Basware and IBM.

Talk

Handling Flaky Tests at Uber’s Scale

In just 10 years, Uber has scaled from zero to over $10 billion in annual revenue [1], mostly coming from apps like the eponymous rider app, the Eats food delivery app, or, recently Uber freight. As an app business, Uber relies on thousands of software engineers delivering code daily.
This scale necessitates automated testing and code integration. Each line written by a dev runs through a large series of tests before being integrated into a release, running up to 80,000 unit tests. In order to manage this scale, Uber uses significant parallelization. Parallelization however may cause unexpected test failure, preventing users from pushing their code and hindering developer productivity. In a first effort to identify and isolate tests, certain tests were identified as flaky and were to be run serially, however this manual behavior of moving tests around did not scale. After all, neither the test nor the underlying software can be assumed to be static.

Join this talk to learn how Uber is tackling the test flakiness problem by introducing a language-agnostic Test Analyzer Service (TAS) solution which automatically changes a tests’ level of isolation between ‘in quarantine’, serial, or parallel and learn about where we see the future of test handling.

Citations
[1] Uber Investor relations, 2018 https://investor.uber.com/

Session Keywords