allow for the delegation of specific test machines to individual trusted
developers. More importantly, this allows not only for manually testing
software on those machines, but also for automating several tasks, such as the
Git bisection mentioned earlier.
Final thoughts
Testing a whole operating system is a challenging task, because there are a lot
of moving parts. OpenQA is a great tool for that, but its main target is
running tests in a virtualized environment. This works fine for several
components (like Qubes Manager and GUI virtualization) but not for
hardware-related features (sys-net, sys-usb, system suspend, and several
others). Before this work, we ran tests on actual laptops manually, but that
was time-consuming and thus not all updates or configurations were tested.
Automation allows our testing to be much more comprehensive, including ensuring
ongoing compatibility with Qubes OS certified hardware.