Modernising firmware testing on POWER with Python and pre-release everything

Presented by Stewart Smith
Friday 11:40 a.m.–12:25 p.m. in Collaborative Lecture Theatre CB11.00.405
Target audience: Developer

Abstract

Some people have the odd expectation that they will be able to (reliably) turn their computers on and off. On top of that, other people seem to have the entirely unreasonable expectation to have their computers reliably boot an Operating System, perform consistently, and not catch fire. In order to meet these outrageous expectations, we need to test firmware before we release it to an unsuspecting public. After all, isn't the gold standard of testing "if it boots, ship it" ? In the era of Blockchain, the gold standard no longer applies. As a firmware maintainer, I want the answer to a simple question: "Will I regret merging this code?". A simple and quick automated test suite is in order! Unfortunately, such things are not always around when you need them. In order to successfully deliver POWER9 systems (new processor, new operating systems, new machines, new BMC software), we would need a high quality, reliable, fast, easy to use, automated firmware test suite that could not only spot regressions, but find new bugs. This is a tale of code and organizational transformation and discovery during an incredibly tight and largely immobile release schedule. We'll cover: - Components of a modern POWER server and how one might test them (on purpose or by accident) - Practical use of Python in a mix of modern and legacy environments without going insane - (ab)using existing testing frameworks for (almost) fun and (hopefully) profit - How to (not) live with old enterprise distros - Transforming development, test, and management organizations, into test focused ones. - How on earth do you design a firmware compliance test before you have a test suite anyway? This talk is brought to you by: - A Perl script calling a perl script that parses XML with regex that ran a shell script to construct a python script to shell out to expect. - Unexpected bugs in the TTY layer - Soft Lockups - SOL Disconnected by BMC.

Presented by

Stewart Smith

Stewart currently works for IBM in the Linux Technology Center on OPAL, the OpenPOWER Abstraction Layer - open source firmware for POWER. He currently finds it weird when you don't have the source code to your firmware. He's also the Vice Chair of the Technical Steering Committee of the OpenPOWER Foundation and sits in several working groups. Previously, Stewart worked for Percona as Director of Server Development with a deep background in database internals including MySQL, MySQL Cluster, Drizzle, InnoDB and HailDB. He was one of the founding core developers of the Drizzle database server project which he worked on at Sun Microsystems and later Rackspace. He started in the database world at MySQL AB working on MySQL Cluster.