Testing

This article argues for the value of example code for software projects. The primary audience are software engineers working on libraries and open-source code. Executable example code is probably the most valuable artifact one can add to projects ranging from single-purpose libraries to larger frameworks. This applies from the API design phase of a library to maintaining them in the long term. The repositories of the Android libraries that I maintain(ed) typically include dedicated sample apps. For example, the sample app for the Sloth library or the sample app for the Spectrum library. ...

This article describes how to add a CLI module to an existing Android Studio project that generates an executable .jar archive. The primary audience are software engineers working on Android. When developing for Android – or other mobile platforms – the change-to-effect latency can be often quite high due to the involved compilation and cross-device communication. The regular unit tests get around this by executing in a VM on the host machine. However, they can be inflexible when it comes to incorporating larger (binary) assets to test against. Also, they do not lend themself to facilitate long fuzzing session or interactive usage. ...

This article explains an often-overlooked effect that can distort metrics during feature roll-out. The primary audience are software engineers and managers working on infrastructure and libraries. I want to motivate this problem with a fictive scenario: Alex wrote a replacement for an old and inefficient component in their app. It took the team a few weeks to implement and they carefully measured on test devices that it improves all core metrics. It is faster, uses less memory, all the good stuff. Following procedure, Alex’s new code is bundled with the next release and they open it to 1% for an A/B test. ...

This article explains advanced A/B test concepts. The primary audience are engineers encountering such A/B tests for the first time. A/B tests allow engineers to validate fixes, test for regressions, and measure improvements. In most situations the standard approach is to create a control and a test group of equal size (e.g. both 5%). Users within the test population (e.g. 10%) are then randomly assigned to one of these. However, sometimes more intricate methods can improve the user experience and make deployment safer and more effective. ...