Facebook wanted to ensure that its new flat iOS 7 design would work out of the box and they beta tested in a rather sneaky but clever way.
Facebook changed the apps navigation by adding access to the News Feed, Messages, and Notifications to the bottom of the app, while also making other changes.
The team at The Verge discovered that Facebook decided on those changes by covertly sneaking certain options onto already downloaded apps.
The social network began tweaking the platform by pushing out updates to 8 million users. The company built “an entire system for creating alternate versions of the native app within the native app.”
Facebook was then able to push out certain features for a subset of its users, while measure the results of its efforts.
According to The Verge “Starting in early 2013, the team put together a system of “different types of Legos — that we can reconfigure really easily … and see the results on the server in real time.”
Testing a sub-set of features while measuring results could be the future of app update release testing. Customers get to chime in on new features without a 100% jarring experience from a new app.