A few years ago while browsing the App Store, I noticed a ‘featured’ app that looked great at first glance, although I was a little surprised to see that it had a very low rating from a significant number of users. Intrigued, I continued to the reviews page to discover that the latest update had been botched big time. Users that had updated had lost all of their data… ouch. The ‘What’s New’ blurb contained a frantic message asking users not to update, while a fixed version awaited an Apple review. It must have been a pretty stressful night or two for that developer!

Recently I released an update for Class Timetable, an app with millions of downloads, and a very large number of active users. The update was significant, and involved migrating data stores, moving to a new cloud storage/sync method, a radically different Apple Watch sync strategy, and a complete UI makeover. I don’t have a team of people working on the app; most of the release was self orchestrated. Just for fun, the update was released a week or two ago, during the back-to-school season for many countries (when Class Timetable usage is through the roof). It was the hardest release I’ve attempted to date, and I’m glad to say that the rollout went astoundingly smoothly, with positive reviews flooding in since day one.

After a few slightly bumpy releases in the past (that I’ve learnt a lot from), I’m proud of how well the update went. I’m certainly not going to claim that I’ve discovered the perfect release procedure, but what I would like to do is share how I managed to pull off a complex, large scale release, with relatively few issues. Hopefully there’s something here that is relevant to both indie developers and organizations alike.

Test Your App Well

If you’re fortunate enough to have a dedicated tester or testing department, great. I didn’t have that, and I’d suggest that even if you do, a wider range of testing environments is a good idea. For Class Timetable, I used the following methods to test the app:

• TestFlight. Set it up well, with an internal group that yourself and others inside your organization can access (if you’re an indie dev, perhaps that just means you), with bleeding edge builds. Create an external testing group, and treat releases here with a much higher regard.

• Use past customer support emails to gather a group of beta testers. Selectively choose people that seem like good candidates (e.g. an angry customer who wants a refund is clearly not, while somebody that sent a feature suggestion is excellent). Send them an email, along the lines of:

Hi James, you’ve emailed us in the past in regards to Class Timetable. We’re releasing a brand new version really soon and would like a few people to give us some feedback before it’s released. Etc …

I’d suggest spending time creating a professionally written, well worded email. The uptake rate is surprisingly high; In my case it seems that people likely to send a feature suggestion email are highly correlated with users likely to try beta software. Gather a reasonable number of testers. For the recent Class Timetable release, about fifty users took part. If you don’t have enough customer support emails, then friends and family are great candidates. For organizations, perhaps that means every internal staff member.

• Set up a TestFlight sign-up website. You can use Fastlane boarding to do this. The sign up flow for onboarding beta testers is great, which increases the likelihood that your candidate testers will actually sign up.

• Treat your gathered testers with high regard. Sure, they’re getting early access, but they’re also providing you with valuable feedback and testing coverage. Make sure they have an email they can reach out to at anytime, with questions or feedback about the beta version. Fix bugs these users find quickly, then reach out to them and ensure that the latest release has fixed their issue. Respond thoughtfully to feature suggestions these users have.

• Feedback along the lines of ‘I can’t figure out how to …’ is almost as important as any bugs users find. Usability issues can create as many bad reviews and customer support emails as bugs, so take this feedback seriously. Often this feedback will hint at real issues, but be judicious: your goal is to prevent problems from reaching production, not to solve non-issues that a few pedantic users would like you to look at.

• fiverr.com can provide reasonable value for money when it comes to testing. The quality of testing ranges from terrible to really great, take note of the quality gigs and use those again next time. $100 spent on fiverr.com for the recent Class Timetable release surfaced a couple of issues, so that seems like money well spent.

Preparing for Release

Alongside testing, there are many faucets to releasing a large scale app. Localization, screenshots, marketing and more. I won’t labor too long on these things, instead, here are a few tools that can be incredibly helpful, especially if you’re an indie developer.

• Fastlane snapshot - for taking localized screenshots across many device sizes.
• Fastlane frameit - for producing App Store ready screenshots inside iPhone frames. I run a customized version as I couldn’t get the provided options to produce exactly what I wanted.
• System Strings - translation services aren’t expensive, but utilizing this tool means saving a small amount, while ensuring that your translations are consistent with existing macOS or iOS wording.
• Linguan - helpful for organizing translations. The app seems a little buggy of late, but it is helpful nonetheless.

Phased Releases

As of recently, Apple offers ‘Phased Releases’ as an App Store release option. The concept is reasonably simple: the update will become instantly available to all users, should they visit the app on the App Store, but automatic updates will be distributed over the course of a week. The first few days, only a few percent of users will receive the automatic update, and it scales up from there. You can pause the update or release to all users at anytime. This is useful in many scenarios, for example, if you host your own services, it allows a last layer of load testing to take place.

Phased Release Distribution

Phased releases don’t mean you can slack off on testing - that’s still essential. All new users, and a small percentage of your existing user base are going to receive the update, so you certainly don’t want the first release to be buggy. Phased releases are optional and off by default, but I’d go as far as saying that even for a well tested app update, you should use still opt for a phased release. Always better safe than sorry.

For Class Timetable, I used the phased release feature in the following way. You don’t need to follow this exact model, although I found that it worked really well for me:

1. Release your production ready build. Pause the phased release a few hours later, and wait a day or so as customer support emails and crash reports start to filter in. Hopefully, you won’t have any, although for a large scale release there’s often one or two issues that only seem to show up at scale. In my case, that meant an iOS bug was causing a sporadic crash on older devices (only reproducible on a device, and only under certain conditions - not exactly easy to test for). The crash rate was about 0.2%, which was significant enough to warrant a hotfix.

2. Prepare a hotfix release for any issues that surfaced. Release the hotfix as another phased release, once it has made it through the review process. Once again, pause the release a few hours later, and wait another day for any further crash reports or emails to arrive. Repeat as needed. Ideally you won’t need to release any hotfixes at all, especially for smaller releases.

3. Once you’re certain that every issue has been ironed out, unpause the phased release. The majority of your users should now receive a well polished update, free of any issues.

On a final note: should something go pear shaped despite every precaution, there is one more option you have. Apple offer expedited reviews, which allow you to effectively skip the review queue. You need a very good reason for an expedited review, and even if you have this, there is still some form of wait. You might be able to get an update out in less than a day, but really, the best answer is to make sure it never gets to this stage.