I’ve become very careful about pushing the submit button. I’ve learned the hard way how necessary that is. I urge you to learn from my past examples, rather than emulating them. In fact, my normal process now is to do my final testing and get everything into place and then I wait overnight before I actually submit, just in case I think of anything else.

And to test on every manner of device I can get my hands on before I do.

You see, back in 2011 there was a bug in Xcode 4.0.1 with LLVM 2.0. The optimizer would generate app packages that crashed immediately on an ARMv6 processor (like the iPhone 3G) but worked fine on an ARMv7 processor (like the iPhone 3GS).

The key word there is optimizer. You see, the way Xcode worked at the time (and likely still does) was that apps built for “Debugging” had optimization turned off, and only apps built for “Release” ran the optimizer.

And so it was that I built an app for a client, and tested it on all the different pieces of hardware I had available to me – but only with Debugging turned on (after all, if any bugs turned up during that testing, I wanted to be able to figure out what was wrong). Then I created the Release build to go to the store and I did one last sanity check with my iPhone 4, but didn’t bother to test it with my old iPhone 3G.

Apple’s App Reviewers also apparently only checked it with an iPhone 4 (or some other ARMv7 device) because they approved it. And the app got into the store. And all Hell broke loose.

Because this wasn’t a new app – it was an updated version of an existing app that I had written for that client. I had taken an app that worked fine and, in the process of fixing a couple of bugs and adding a couple of new features, had rendered it utterly unusable by a substantial fraction of the app’s existing users. Anyone with an older device who downloaded the update had the app crash immediately when they tried to start it. They couldn’t run the app, couldn’t see their data, and couldn’t get to the feedback address in the app to let us know. So they turned to the App Store review process. In droves.

Within a few minutes of seeing one of the bad reviews, I had installed the shipping version of the app onto my iPhone 3G and found the problem. A quick web search told me I wasn’t alone, but that was small consolation. I turned off the optimization and resubmitted, but it was days before the new version got through the process, and some users never got over it (although thankfully my client eventually did).

It was Apple’s bug, and arguably Apple’s fault, but the users didn’t care. And ultimately, the decision to release without testing the final version on older hardware was mine, as was the ultimate blame.