Benefits of Ephemeral iOS Builds

Modern application development practices like continuous delivery and DevOps accelerate the delivery of new features to end users. There are mature DevOps tools available in the market to deploy web apps. However, mobile DevOps has unique challenges especially on the Apple platform. In this post, we will cover the process of self-managed iOS builds and benefits of using ephemeral builds.

iOS Continuous Integration

A few years ago, only one or two developers would work on an iOS app project so there wasn’t a real need to set up continuous integration (CI). However, as the number of iOS applications is growing, the number of developers working on any one iOS app is growing too. When multiple developers start working on the same app, practices like continuous integration become essential to avoid integration issues. Continuous integration is the process of analyzing, building, and testing an iOS app after a developer makes code changes. The process of setting up CI for iOS apps was never easy as iOS apps can only be built and deployed from macOS servers. The basic requirements for setting up iOS continuous integration are:

• Establishing a macOS server; either on-premise or in the cloud from services like MacStadium
• ‍Selecting the right CI server e.g. Jenkins, TeamCity or similar
• Installing the required software on the macOS server to build, test and deploy iOS apps

On top of these basic requirements, the process also involves complicated steps like code signing and dealing with Apple distribution certificates. It can be an error-prone and time-consuming process.

In most cases, iOS teams start with macOS servers on MacBook Pros or Mac minis and setup those machines manually for CI. It’s very common to have a pile of Mac minis under a desk acting as CI servers for iOS apps. This is a great way to get started with CI for iOS apps; however, problems arise when the team increases its build frequency and demand. That’s when in-house maintenance of Mac CI servers can become a pain.

Handcrafted Build Servers

To setup a CI server, you must also install all of the necessary related software on the build server. In the case of iOS CI, this is typically Xcode and other custom software like CocoaPods, Carthage, Fastlane, etc. The common mistake that iOS teams make is to install software manually and update it whenever needed. In addition, you must configure the CI server (Jenkins, TeamCity or similar) to run your build jobs from the web user interface. The manual configuration of a CI server becomes unmanageable very soon. For example, whenever Apple releases a new version of Xcode, an engineer has to manually upgrade all of the build machines, which typically isn’t feasible and efficient.  

Long-Lived Builds

Once the manual setup of the CI server is finished, engineers usually get back to work on writing code and building the app. CI servers setup manually often get ignored as long as builds are working fine. This causes long-lived builds on CI servers. The long-lived environment is a static environment which never gets destroyed and continuously accepts new builds. A long-lived environment for iOS builds can last up to weeks or months or until it shows symptoms of illness. Typically, long-lived builds are only treated when they are sick. 

Every build performed on a long-lived CI server generates artifacts like test results, app binaries, code coverage reports, etc. Some common iOS build artifacts include:

• App binaries (.app or .ipa format)
• Code coverage and test reports
• Screenshots of the test steps
• dSYMS files
• Derived data for each iOS app

Some build artifacts can be heavy in size (for example, .ipa files can range from 30MB to 100MB), and screenshots of the XCUITests can be big as well. These artifacts accumulate on the CI server which results in high CPU usage and slow builds. One of the major side-effects of the slow builds is developers waiting a long time for the build to finish which decreases their productivity. There can be many other pain points when running a long-lived build machine:  

• As time goes by, the server becomes less efficient due to disk space and memory unitization of the new builds.
• Long-lived builds can become the cause of test flakiness due to shared environments between the builds.
• Manual build configuration steps are hard to repeat and often need to be documented.
• Frequent changes in the CI build steps causes random build failures and requires unnecessary investigation if the changes are not communicated properly.
• There can be a lack of confidence in the build, and bugs found on the environment aren’t reproducible. Long-lived environments are often the cause of a false positive or false negative.

Common solutions for long-lived build problems are build automation, configuration management, and infrastructure as code. Build automation and infrastructure as code avoids manual changes in the build process. Configuration management ensures the integrity of the environment across multiple stages, i.e. from a developer's machine to production. You may also wish to automatically destroy and rebuild your CI infrastructure as needed. This is where the ephemeral builds come into the picture.

Ephemeral Builds

An ephemeral build is a fancy name used for short-lived, temporary, dynamic, or on-demand builds. In an ephemeral build process, you create a fresh environment for each build and destroy it when all the build steps are finished. By using ephemeral builds, you can avoid a lot of build issues if the environments are configured correctly.  

In the case of iOS continuous integration, ephemeral builds are perfect because there is no need to keep the build artifact unless there is a genuine requirement. A typical iOS CI build process will have the following steps: 

• Lint iOS source code using SwiftLint or SwiftFormat
• ‍Build an app using xcodebuild
• ‍Test iOS apps with XCTest or other test frameworks
• Create an archive and IPA file and upload to TestFlight or another third-party app hosting service
• Notify build status for relevant users with email or Slack

Once the build has been performed, there is no need to keep the environment as long as all the required artifacts are uploaded to the relevant services. You can upload the test reports or other build artifacts to any server (like AWS S3bucket) if needed.

Benefits of Ephemeral Builds

There are some great benefits of using ephemeral builds for CI, including:

• No maintenance overhead
• Clean builds and consistent environments at all stages
• As environments are provisioned in the code, there won’t be any inconsistency in the installation of the software packages
• Ability to build and destroy the environment per project requirements without manual invention
• Artifact consistency as there is no sharing or using old artifacts

While an ephemeral build gives you more flexibility and control over your build process, there are some factors that you should keep in mind:

• The success of ephemeral builds depends on strong automated build provisioning scripts.  
• Since builds start with a fresh machine that is then destroyed at the end of each build, you need to write additional scripts to upload your test results or app binaries to other dedicated servers.
• The dependencies of the iOS apps need to be cached to speed up the builds.

Ephemeral builds can provide a great return on investment on your CI pipeline.

Prepare for Ephemeral iOS Builds

At this point, you might be convinced that ephemeral builds are a way to go for your self-managed CI servers. Now, let’s discuss the steps needed to transition your iOS apps from long-lived builds to ephemeral builds.

Requirements 

1. Setup the macOS servers.

You can buy the macOS servers yourself and manage them internally, but the initial cost of the hardware will be high. And as Apple releases new Mac hardware, you may want to replace older machines with the newer versions, resulting in additional costs. However, the maintenance cost is much higher than the cost of the hardware itself, taking valuable engineer time to update and maintain the servers. MacStadium provides a wide variety of macOS servers in the cloud that are optimized for ephemeral build environments.

2. Choose your CI server.

Once you have sorted the macOS servers, the next big step is to select the right CI server for your project needs. The process of setting up a CI server takes time and money, so you need to be very careful about selecting the right CI server for your iOS app. Here is a list of CI services with some recommendations; refer to it as a starting point. Some of the commonly used self-managed CI servers are Jenkins, TeamCity, and BuildKite.

3. Select a configuration management tool.

There are various configuration management tools like Ansible, Chef, and Puppet available in the market which can ease the process of automating your iOS build environments. Ansible is easy to use tool that can work for automating iOS infrastructure. However, if you are familiar with Ruby, you might prefer Chef or Puppet.

Steps for Ephemeral Builds

Once you select your tooling, you need to implement the steps needed for a short-lived, dynamic iOS ecosystem, including:

• Make a list of all the iOS project dependencies that need to be installed on each fresh macOS server
• Use configuration tools like Ansible to automate the installation of all software per build
• Automate your iOS build process using native iOS developer tools or third-party tools like Fastlane
• ‍Get a dedicated server to upload build artifacts, test reports, or app binaries from the ephemeral build if needed
• Test your scripts multiple times before using on live projects

At this stage, you should be confident that your CI infrastructure is completely automated from the code and won’t break without proper reason. Your builds should be fast and require less maintenance. All the infrastructure is coded, so there is no possibility of manual changes from the web user interface. In summary, you will have smooth CI pipelines using ephemeral builds.  

Conclusion

Continuous integration for iOS apps can be very frustrating if not managed correctly. However, by using techniques like ephemeral builds on dedicated macOS servers hosted in the cloud and choosing the right tools, we can achieve better results in the process of iOS continuous integration.