Continuous integration for small iOS/macOS teams
There are quite a few case studies out there highlighting how industry giants like Shopify and Pandora manage their Mac continuous integration (CI) fleets. However, it’s much more difficult to find good advice for smaller teams that can’t dedicate an entire team to CI management or aren’t willing to invest five-figure sums (or more) per year for their setup.
In this new series, we’re sharing our cost-efficient approach to macOS continuous integration. Make sure to check back so you don’t miss out on all the posts!
History and early experiments
In the early years of PSPDFKit, we experimented with Travis CI, BuddyBuild, and CircleCI. Unfortunately, all of these solutions ended up being too slow and inflexible for us.
Our codebase is large (1.5 MLOC), and compiling can take about 30 minutes. Most of the available virtualized hardware is simply not powerful enough to get results back within a reasonable amount of time. We consider 30 minutes the absolute maximum acceptable amount of time, but of course the faster, the better.
GitHub Actions, for example, specifies 2-core CPU and 7 GB of RAM, and 14 GB disk space(opens in a new tab). With such a configuration, a VM wouldn’t finish compiling in the usual 60-minute limit. We hit similar timeouts with BuddyBuild and, to a lesser degree, CircleCI.
Our approach
This article is the beginning of a series about our approach to CI.
We handle a fleet of (bare-metal) Mac minis. Why no virtualization? Read more in Managing macOS hardware: Virtualization or bare metal?
Our machines are provisioned automatically to ensure they’re all running the exact same setup. Learn more about automating macOS with Chef.
To orchestrate our builds, we use Buildkite. Learn about our evaluation approach in Continuous integration: From Jenkins to Buildkite.
We defined all our pipelines in Terraform configuration files. Read more about how we're using Terraform to manage our CI pipelines.
Testing knowledge
We write about our approach to testing and CI occasionally, and while technology constantly evolves, many of the choices we made still work well for us today.
In the MacStadium panel about CI best practices for iOS and macOS app development(opens in a new tab), I discussed our approach to testing, virtualization, and hardware management alongside panelists from Aspyr Media and Pandora.
There’s nothing better than having someone automatically yell at you if your code might not be optimal. Learn how we’re using clang-tidy for linting, and how it can be integrated into your CI workflow.
Just as style guides and linters enforce code formatting rules, having similar rules around other forms of writing can help ensure quality and define a more consistent tone across communications. Learn how to add spellchecking to your tests.
Running UI tests on iOS with ludicrous speed is an approach that uses KIF (Keep It Functional), but many of the tricks here are useful no matter which UI testing framework you use.
To ensure we’re actually testing all the crucial code segments, we need a way to visualize what has been touched by our test code and what has not. This is where continuous iOS code coverage with Jenkins and Slather comes into play.
To increase code sharing, we decided to move all our code to a monorepo. Everything was great, except the time it took to run tests on a PR. Here’s how selective pull request testing can help.
We convert Xcode test runs to JUnit, the fast way, using a fastlane plugin — as opposed to xcpretty — for more reliable testing output.
Talk: Continuous integration for the rest of us
Our Founder Peter Steinberger(opens in a new tab) gave a talk(opens in a new tab) about this topic at iOS Conf Singapore 2021(opens in a new tab), the largest iOS developer conference in Southeast Asia.
Continued reading
The next posts in this series will dive deeper into our approach to CI, including:
- How we manage hardware
- How we automate CI
- How we orchestrate our builds
- How we manage CI pipelines with Terraform
Be sure to check them out!
