Faster Compilation with Ccache 4.0

At PSPDFKit, we use Ccache — a wrapper that sits between Xcode and Clang — to accelerate our build times. We’ve been using it successfully for more than five years now. In this post, we explain how Ccache can be used in modern projects that mix Objective-C and Swift.

Ccache and Modules

As Swift moved to a stable ABI this year, we started mixing Objective-C and Swift in our SDK. In order for that to work, we had to enable Modules, which was a setting Ccache didn’t support.

This changed with the release of ccache 4.0(opens in a new tab), and -fmodules is now officially supported! This will speed up compilation times for a warm cache by 3–5 minutes, which is significant.

Before you get too excited: Ccache helps for C, C++, and Objective-C code. It doesn’t know about Swift (yet).

File Cloning on APFS

The latest release of Ccache learned quite a few new tricks, so our recommended configuration (~/.ccache/ccache.conf) is now the following:

# Modules requires both direct and depend mode.
run_second_cpp = true
depend_mode = true
direct_mode = true

# Faster file copying (cloning) on APFS.
file_clone = true
inode_cache = true

# Accept more file changes before recompiling.
sloppiness = modules, include_file_mtime, include_file_ctime, time_macros, pch_defines, file_stat_matches, clang_index_store, system_headers

max_size = 100.0G

All settings are thoroughly explained in the Ccache manual(opens in a new tab). The important part is to add modules to sloppiness and enable both direct and depend mode to support -fmodules.

The most exciting change is that 4.0 learned to use file cloning (AKA “reflinks”) on Btrfs, XFS, and APFS to copy data to and from the cache efficiently.

Earlier versions could use hard links; however, those were problematic as they could be overridden. If you’re curious, reflinks are implemented via a new header and C function clonefile(opens in a new tab) on macOS.

Multiple Source Files

After we enabled everything, we encountered a new problem: Caching failed because Ccache detected multiple input files.

This line is relevant (enable logging to get this output, via log_file = /tmp/ccache.log in ccache.conf):

[2020-10-22T13:28:08.138633 43404] Multiple input files: /Users/steipete/Builds/PSPDFKit-apwojecgwxmuoieabhwzqzmecixq/Build/Intermediates.noindex/PSPDFKit.build/Debug-iphonesimulator/PSPDFKit.framework.build/all-product-headers.yaml and /Users/steipete/Projects/PSPDFKit/iOS/PSPDFKit/Glyphs/PSPDFTextBlock.m

For some reason all-product-headers.yaml was detected as an input file! It’s referenced via the -ivfsoverlay parameter, which can “overlay the virtual filesystem described by file over the real file system(opens in a new tab).”

I’ve been reading the file and it doesn’t seem as it would redefine any paths. Instead — at least for our project — it simply lists the used headers.

Unfortunately, there’s no way to teach Ccache to ignore a specific setting; this has to be changed in the source code. To fix this, I forked Ccache and opened a pull request where this flag is simply ignored(opens in a new tab).

Custom Homebrew Formula

The build instructions for Ccache(opens in a new tab) are easy enough, so for a first test, I simply replaced the binary that was installed via Homebrew. I verified that the files are now correctly cached, and I moved on to create a custom formula so that we can use the fork in our CI infrastructure.

Adding a custom “tap” to Homebrew is easy — it’s simply a GitHub repository. One file is one formula. We already have one at PSPDFKit(opens in a new tab) so I simply added the customized formula there.

• First, tag your custom version. I used v4.0.pspdfkit as the tag.
• Next, find the existing formula via https://formulae.brew.sh/ and copy it to the new tap.
• Modify the link to point to your custom archive. GitHub offers zipped source download links when you click on a tag, so it’s easy to replace the existing URL.
• Homebrew also uses SHA-256 to verify downloads, so calculate the new hash from the terminal via openssl dgst -sha256.
• If there are any bottles, remove them(opens in a new tab), as they’re prebuilt binary variants that won’t contain our modifications.

That’s it! Now you can install the new formula:

brew tap pspdfkit-labs/tap
brew reinstall pspdfkit-labs/tap/ccache

Conclusion

The new version of Ccache is significantly faster thanks to file cloning on APFS, and with our customization, it can now also be used again for Objective-C code, which will greatly reduce our compilation times.