Packaging a daemon for macOS

Vincent Bernat

There are three main ways to distribute a command-line daemon for macOS:

  1. Distributing source code and instructions on how to compile it.
  2. Using a third-party package manager, like Homebrew.
  3. Providing an installer package.

Homebrew#

Homebrew is a popular package management system. It works like the BSDs’ ports collections by downloading, compiling and installing the requested software, while also installing any required dependencies automatically. Creating a new package is quite easy and there are a lot of examples available.

However, there are some limitations:

  • You don’t really build a package but execute a recipe to locally install the software.
  • You need to install development tools, either a whole Xcode installation1 or the command line version.
  • If you need to execute some steps as root, you will have to explain them to the user for them to execute by hand. This includes the creation of a system user or the installation of a daemon through launchd.

If you can’t live with these limitations, you may need to build an installer package.

Building an installer package#

macOS comes with a graphical and a command-line installer. The graphical one is run by opening a package from the Finder. Then, the user experiences some familiar wizard allowing them to install the software in a matter of a few seconds.

macOS graphical installer
Example of macOS graphical installer

The documentation around how to build such a package is somewhat clumsy. You may find outdated information or pieces of information that only apply to projects using Xcode. I will try to provide here accurate bits in the following context:

  1. You are packaging a pure command-line tool.
  2. You are using Autoconf and Automake as a build system.
  3. You want to support several architectures.
  4. You want to support older versions of macOS.

Creating a package#

Building such a package was previously done with some graphical tool called PackageMaker. This tool is not available any more and developers are asked to switch to pkgbuild and productbuild. There is a very neat Stackoverflow article on how these tools work.

A package is built in two steps:

  1. Build component packages.
  2. Combine them into a product archive.

A component package contains a set of files and a set of scripts to execute at various steps of the installation. You can have several component packages, for example a package for the daemon and a package for the client. They are built with pkgbuild.

A product archive contains the previously created component packages as well as a file describing various options of the installer (required and optional components, license, welcome text, …).

To create a component package, we need to install the needed files in some directory:

$ ./configure --prefix=/usr/local --sysconfdir=/private/etc
$ make
$ make install DESTDIR=$PWD/osx-pkg

Update (2022-08)

Since macOS 10.11, it is not possible to install in /usr. You need to switch to /usr/local. This document has been updated to reflect this change.

We will now put the content of osx-pkg into a component package with pkgbuild:

$ mkdir pkg1
$ pkgbuild --root osx-pkg \
>    --identifier org.someid.daemon \
>    --version 0.47 \
>    --ownership recommended \
>    pkg1/output.pkg
pkgbuild: Inferring bundle components from contents of osx-pkg
pkgbuild: Wrote package to output.pkg

You need to be careful with the identifier. It needs to be unique and identify your software as well as the specific component. Then, you need to create an XML file describing the installer. Let’s call it distribution.xml:

<?xml version="1.0" encoding="utf-8" standalone="no"?>
<installer-gui-script minSpecVersion="1">
    <title>Some daemon</title>
    <organization>org.someid</organization>
    <domains enable_localSystem="true"/>
    <options customize="never" require-scripts="true" rootVolumeOnly="true" />
    <!-- Define documents displayed at various steps -->
    <welcome    file="welcome.html"    mime-type="text/html" />
    <license    file="license.html"    mime-type="text/html" />
    <conclusion file="conclusion.html" mime-type="text/html" />
    <!-- List all component packages -->
    <pkg-ref id="org.someid.daemon"
             version="0"
             auth="root">output.pkg</pkg-ref>
    <!-- List them again here. They can now be organized
         as a hierarchy if you want. -->
    <choices-outline>
        <line choice="org.someid.daemon"/>
    </choices-outline>
    <!-- Define each choice above -->
    <choice
        id="org.someid.daemon"
        visible="false"
        title="some daemon"
        description="The daemon"
        start_selected="true">
      <pkg-ref id="org.someid.daemon"/>
    </choice>
</installer-gui-script>

Fortunately, this file is documented on Apple Developer Library. Since we only describe one package, we pass customize="never" as an option to skip the choice part. You can however remove this attribute when you have several component packages. The attribute rootVolumeOnly="true" explains that this daemon can only be installed system-wide. It is marked as deprecated but still works. Its replacement (domains tag) displays an unusual and buggy pane which Apple doesn’t use in any of its packages.

You need to put the HTML2 documents into a resources directory. It is also possible to choose the background by specifying a <background/> tag.

The following command will generate the product archive:

$ productbuild --distribution distribution.xml \
>              --resources resources \
>              --package-path pkg1 \
>              --version 0.47 \
>              ../final.pkg
productbuild: Wrote product to ../final.pkg

Scripts#

The installer can execute some scripts during installation. For example, suppose we would like to register our daemon with launchd for it to be run at system start. You first need to write some org.someid.plist file and ensure it will get installed in /Library/LaunchDaemons:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>
  <string>org.someid</string>
  <key>ProgramArguments</key>
  <array>
    <string>/usr/local/sbin/mydaemon</string>
    <string>-d</string>
  </array>
  <key>RunAtLoad</key><true/>
  <key>KeepAlive</key><true/>
</dict>
</plist>

Then create a scripts/postinstall file with the following content:

#!/bin/sh

set -e
/bin/launchctl load "/Library/LaunchDaemons/org.someid.plist"

You also need a scripts/preinstall file which will allow your program to be smoothly upgraded:

#!/bin/bash

set -e
if /bin/launchctl list "org.someid" &> /dev/null; then
    /bin/launchctl unload "/Library/LaunchDaemons/org.someid.plist"
fi

Ensure that these scripts are executable and add --scripts scripts to pkgbuild invocation.

If you need to add a system user, use dscl.

Dependencies#

There is no dependency management built into macOS installer. Therefore, you should ensure that everything is present in the package or in the base system. For example, lldpd relies on libevent, an event notification library. This library is not provided with macOS. When it is not found, the build system will use an embedded copy. However, if you have installed libevent with Homebrew, you will get binaries linked to this local libevent installation. Your package won’t work on another host.

The output of otool -L can detect unwanted dependencies:

$ otool -L build/usr/local/sbin/lldpd
build/usr/local/sbin/lldpd:
        /usr/lib/libresolv.9.dylib (compatibility version 1.0.0, current version 1.0.0)
        /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
        /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 744.19.0)
        /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 945.18.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 169.3.0)

Compilation for older versions of macOS#

The package as built above may only work on the same version of macOS that you use to compile. If you want a package that will work on Mac OS X 10.6, you will need to download the appropriate SDK.3

Then, you need to tell the compiler the target version of macOS and the SDK location. You can do this by setting CFLAGS and LDFLAGS (and CXXFLAGS if you use C++):

$ SDK=/Developer/SDKs/MacOSX10.6.sdk
$ ./configure --prefix=/usr/local --sysconfdir=/private/etc \
>         CFLAGS="-mmacosx-version-min=10.6 -isysroot $SDK" \
>         LDFLAGS="-mmacosx-version-min=10.6 -isysroot $SDK"

The -mmacosx-version-min flag will be used by various macros in the headers to mark some functions as available or not, depending on the version of macOS you target. It will also be used by the linker.

The -isysroot will tell the location of the SDK. Headers and libraries will be looked up in this location first.

Update (2022-08)

Starting with SDK 10.9, you don’t have to specify -isysroot anymore.

Universal binaries#

Mac OS X 10.6 was available for both IA-32 and x86-64 architectures. If you want to support both installations with a single package, you need to build universal binaries. The Mach object file format, used by macOS, allows several versions of the executable in the same file. The operating system will select the most appropriate one.

An easy way to generate such files is to pass -arch i386 -arch x86_64 to the compiler:

$ ./configure --prefix=/usr/local --sysconfdir=/private/etc \
>     CC="gcc -arch i386 -arch x86_64" \
>     CPP="gcc -E"

However, this is a dangerous option. Suppose that your ./configure script tries to determine some architecture-dependent parameter, like the size of an integer (with AC_CHECK_SIZEOF); it will compute it for the host architecture. The binary generated for the other architecture will therefore use a wrong value and may crash at some point.

The correct way to generate a universal binary is to execute two separate compilations and to build the universal binary with lipo:

$ for arch in i386 x86_64; do
>   mkdir $arch ; cd $arch
>   ../configure --prefix=/usr/local --sysconfdir=/private/etc \
>       CC="gcc -arch $ARCH" \
>       CPP="gcc -E"
>   make
>   make install DESTDIR=$PWD/../target-$ARCH
>   cd ..
> done
[…]
$ lipo -create -output daemon i386/usr/local/sbin/daemon x86_64/usr/local/sbin/daemon

Since lipo only works on file, I have written a Python script applying lipo recursively to several directories.

Update (2022-08)

Nowadays, you should replace i386 by arm64. Apple recommends using -target to combine both the architecture and the SDK to target—for example, -target arm64-apple-macos11. However, the -arch flag is still working and you can combine it with -mmacosx-version-min if needed. Note that it requires you to compile on a platform supporting running both architectures (currently, an Apple M1 or M2).

Putting everything together#

Now, we need to automate a bit. You could provide some nifty script. I propose an appropriate Makefile.am instead. This way, we can use the output variables, like @VERSION@, from ./configure to generate some of the files, like distribution.xml. You need to add that in your configure.ac:

AC_CONFIG_FILES([osx/Makefile osx/distribution.xml osx/im.bernat.lldpd.plist])
AC_CONFIG_FILES([osx/scripts/preinstall],  [chmod +x osx/scripts/preinstall])
AC_CONFIG_FILES([osx/scripts/postinstall], [chmod +x osx/scripts/postinstall])
AC_SUBST([CONFIGURE_ARGS], [$ac_configure_args])

Let’s have a look at osx/Makefile.am. First, we define some variables:

PKG_NAME=@PACKAGE@-@VERSION@.pkg
PKG_TITLE=@PACKAGE@ @VERSION@
PKG_DIR=@PACKAGE@-@VERSION@
ARCHS=@host_cpu@

If we want to build for several architectures, we will type make ARCHS="x86_64 i386". Otherwise it defaults to the current host’s architecture.

We use install-data-local target to install files specific to macOS:

install-data-local:
    install -m 0755 -d $(DESTDIR)/Library/LaunchDaemons
    install -m 0644 im.bernat.@PACKAGE@.plist $(DESTDIR)/Library/LaunchDaemons
uninstall-local:
    rm -f $(DESTDIR)/Library/LaunchDaemons/im.bernat.@PACKAGE@.plist

The main target is the product archive, built with productbuild:

../$(PKG_NAME): pkg.1/$(PKG_NAME) distribution.xml resources
    $(PRODUCTBUILD) \
        --distribution distribution.xml \
        --resources resources \
        --package-path pkg.1 \
        --version @VERSION@ \
        $@

Its main dependency is the component package:

pkg.1/$(PKG_NAME): $(PKG_DIR) scripts
    [ -d pkg.1 ] || mkdir pkg.1
    $(PKGBUILD) \
        --root $(PKG_DIR) \
        --identifier im.bernat.@PACKAGE@.daemon \
        --version @VERSION@ \
        --ownership recommended \
        --scripts scripts \
        $@

Now, we need to build $(PKG_DIR):

$(PKG_DIR): stamp-$(PKG_DIR)
stamp-$(PKG_DIR): $(ARCHS:%=%/$(PKG_DIR))
    $(srcdir)/lipo $(PKG_DIR) $^
    touch $@

The $(ARCHS:%=%/$(PKG_DIR)) will expand to x86_64/$(PKG_DIR) i386/$(PKG_DIR). This is a substitution reference.4

Before applying our lipo script, we need to be able to build the architecture-dependent package directories:

pkg_curarch = $(@:stamp-%=%)
$(ARCHS:%=%/$(PKG_DIR)): %/$(PKG_DIR): stamp-%
$(ARCHS:%=stamp-%): stamp-%: im.bernat.lldpd.plist
    [ -d $(pkg_curarch) ] || mkdir -p $(pkg_curarch)
    (cd $(pkg_curarch) && \
        $(abs_top_srcdir)/configure @CONFIGURE_ARGS@ \
            CC="@CC@ -arch $(pkg_curarch)" \
            CPP="@CPP@")
    (cd $(pkg_curarch) && \
        $(MAKE) install DESTDIR=$(abs_builddir)/$(pkg_curarch)/$(PKG_DIR))
    touch $@

This may seem a bit obscure, but this is really like what we have described previously. Note that we use @CONFIGURE_ARGS@ which is a variable we have defined in configure.ac.

Here is how a user would create its own macOS package:

$ SDK=/Developer/SDKs/MacOSX10.6.sdk
$ mkdir build && cd build
$ ../configure --prefix=/usr/local --sysconfdir=/private/etc --with-embedded-libevent \
>   CFLAGS="-mmacosx-version-min=10.6 -isysroot $SDK" \
>   LDFLAGS="-mmacosx-version-min=10.6 -isysroot $SDK"
[…]
$ make -C osx pkg ARCHS="i386 x86_64"
[…]
productbuild: Wrote product to ../lldpd-0.7.5-21-g5b90c4f-dirty.pkg
The package has been built in ../lldpd-0.7.5-21-g5b90c4f-dirty.pkg.

  1. Even if Xcode is free, you will need to register to Apple and show a valid credit card number. That’s quite an interesting requirement. ↩︎

  2. It is possible to use other formats (like RTF or just plain text) for documents but HTML seems the best choice if you want some formatting without fiddling with RTF. ↩︎

  3. Unfortunately, Apple makes it difficult to find them. The easiest way to get the SDK for Mac OS X 10.6 is to download Xcode 4.3.3, mount the image and copy the SDK to /Developer/SDKs↩︎

  4. This is equivalent to the use of patsubst function. However, this function is specific to GNU make. The substitution reference pattern was accepted recently in POSIX and is coined as pattern macro expansion↩︎