Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
A brief "getting started" tutorial
Some parts of this short tutorial assume you have a working Munki repo configured that you can mount from your admin machine. Munki isn't strictly required to use AutoPkg - for example, it can simply build packages. If you aren't using Munki, there are additional processors you can add on to work with Jamf Pro, LANrev, Filewave, SCCM, BigFix, and more. But Munki is free and open source, so we'll use it for this demonstration.
Installation and configuration
AutoPkg is distributed as an installer package, and it's recommended to also install Git for the purpose of installing and updating community recipes. AutoPkg requires 10.6 or later.
Install the latest pkg release. This will also install and load a LaunchDaemon used by AutoPkg's packaging component. This allows AutoPkg to build packages without requiring root privileges.
Git is used by AutoPkg to add new community recipe repos, and to keep them up to date. One way to install Git is to install the Xcode Command Line tools. On Mavericks or higher, you can prompt the system to install them simply by typing the
git command, or
xcode-select --install. If you have installed Xcode, you already have Git. Git is also available via the official Git installer package.
Optional: configure for use with Munki
If we're using Munki, we must also define the
MUNKI_REPO preference as the path where the Munki repository is mounted:
defaults write com.github.autopkg MUNKI_REPO /path/to/munki_repo
Also, ensure the latest Munki tools are installed.
Install some recipes
autopkg repo-add recipes
You'll see something like:
Attempting git clone... Cloning into '/Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes'... Adding /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes to RECIPE_SEARCH_DIRS... Recipe search path is: ( ".", "~/Library/AutoPkg/Recipes", "/Library/AutoPkg/Recipes", "/Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes" )
Now look at what recipes are available:
You should see a list of available recipes:
Adium.munki Adium.pkg AdobeAIR.pkg AdobeAcrobatPro9Update.munki AdobeAcrobatProXUpdate.munki AdobeAir.munki AdobeFlashPlayer.munki AdobeFlashPlayer.pkg
..and so on.
You can get info on a specific recipe:
% autopkg info GoogleChrome.pkg Description: Downloads latest Google Chrome disk image and builds a package. Munki import recipe: False Has check phase: True Builds package: True Recipe file path: /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/GoogleChrome/GoogleChrome.pkg.recipe Input values: "DOWNLOAD_URL" = "https://dl.google.com/chrome/mac/stable/GGRO/googlechrome.dmg"; IDENTIFIER = "com.github.autopkg.pkg.googlechrome"; NAME = GoogleChrome;
Running a recipe
In our first example we'll run
autopkg with a full path to a recipe. Note the
-v verbosity flag, which will print out some useful progress information:
% autopkg run -v ~/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/GoogleChrome/GoogleChrome.pkg.recipe WARNING: /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/GoogleChrome/GoogleChrome.pkg.recipe is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding... Processing /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/GoogleChrome/GoogleChrome.pkg.recipe... URLDownloader URLDownloader: Storing new Last-Modified header: Tue, 20 Aug 2013 16:50:00 GMT URLDownloader: Storing new ETag header: "3ae73" URLDownloader: Downloaded /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/downloads/GoogleChrome.dmg EndOfCheckPhase AppDmgVersioner AppDmgVersioner: Mounted disk image /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/downloads/GoogleChrome.dmg AppDmgVersioner: BundleID: com.google.Chrome AppDmgVersioner: Version: 29.0.1547.57 PkgRootCreator PkgRootCreator: Created /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/GoogleChrome PkgRootCreator: Created /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/GoogleChrome/Applications Copier Copier: Mounted disk image /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/downloads/GoogleChrome.dmg Copier: Copied /private/tmp/dmg.DuSR60/Google Chrome.app to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/GoogleChrome/Applications/Google Chrome.app PkgCreator PkgCreator: Connecting PkgCreator: Sending packaging request PkgCreator: Disconnecting Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/receipts/GoogleChrome.pkg-receipt-20130821-105659.plist The following new items were downloaded: /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/downloads/GoogleChrome.dmg The following packages were built: /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/GoogleChrome-29.0.1547.57.pkg
Each line of status above is prefixed with a "processor", which are a list of tasks to perform in the recipe. The GoogleChrome.pkg recipe is fairly simple:
URLDownloaderdownloads a file at a given URL, and also records available header information given by the web server.
EndOfCheckPhaseis used only to mark the place in the recipe that constitutes a "check" without doing any further processing. This is used when the
--checkoption is given to
AppDmgVersionerpeeks inside the disk image and extracts the BundleID and version from the application.
PkgRootCreatorcreates a root directory in which to stage our new package's contents.
Copiercopies files from the disk image to the pkg root directory.
PkgCreatorsends a packaging request to the autopkgserver, which builds a package for us.
The verbosity level can be increased to expose more internal data.
You may also notice warnings about missing trust info. Those warnings are only warnings and won't stop the AutoPkg recipe from running. That said, it is more secure to make a recipe override and update trust info for recipes. More details at AutoPkg and recipe parent trust info
Running a recipe by name
While you can provide a full path to a recipe to run it, you can also just use the recipe name that appears in the output of
autopkg list-recipes, which is a more convenient way to specify a recipe to run. This time, let's run a Munki recipe.
% autopkg run -v GoogleChrome.munki WARNING: GoogleChrome.munki is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding... Processing GoogleChrome.munki... URLDownloader URLDownloader: Storing new Last-Modified header: Tue, 20 Aug 2013 16:50:00 GMT URLDownloader: Storing new ETag header: "3ae73" URLDownloader: Downloaded /Users/gneagle/Library/AutoPkg/Cache/com.disneyanimation.autopkg.munki.google-chrome/downloads/GoogleChrome.dmg EndOfCheckPhase MunkiImporter MunkiImporter: Copied pkginfo to /Users/Shared/munki_repo/pkgsinfo/apps/GoogleChrome-29.0.1547.57.plist MunkiImporter: Copied pkg to /Users/Shared/munki_repo/pkgs/apps/GoogleChrome-29.0.1547.57.dmg Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.disneyanimation.autopkg.munki.google-chrome/receipts/GoogleChrome-receipt-20130821-162709.plist The following new items were downloaded: /Users/gneagle/Library/AutoPkg/Cache/com.disneyanimation.autopkg.munki.google-chrome/downloads/GoogleChrome.dmg The following new items were imported: Name Version Catalogs Pkginfo Path ---- ------- -------- ------------ GoogleChrome 29.0.1547.57 production apps/GoogleChrome-29.0.1547.57.plist
This recipe does a few things differently from the GoogleChrome.pkg recipe.
Since Munki doesn't need Chrome to be repackaged, this recipe simply downloads the latest disk image and imports it into Munki. It does not, however, run
makecatalogs (which, if you manage a Munki repo, you know is a very important step!) For now, we'll do that manually:
Later we'll see how we can automate this step as well.
Repeated runs and idempotence
Now, run the same command again:
autopkg run -v GoogleChrome.munki
% autopkg run -v GoogleChrome.munki WARNING: GoogleChrome.munki is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding... Processing GoogleChrome.munki... URLDownloader URLDownloader: Item at URL is unchanged. URLDownloader: Using existing /Users/gneagle/Library/AutoPkg/Cache/com.disneyanimation.autopkg.munki.google-chrome/downloads/GoogleChrome.dmg EndOfCheckPhase MunkiImporter MunkiImporter: Item GoogleChrome.dmg already exists in the munki repo as pkgs/apps/GoogleChrome-29.0.1547.57.dmg. Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.disneyanimation.autopkg.munki.google-chrome/receipts/GoogleChrome-receipt-20130821-163151.plist Nothing downloaded, packaged or imported.
(If you don't see something similar to the above, you probably didn't run
makecatalogs as directed above!)
MunkiImporter processors contain internal logic to be able to determine whether their task has already been completed, and that they don't need to re-download or re-import the items. To verify that this is the case, try the following:
- Delete the newly-imported GoogleChrome pkginfo plist (and if you'd like, the dmg), run
makecatalogs, and re-run
autopkg run -v GoogleChrome.munki. It should verify that cached item is unchanged, but that it is missing from the Munki repo, and import it.
- Delete the cached item found in the
~/Library/AutoPkg/Cachedirectory, and re-run
autopkg run -v GoogleChrome.munki. It should re-cache the file, and as long as the version in the Munki repo maches, it should not re-import.
- Repeat 2., but give the
autopkg. This will cause it to re-cache the file and report the download, but not proceed past the
MunkiImporter processor also supports an input variable called
force_munkiimport, that forces an import if it's set (to anything). This might be useful in testing modifications to recipes. This can be set temporarily using the
-k/--key option, as in the example below. For info on this override method here.
autopkg run -k force_munkiimport=yes Recipe
Running multiple recipes
AutoPkg can run more than one recipe in a given session.
One of the recipes available in the http://github.com/autopkg/recipes.git recipe repo is "MakeCatalogs.munki". It does what you'd expect, but it's behavior is more subtle than just running
makecatalogs. By default, it checks the results of any recipes that were run in the same session and only does a
makecatalogs if something new was imported into the Munki repo. So running this recipe by itself is generally not very useful:
% autopkg run -v MakeCatalogs.munki Processing MakeCatalogs.munki... MakeCatalogsProcessor MakeCatalogsProcessor: No need to rebuild catalogs. Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.makecatalogs/receipts/MunkiMakeCatalogs-receipt-20130821-163856.plist Nothing downloaded, packaged or imported.
But if you add this recipe to the end of a run of other recipes that do import something, it does the right thing:
% autopkg run -v Firefox.munki Thunderbird.munki MakeCatalogs.munki Processing Firefox.munki... MozillaURLProvider MozillaURLProvider: Found URL http://download-origin.cdn.mozilla.net/pub/mozilla.org//firefox/releases/latest/mac/en-US/Firefox%2023.0.1.dmg URLDownloader URLDownloader: Storing new Last-Modified header: Wed, 14 Aug 2013 15:36:04 GMT URLDownloader: Storing new ETag header: "c47c4-2a4e1aa-4e3ea1d85c100" URLDownloader: Downloaded /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.firefox-rc-en_US/downloads/Firefox.dmg EndOfCheckPhase MunkiImporter MunkiImporter: Copied pkginfo to /Users/Shared/munki_repo/pkgsinfo/apps/firefox/Firefox-23.0.1.plist MunkiImporter: Copied pkg to /Users/Shared/munki_repo/pkgs/apps/firefox/Firefox-23.0.1.dmg Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.firefox-rc-en_US/receipts/Firefox-receipt-20130821-164549.plist Processing Thunderbird.munki... MozillaURLProvider MozillaURLProvider: Found URL http://download-origin.cdn.mozilla.net/pub/mozilla.org//thunderbird/releases/latest/mac/en-US/Thunderbird%2017.0.8.dmg URLDownloader URLDownloader: Storing new Last-Modified header: Thu, 01 Aug 2013 22:56:24 GMT URLDownloader: Storing new ETag header: "f68c13-23a1d47-4e2eac0571600" URLDownloader: Downloaded /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.thunderbird/downloads/Thunderbird.dmg EndOfCheckPhase MunkiImporter MunkiImporter: Copied pkginfo to /Users/Shared/munki_repo/pkgsinfo/apps/thunderbird/Thunderbird-17.0.8.plist MunkiImporter: Copied pkg to /Users/Shared/munki_repo/pkgs/apps/thunderbird/Thunderbird-17.0.8.dmg Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.thunderbird/receipts/Thunderbird-receipt-20130821-164556.plist Processing MakeCatalogs.munki... MakeCatalogsProcessor MakeCatalogsProcessor: Munki catalogs rebuilt! Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.makecatalogs/receipts/MakeCatalogs-receipt-20130821-164556.plist The following new items were downloaded: /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.firefox-rc-en_US/downloads/Firefox.dmg /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.thunderbird/downloads/Thunderbird.dmg The following new items were imported: Name Version Catalogs Pkginfo Path ---- ------- -------- ------------ Firefox 23.0.1 testing apps/firefox/Firefox-23.0.1.plist Thunderbird 17.0.8 testing apps/thunderbird/Thunderbird-17.0.8.plist
You can see it downloaded and imported Firefox, then Thunderbird, then rebuilt the Munki catalogs. If we run these recipes again, we see it skips the tasks that are not needed:
% autopkg run -v Firefox.munki Thunderbird.munki MakeCatalogs.munki Processing Firefox.munki... MozillaURLProvider MozillaURLProvider: Found URL http://download-origin.cdn.mozilla.net/pub/mozilla.org//firefox/releases/latest/mac/en-US/Firefox%2023.0.1.dmg URLDownloader URLDownloader: Item at URL is unchanged. URLDownloader: Using existing /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.firefox-rc-en_US/downloads/Firefox.dmg EndOfCheckPhase MunkiImporter MunkiImporter: Item Firefox.dmg already exists in the munki repo as pkgs/apps/firefox/Firefox-23.0.1.dmg. Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.firefox-rc-en_US/receipts/Firefox-receipt-20130821-164921.plist Processing Thunderbird.munki... MozillaURLProvider MozillaURLProvider: Found URL http://download-origin.cdn.mozilla.net/pub/mozilla.org//thunderbird/releases/latest/mac/en-US/Thunderbird%2017.0.8.dmg URLDownloader URLDownloader: Item at URL is unchanged. URLDownloader: Using existing /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.thunderbird/downloads/Thunderbird.dmg EndOfCheckPhase MunkiImporter MunkiImporter: Item Thunderbird.dmg already exists in the munki repo as pkgs/apps/thunderbird/Thunderbird-17.0.8.dmg. Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.thunderbird/receipts/Thunderbird-receipt-20130821-164923.plist Processing MakeCatalogs.munki... MakeCatalogsProcessor MakeCatalogsProcessor: No need to rebuild catalogs. Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.makecatalogs/receipts/MakeCatalogs-receipt-20130821-164923.plist Nothing downloaded, packaged or imported.
This behavior keeps autopkg from re-importing the same items over and over into the Munki repo and from running
Recipes include defaults for their input variables, but it's generally assumed that any recipe used in production will need to have some things overridden. For example, no two Munki repos follow exactly the same directory structure, so Munki recipes will likely want to at least override the destination subdirectory for the pkg/pkginfo files. Visit the Recipe Overrides page to learn more.