Getting Started

Anthony Reimer edited this page Mar 19, 2018 · 28 revisions

A brief "getting started" tutorial

Some parts of this short tutorial assume you have a working Munki repo configured that you can mount from your admin machine. Munki isn't strictly required to use AutoPkg - for example, it can simply build packages. If you aren't using Munki, there are additional processors you can add on to work with Jamf Pro, LANrev, Filewave, SCCM, BigFix, and more. But Munki is free and open source, so we'll use it for this demonstration.

Installation and configuration

AutoPkg is distributed as an installer package, and it's recommended to also install Git for the purpose of installing and updating community recipes. AutoPkg requires 10.6 or later.

Install AutoPkg

Install the latest pkg release. This will also install and load a LaunchDaemon used by AutoPkg's packaging component. This allows AutoPkg to build packages without requiring root privileges.

Install Git

Git is used by AutoPkg to add new community recipe repos, and to keep them up to date. One way to install Git is to install the Xcode Command Line tools. On Mavericks or higher, you can prompt the system to install them simply by typing the git command, or xcode-select --install. If you have installed Xcode, you already have Git. Git is also available via the official Git installer package.

Optional: configure for use with Munki

If we're using Munki, we must also define the MUNKI_REPO preference as the path where the Munki repository is mounted:

defaults write com.github.autopkg MUNKI_REPO /path/to/munki_repo

Also, ensure the latest Munki tools are installed.

Install some recipes

autopkg repo-add recipes

You'll see something like:

Attempting git clone...
Cloning into '/Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes'...

Adding /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes to RECIPE_SEARCH_DIRS...
Recipe search path is: (
    ".",
    "~/Library/AutoPkg/Recipes",
    "/Library/AutoPkg/Recipes",
    "/Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes"
)

Now look at what recipes are available:

autopkg list-recipes

You should see a list of available recipes:

Adium.munki
Adium.pkg
AdobeAIR.pkg
AdobeAcrobatPro9Update.munki
AdobeAcrobatProXUpdate.munki
AdobeAir.munki
AdobeFlashPlayer.munki
AdobeFlashPlayer.pkg

..and so on.

You can get info on a specific recipe:

% autopkg info GoogleChrome.pkg
Description:         Downloads latest Google Chrome disk image and builds a package.
Munki import recipe: False
Has check phase:     True
Builds package:      True
Recipe file path:    /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/GoogleChrome/GoogleChrome.pkg.recipe
Input values: 
 
    "DOWNLOAD_URL" = "https://dl.google.com/chrome/mac/stable/GGRO/googlechrome.dmg";
    IDENTIFIER = "com.github.autopkg.pkg.googlechrome";
    NAME = GoogleChrome;

Running a recipe

In our first example we'll run autopkg with a full path to a recipe. Note the -v verbosity flag, which will print out some useful progress information:

% autopkg run -v ~/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/GoogleChrome/GoogleChrome.pkg.recipe
WARNING: /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/GoogleChrome/GoogleChrome.pkg.recipe is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding...
Processing /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/GoogleChrome/GoogleChrome.pkg.recipe...
URLDownloader
URLDownloader: Storing new Last-Modified header: Tue, 20 Aug 2013 16:50:00 GMT
URLDownloader: Storing new ETag header: "3ae73"
URLDownloader: Downloaded /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/downloads/GoogleChrome.dmg
EndOfCheckPhase
AppDmgVersioner
AppDmgVersioner: Mounted disk image /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/downloads/GoogleChrome.dmg
AppDmgVersioner: BundleID: com.google.Chrome
AppDmgVersioner: Version: 29.0.1547.57
PkgRootCreator
PkgRootCreator: Created /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/GoogleChrome
PkgRootCreator: Created /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/GoogleChrome/Applications
Copier
Copier: Mounted disk image /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/downloads/GoogleChrome.dmg
Copier: Copied /private/tmp/dmg.DuSR60/Google Chrome.app to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/GoogleChrome/Applications/Google Chrome.app
PkgCreator
PkgCreator: Connecting
PkgCreator: Sending packaging request
PkgCreator: Disconnecting
Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/receipts/GoogleChrome.pkg-receipt-20130821-105659.plist

The following new items were downloaded:
    /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/downloads/GoogleChrome.dmg

The following packages were built:
    /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.pkg.googlechrome/GoogleChrome-29.0.1547.57.pkg

Each line of status above is prefixed with a "processor", which are a list of tasks to perform in the recipe. The GoogleChrome.pkg recipe is fairly simple:

  1. URLDownloader downloads a file at a given URL, and also records available header information given by the web server.
  2. EndOfCheckPhase is used only to mark the place in the recipe that constitutes a "check" without doing any further processing. This is used when the --check option is given to autopkg.
  3. AppDmgVersioner peeks inside the disk image and extracts the BundleID and version from the application.
  4. PkgRootCreator creates a root directory in which to stage our new package's contents.
  5. Copier copies files from the disk image to the pkg root directory.
  6. PkgCreator sends a packaging request to the autopkgserver, which builds a package for us.

The verbosity level can be increased to expose more internal data.

You may also notice warnings about missing trust info. Those warnings are only warnings and won't stop the AutoPkg recipe from running. That said, it is more secure to make a recipe override and update trust info for recipes. More details at AutoPkg and recipe parent trust info

Running a recipe by name

While you can provide a full path to a recipe to run it, you can also just use the recipe name that appears in the output of autopkg list-recipes, which is a more convenient way to specify a recipe to run. This time, let's run a Munki recipe.

% autopkg run -v GoogleChrome.munki
WARNING: GoogleChrome.munki is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding...
Processing GoogleChrome.munki...
URLDownloader
URLDownloader: Storing new Last-Modified header: Tue, 20 Aug 2013 16:50:00 GMT
URLDownloader: Storing new ETag header: "3ae73"
URLDownloader: Downloaded /Users/gneagle/Library/AutoPkg/Cache/com.disneyanimation.autopkg.munki.google-chrome/downloads/GoogleChrome.dmg
EndOfCheckPhase
MunkiImporter
MunkiImporter: Copied pkginfo to /Users/Shared/munki_repo/pkgsinfo/apps/GoogleChrome-29.0.1547.57.plist
MunkiImporter: Copied pkg to /Users/Shared/munki_repo/pkgs/apps/GoogleChrome-29.0.1547.57.dmg
Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.disneyanimation.autopkg.munki.google-chrome/receipts/GoogleChrome-receipt-20130821-162709.plist

The following new items were downloaded:
    /Users/gneagle/Library/AutoPkg/Cache/com.disneyanimation.autopkg.munki.google-chrome/downloads/GoogleChrome.dmg

The following new items were imported:
    Name                     Version          Catalogs                         Pkginfo Path
    ----                     -------          --------                         ------------
    GoogleChrome             29.0.1547.57     production                       apps/GoogleChrome-29.0.1547.57.plist

This recipe does a few things differently from the GoogleChrome.pkg recipe. Since Munki doesn't need Chrome to be repackaged, this recipe simply downloads the latest disk image and imports it into Munki. It does not, however, run makecatalogs (which, if you manage a Munki repo, you know is a very important step!) For now, we'll do that manually:

/usr/local/munki/makecatalogs

Later we'll see how we can automate this step as well.

Repeated runs and idempotence

Now, run the same command again: autopkg run -v GoogleChrome.munki

% autopkg run -v GoogleChrome.munki
WARNING: GoogleChrome.munki is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding...
Processing GoogleChrome.munki...
URLDownloader
URLDownloader: Item at URL is unchanged.
URLDownloader: Using existing /Users/gneagle/Library/AutoPkg/Cache/com.disneyanimation.autopkg.munki.google-chrome/downloads/GoogleChrome.dmg
EndOfCheckPhase
MunkiImporter
MunkiImporter: Item GoogleChrome.dmg already exists in the munki repo as pkgs/apps/GoogleChrome-29.0.1547.57.dmg.
Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.disneyanimation.autopkg.munki.google-chrome/receipts/GoogleChrome-receipt-20130821-163151.plist

Nothing downloaded, packaged or imported.

(If you don't see something similar to the above, you probably didn't run makecatalogs as directed above!)

Both the URLDownloader and MunkiImporter processors contain internal logic to be able to determine whether their task has already been completed, and that they don't need to re-download or re-import the items. To verify that this is the case, try the following:

  1. Delete the newly-imported GoogleChrome pkginfo plist (and if you'd like, the dmg), run makecatalogs, and re-run autopkg run -v GoogleChrome.munki. It should verify that cached item is unchanged, but that it is missing from the Munki repo, and import it.
  2. Delete the cached item found in the ~/Library/AutoPkg/Cache directory, and re-run autopkg run -v GoogleChrome.munki. It should re-cache the file, and as long as the version in the Munki repo maches, it should not re-import.
  3. Repeat 2., but give the --check option to autopkg. This will cause it to re-cache the file and report the download, but not proceed past the EndOfCheckPhase processor.

Note: The MunkiImporter processor also supports an input variable called force_munkiimport, that forces an import if it's set (to anything). This might be useful in testing modifications to recipes. This can be set temporarily using the -k/--key option, as in the example below. For info on this override method here.

autopkg run -k force_munkiimport=yes Recipe

Running multiple recipes

AutoPkg can run more than one recipe in a given session.

One of the recipes available in the http://github.com/autopkg/recipes.git recipe repo is "MakeCatalogs.munki". It does what you'd expect, but it's behavior is more subtle than just running makecatalogs. By default, it checks the results of any recipes that were run in the same session and only does a makecatalogs if something new was imported into the Munki repo. So running this recipe by itself is generally not very useful:

% autopkg run -v MakeCatalogs.munki
Processing MakeCatalogs.munki...
MakeCatalogsProcessor
MakeCatalogsProcessor: No need to rebuild catalogs.
Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.makecatalogs/receipts/MunkiMakeCatalogs-receipt-20130821-163856.plist

Nothing downloaded, packaged or imported.

But if you add this recipe to the end of a run of other recipes that do import something, it does the right thing:

% autopkg run -v Firefox.munki Thunderbird.munki MakeCatalogs.munki
Processing Firefox.munki...
MozillaURLProvider
MozillaURLProvider: Found URL http://download-origin.cdn.mozilla.net/pub/mozilla.org//firefox/releases/latest/mac/en-US/Firefox%2023.0.1.dmg
URLDownloader
URLDownloader: Storing new Last-Modified header: Wed, 14 Aug 2013 15:36:04 GMT
URLDownloader: Storing new ETag header: "c47c4-2a4e1aa-4e3ea1d85c100"
URLDownloader: Downloaded /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.firefox-rc-en_US/downloads/Firefox.dmg
EndOfCheckPhase
MunkiImporter
MunkiImporter: Copied pkginfo to /Users/Shared/munki_repo/pkgsinfo/apps/firefox/Firefox-23.0.1.plist
MunkiImporter: Copied pkg to /Users/Shared/munki_repo/pkgs/apps/firefox/Firefox-23.0.1.dmg
Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.firefox-rc-en_US/receipts/Firefox-receipt-20130821-164549.plist
Processing Thunderbird.munki...
MozillaURLProvider
MozillaURLProvider: Found URL http://download-origin.cdn.mozilla.net/pub/mozilla.org//thunderbird/releases/latest/mac/en-US/Thunderbird%2017.0.8.dmg
URLDownloader
URLDownloader: Storing new Last-Modified header: Thu, 01 Aug 2013 22:56:24 GMT
URLDownloader: Storing new ETag header: "f68c13-23a1d47-4e2eac0571600"
URLDownloader: Downloaded /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.thunderbird/downloads/Thunderbird.dmg
EndOfCheckPhase
MunkiImporter
MunkiImporter: Copied pkginfo to /Users/Shared/munki_repo/pkgsinfo/apps/thunderbird/Thunderbird-17.0.8.plist
MunkiImporter: Copied pkg to /Users/Shared/munki_repo/pkgs/apps/thunderbird/Thunderbird-17.0.8.dmg
Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.thunderbird/receipts/Thunderbird-receipt-20130821-164556.plist
Processing MakeCatalogs.munki...
MakeCatalogsProcessor
MakeCatalogsProcessor: Munki catalogs rebuilt!
Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.makecatalogs/receipts/MakeCatalogs-receipt-20130821-164556.plist

The following new items were downloaded:
    /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.firefox-rc-en_US/downloads/Firefox.dmg
    /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.thunderbird/downloads/Thunderbird.dmg

The following new items were imported:
    Name                     Version          Catalogs                         Pkginfo Path
    ----                     -------          --------                         ------------
    Firefox                  23.0.1           testing                          apps/firefox/Firefox-23.0.1.plist
    Thunderbird              17.0.8           testing                          apps/thunderbird/Thunderbird-17.0.8.plist

You can see it downloaded and imported Firefox, then Thunderbird, then rebuilt the Munki catalogs. If we run these recipes again, we see it skips the tasks that are not needed:

% autopkg run -v Firefox.munki Thunderbird.munki MakeCatalogs.munki
Processing Firefox.munki...
MozillaURLProvider
MozillaURLProvider: Found URL http://download-origin.cdn.mozilla.net/pub/mozilla.org//firefox/releases/latest/mac/en-US/Firefox%2023.0.1.dmg
URLDownloader
URLDownloader: Item at URL is unchanged.
URLDownloader: Using existing /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.firefox-rc-en_US/downloads/Firefox.dmg
EndOfCheckPhase
MunkiImporter
MunkiImporter: Item Firefox.dmg already exists in the munki repo as pkgs/apps/firefox/Firefox-23.0.1.dmg.
Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.firefox-rc-en_US/receipts/Firefox-receipt-20130821-164921.plist
Processing Thunderbird.munki...
MozillaURLProvider
MozillaURLProvider: Found URL http://download-origin.cdn.mozilla.net/pub/mozilla.org//thunderbird/releases/latest/mac/en-US/Thunderbird%2017.0.8.dmg
URLDownloader
URLDownloader: Item at URL is unchanged.
URLDownloader: Using existing /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.thunderbird/downloads/Thunderbird.dmg
EndOfCheckPhase
MunkiImporter
MunkiImporter: Item Thunderbird.dmg already exists in the munki repo as pkgs/apps/thunderbird/Thunderbird-17.0.8.dmg.
Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.thunderbird/receipts/Thunderbird-receipt-20130821-164923.plist
Processing MakeCatalogs.munki...
MakeCatalogsProcessor
MakeCatalogsProcessor: No need to rebuild catalogs.
Receipt written to /Users/gneagle/Library/AutoPkg/Cache/com.github.autopkg.munki.makecatalogs/receipts/MakeCatalogs-receipt-20130821-164923.plist

Nothing downloaded, packaged or imported.

This behavior keeps autopkg from re-importing the same items over and over into the Munki repo and from running makecatalogs unnecessarily.

What's next?

Recipes include defaults for their input variables, but it's generally assumed that any recipe used in production will need to have some things overridden. For example, no two Munki repos follow exactly the same directory structure, so Munki recipes will likely want to at least override the destination subdirectory for the pkg/pkginfo files. Visit the Recipe Overrides page to learn more.

Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.