Manipulators
Manipulators are MIK plugins that let you perform tasks at specific times in the MIK execution lifecycle, or to change how fetchers, file getters, and metadata parsers work. All the code for a manipulator is encapsulated in a single PHP class file. Manipulators are registered in the MIK configuration file in the [MANIPULATORS] section, and may take parameters. The signatures for manipulators identify the group they are in, followed by an equal sign, followed by the manipulator's parameters, which are delimited by the pipe symbol (|). The first parameter is always the name of the manipulator. For example, in the following example, the "NormalizeDate" manipulator is being registered, taking the parameters "Date", "dateIssued", and "m":
[MANIPULATORS]
metadatamanipulators[] = "NormalizeDate|Date|dateIssued|m"
Each manipulator has its own wiki page, which explains its function and parameters. Below is an overview of the most popular manipulators.
Fetcher manipulators filter the records that you want in a given conversion job. For example, the CdmCompound fetcher manipulator lets you specify whether the MIK job is applied to Document, Document-PDF, Document-EAD, Postcard, Picture Cube, or Monograph compound objects from CONTENTdm. The RandomSet fetcher manipulator will generate a random set of objects to create ingest packages for.
Fetcher manipulators are cumulative. In other words, if your toolchain uses more than one fetcher manipulator, the output of the first one registered in your .ini file is the input of the next one.
In addition to the two fetcher manipulators mentioned above, MIK provides some others:
- Specific Set
- Random Set
- Range Set
- CdmCompound
- CdmNoParent
- CdmSingleFileByExtension
- CsvSingleFileByExtension
The Specific Set fetcher manipulator lets you define a list of objects to process, which is very useful during testing, or you want to reprocess specific objects for some reason. To assist in the latter case, MIK provides a script that will generate a list of source IDs from the log file generated during a job, so you can reprocess the problematic objects easily.
Metadata manipulators are components of a toolchain that take a snippet from the mappings file and modify it before adding it to the MODS or DC XML file. You register metadata manipulators in the toolchain's .ini file (as illustrated above). One metadata manipulator you will use often is the SplitRepeatedValues manipulator, which splits repeated subjects terms and other values in your source metadata into separate MODS elements. Another common one is FilterDate, which lets you modify the format that dates are in.
An important design goal of metadata manipulators is that they let you clean up your source metadata before it gets imported into Islandora. For example, if you had a set of dates that were in US-style mm/dd/yy format and you wanted to convert them into yyyy-mm-dd format, you could do that with a metadata manipulator.
Documentation for the following metadata manipulators is available:
- Add CONTENTdm Data
- Add CSV Data
- Add UUID to MODS
- Split Repeated Values
- Normalize Date
- Simple Replace
- InsertXmlFromTemplate
Note that the data altered by one metadata manipulator can itself be manipulated by other manipulators. Manipulators run in the order they appear in the .ini file. For example, in the following configuration, any occurrences of 'colour' will end up being replaced with 'hue', since the output of the first one becomes the input of the second.
[MANIPULATORS]
metadatamanipulators[] = "SimpleReplace|/colour/|color"
metadatamanipulators[] = "SimpleReplace|/color/|hue"
This example uses two instances of the SimpleReplace metadata manipulator, but this behavior applies across all metadata manipulators.
File getter manipulators let you modify or refine information about where MIK can find certain types of files. For example, in a CONTENTdm toolchain, you might want to tell MIK to get the master TIFF files for a collection of images in a specific directory on a file server. The file getter manipulators currently available include:
File manipulators work similar to metadata manipulators except they modify (or create) files that are then imported into Islandora. Another use for them is to validate files; for example, MIK provides a file manipulator that retrieves an object's thumbnail from CONTENTdm.
Please note that file manipulators may be deprecated for the 1.0 release of MIK in favour of post-write hooks. If you need to retrieve or modify a source file, or create an additional datastream file for loading into Islandora, we recommend that you implement a post-write hook instead of a file manipulator. In any event please open an issue so we can discuss your use case or leave a comment on the deprecation issue.
Content on the Move to Islandora Kit wiki is licensed under a Creative Commons Attribution 4.0 International License.
