Usage@Ingest routines

• Usage: admininistration of Ingest routines

Ingest routines are the second way to ingest SIPs next to the direct ingestion of SIP (the other way is Single ingest).

Attribute	Description
Name	name of the ingest routine
Producer profile	producer profile to be used in the ingest process
Tranfsfer area path	file path to a folder which the ingest routine will be scanning for an occurrence of new data to be ingested, this path is relative to the path of the transfer area of the producer
CRON expression	CRON expression specifying when the batch ingestion should be triggered, expression must consist of six fields: https://riptutorial.com/spring/example/21209/cron-expression
Workflow configuration	JSON configuration of the ingest workflow
Active	boolean value, if it is set to true, the ingest routine is activated, otherwise deactivated
Auto	Flag indicating whether the ingests are driven only by CRON (blocking) or by CRON and prefixes (non-blocking). Flag is editable only during routine creation. See more information bellow.

NOTE: ARCLib computes the resulting ingest workflow config by merging the producer ingest workflow config and the config provided with the ingest routine.

Action	Description
Create	From the dashboard, user creates a new ingest routine.
Edit	From the dashboard, user clicks on one ingest routine from list. On the ingest routine detail page the ingest routine can be edited. User clicks on the Save and close button to save the changes.
Delete	From the dashboard, user clicks on the Delete button next to the given ingest routine.

Auto flag

If flag is set to true:

• Before transferring to transfer area, Producer has to prefix every SIP ZIP (only the .zip part, not the .sums part) with prefix TRANSFERRING_.
• After the transfer is finished, Producer must rename remove the TRANSFERRING_ prefix.
• ARCLib periodically (according to specified CRON) scans transfer area for SIPs without any prefix, set prefix of those to PROCESSING_ and starts new ingest batch.
• In case of failure, if the SIP is still present at the transfer area, the prefix in transfer area is set to FAILED_.
• In case of success, if the SIP is still present at the transfer area, the prefix in transfer area is set to ARCHIVED_ or the SIP is deleted from transfer area if arclib.deleteSipFromTransferArea = true.
• Auto routines are non-blocking i.e. ingest batches created by the same routine can run in parallel.

If flag is set to false:

• Neither producer nor ARCLib prefixes SIPs in the transfer area.
• ARCLib periodically (according to specified CRON) scans transfer area for all SIPs.
• In case of failure, the SIP is left untouched in transfer area.
• In case of success, the SIP is left untouched in transfer area or the SIP is deleted from transfer area if arclib.deleteSipFromTransferArea = true.
• Routines with auto = false are blocking i.e. new ingest batch is not created if any other ingest batch created by the same routine is still processing.