Unable to import manifest from URL/paste

[drbauder]

From what I can tell, I've correctly installed the IIIF Toolkit plugin, but when I try to submit a manifest either via URL or by pasting the JSON, nothing happens. It doesn't seem to matter what options I choose on the import page. When I click Import, I am taken to the status page, but no new items appear in the Status Panel and nothing appears to import.

Omeka 2.4.1, using this for the import via URL or paste: http://dms-data.stanford.edu/data/manifests/kn/mw497gz1295/manifest.json

[dicksonlaw583]

Please make sure that your background job runner is configured correctly. Go into application/config/config.ini in your Omeka installation and make sure that background.php.path is pointed to the correct PHP CLI binary. You can check your setup by performing a batch edit over several items.

As a temporary alternative to importing, if you do not need to annotate or rearrange canvases, you can use remote embedding blocks for Exhibits Builder to present your manifest.

In addition, I recommend updating the manifest because the image server has already upgraded to Image API 2.0, but the manifest still reads 1.1. Image API 1.1 pulls images from .../native.jpg, while 2.0 pulls images from .../default.jpg, so the mismatch would cause images to fail to load in the importer.

[drbauder]

Setting background.php.path has no effect either way. With and without it set, I can batch edit items (e.g., under Admin > Items, select multiple items and add a tag to all of them).

Oddly enough, we have another instance of Omeka running on the same server, but version 2.4.0. Importing the same manifest from the URL above works fine. I can't see any real difference in the application config files or the site configurations.

[dicksonlaw583]

Do you have any error logs captured from the instance that doesn't run the job properly?

[drbauder]

Sorry for the delay getting back to you! I enabled debugging and error logging, as described at https://omeka.org/codex/Retrieving_Error_Messages.

application/logs/errors.log logs nothing at all while performing an import. The file should be writeable by everybody.

In my apache error log, I see this:
[Tue Aug 08 08:32:06.307874 2017] [:error] [pid 1142] [client xx.xx.xx.xx:59351] PHP Warning: urlencode() expects parameter 1 to be string, array given in /var/www/omeka241/application/libraries/Zend/Controller/Router/Route/Module.php on line 279, referer: https://omeka.ucalgary.ca/admin/iiif-items/import

But that error appears even in the working 2.4.0 instance.

[dicksonlaw583]

@drbauder

Please retry with the branch referenced on #6. This targets just the message cited in your error log, but I'm hoping that this is what's holding up the imports on your non-working instance.

[drbauder]

Hi @dicksonlaw583,

That fixes the error, but the behaviour of the import remains the same. Still no updated status.

I'm not sure if this will help, but when I query the omeka_processes table, I can see that an entry is created:

id	class	user_id	pid	status	args	started	stopped
59	Omeka_Job_Process_Wrapper	33	NULL	starting	a:1:{s:3:"job";s:349:"{"className":"IiifItems_Job_Import","options":{"isPublic":0,"isFeatured":0,"importType":"Manifest","importSource":"Url","importSourceBody":"http://dms-data.stanford.edu/data/manifests/kn/mw497gz1295/manifest.json","importPreviewSize":96,"importAnnoSize":null,"isReversed":0,"parent":""},"createdAt":"2017-08-08T11:54:49-06:00","createdBy":33}";}	2017-08-08 11:54:49	NULL

[dicksonlaw583]

Can you turn on debug mode on your non-working Omeka instance, start an import job and tell me what is the last debug line you see in /var/www/omeka241/application/logs/errors.log?

To turn on debug mode:

• In .htaccess, uncomment the line SetEnv APPLICATION_ENV development
• In application/config/config.ini, change the value log.priority from Zend_Log::WARN to Zend_Log::DEBUG.

[drbauder]

Hmm. I made those changes. Nothing is logged at all when I try an import.

[dicksonlaw583]

I'm sorry, there's one more entry to change for debug mode:

• In application/config/config.ini, change the value log.errors to true.

I have created a new installation of 2.4.1 in my development environment, but so far I have been unable to replicate your situation. The process completed, although the items counted as failed because the image server has moved onto 2.0 while the manifest still reads 1.1.

In addition, can you confirm whether there is a table named omeka_iiif_items_job_statuses in your non-working installation? Is the process still starting? Is it a new Omeka instance or an existing Omeka instance?

[drbauder]

I actually had log.errors = true set from my earlier attempts at debugging, and I just confirmed that it is still set. Still no errors in the log file.

I do not seem to have the omeka_iiif_items_job_statuses table. These are the tables I do have:

omeka_collections
omeka_csv_import_imported_items
omeka_csv_import_imports
omeka_csv_import_plus_imported_records
omeka_csv_import_plus_imports
omeka_csv_import_plus_logs
omeka_element_sets
omeka_element_texts
omeka_element_types
omeka_elements
omeka_exhibit_block_attachments
omeka_exhibit_page_blocks
omeka_exhibit_pages
omeka_exhibits
omeka_files
omeka_getty_suggests
omeka_history_log_changes
omeka_history_log_entries
omeka_iiif_items_cached_json_data
omeka_iiif_items_job_statuses
omeka_item_order_item_orders
omeka_item_types
omeka_item_types_elements
omeka_items
omeka_keys
omeka_lc_suggests
omeka_oai_pmh_repository_tokens
omeka_oaipmh_harvester_harvests
omeka_oaipmh_harvester_records
omeka_options
omeka_plugins
omeka_processes
omeka_records_tags
omeka_schema_migrations
omeka_search_texts
omeka_sessions
omeka_simple_pages_pages
omeka_svp_assigns
omeka_svp_terms
omeka_svp_vocabs
omeka_tags
omeka_users
omeka_users_activations

I'm not certain the process is starting. I tried using top to keep an eye on processes, but only saw a brief spike in apache and MySQL, no php processes spinning up. But I'm not sure that's what I should be looking for.

This is an existing Omeka instance.

[dicksonlaw583]

You actually do have a omeka_iiif_items_job_statuses table.

Since there are other plugins on your installation that may be running jobs, please look in your omeka_processes table for other long-running jobs that may still be in progress or attempting to start. If there are too many of them, Omeka would simply refuse to start any new ones, including the one that IIIF Toolkit's importer works with. You would have to interrupt the ones that are stuck and remove their entries from omeka_processes.

[drbauder]

Ha, oops, missed that! I will take a look at the processes table and let you know.

[drbauder]

All of the jobs listed in the omeka_processes table have the status starting. I took a backup of the table and tried deleting all rows, but that didn't solve the issue. Initiating an import adds another row to the table, but it never progresses beyond starting. Here's a sample of the first few rows in the table:

id	class	user_id	pid	status	left(args, 150)	started	stopped
1	Omeka_Job_Process_Wrapper	6	NULL	starting	a:1:{s:3:"job";s:168:"{"className":"CsvImport_ImportTask","options":{"importId":1,"memoryLimit":null,"batchSize":null,"method":"start"},"createdAt":"2	2015-03-31 09:34:17	0000-00-00 00:00:00
2	Omeka_Job_Process_Wrapper	6	NULL	starting	a:1:{s:3:"job";s:168:"{"className":"CsvImport_ImportTask","options":{"importId":2,"memoryLimit":null,"batchSize":null,"method":"start"},"createdAt":"2	2015-03-31 16:04:02	0000-00-00 00:00:00
3	Omeka_Job_Process_Wrapper	6	NULL	starting	a:1:{s:3:"job";s:167:"{"className":"CsvImport_ImportTask","options":{"importId":1,"memoryLimit":null,"batchSize":null,"method":"undo"},"createdAt":"20	2015-03-31 16:07:23	0000-00-00 00:00:00
4	Omeka_Job_Process_Wrapper	33	NULL	starting	a:1:{s:3:"job";s:105:"{"className":"IiifItems_Job_AddUuid","options":[],"createdAt":"2017-07-25T10:40:20-06:00","createdBy":33}";}	2017-07-25 10:40:20	NULL
5	Omeka_Job_Process_Wrapper	29	NULL	starting	a:1:{s:3:"job";s:317:"{"className":"IiifItems_Job_Import","options":{"isPublic":0,"isFeatured":0,"importType":"Manifest","importSource":"Url","importS	2017-07-25 16:36:37	NULL

[dicksonlaw583]

It appears that something is stuck or misconfigured on the non-working instance's long-running job dispatcher. Try looking in application/config/config.ini for entries that govern the dispatcher. Mine looks like this and it works:

jobs.dispatcher.default = "Omeka_Job_Dispatcher_Adapter_Synchronous"
jobs.dispatcher.longRunning = "Omeka_Job_Dispatcher_Adapter_BackgroundProcess"

If you have direct access to the server, you can also try rebooting the server to try and get the stuck jobs out of your way. This is recommended after you change the dispatcher configurations or manually remove rows from the processes table.

[drbauder]

I have the same entries in config.ini. Rebooting the server had no effect, unfortunately.

[dicksonlaw583]

Can you start an import job with your CSV Import plugin? If you can, please reply with the version number of the CSV Import plugin you're using. If not, it is an issue specific to that Omeka instance instead of any one plugin. You can ask on forum.omeka.org for hints as to what else can hold up long-running jobs.

[drbauder]

Hi @dicksonlaw583,

I've been troubleshooting our main Omeka installation, and there does seem to be a problem with running jobs. It seems that under a new and fully-updated instance of Omeka, IiifItems works just fine, as do other long-running jobs, so we'll be migrating and updating our production installation soon.

Thanks again for all your help!

[dicksonlaw583]

@drbauder

You are welcome. I will be closing this ticket since it is not an issue specific to the plugin, but feel free to open another ticket if a new glitch happens on your migrated installation.