Import contacts with Feed Me

[martinhellwagner]

I was wondering if it's possible to run a Feed Me feed against an API GET endpoint that imports contacts into a Craft Campaign mailing list. Currently, the only way seems to import these contacts as users, and then run an import for these users into a Craft Campaign mailing list – which seems to be quite a detour since I don't need these Mailing List Contacts as users.

[bencroker]

The plugin currently allows importing contacts via CSV file or user sync only, but I’ll look into adding compatibility with Feed Me.

[martinhellwagner]

@bencroker

Thanks a lot! Is there a timeframe for this enhancement? We'd need the import by the middle of June. If that's too soon for this feature, I'll look for a workaround via a CSV file. Speaking of: is there a CLI command we could use for triggering an import with a Cron Job or so?

[bencroker]

Thanks a lot! Is there a timeframe for this enhancement?

No, but I’ll aim to get it in by next week.

[martinhellwagner]

@bencroker

Wow, thanks – much appreciated! 🙏🏼

[bencroker]

Added in eb5b1aa. You can help test this by running composer require putyourlightson/craft-campaign:dev-develop as 2.8.0.

[martinhellwagner]

@bencroker

That was fast! 😍 Works like a charm – thanks a lot!

[bencroker]

Released in version 2.8.0.

[martinhellwagner]

@bencroker

I successfully implemented the request that fetches the data from an API. The data is sent to the Feed Me feed like this:

        Event::on(DataTypes::class, DataTypes::EVENT_BEFORE_FETCH_FEED, function(FeedDataEvent $event) {
            ...
            $event->response = [
                'success' => true,
                'data' => $this->fetchData($tokenType, $accessToken),
            ];
        });

However, even though valid JSON is returned from the fetchData method, the following error appears on our Stage (even though it is working properly on my local machine):

Could that be a problem with the Campaign plugin, or should I talk to the Craft / Feed Me team about that?

[bencroker]

I’m not sure but can investigate. Did you repeat the test to see if the error occurs multiple times?

[martinhellwagner]

I repeated the test multiple times, and it always fails with the same error on Stage. I never get the error on my local system though. I've also tried the cURL request that fetches the data on my local environment and on Stage – it works on both systems. I'm guessing it might be a problem with feeding the response's JSON data into the $event->response array.

[bencroker]

Hmm, can you resave the feed setup details again? I’m really not sure where the issue might be.

[martinhellwagner]

I've also tried to resave the feed multiple times – unfortunately, this doesn't change anything. I've also already e-mailed the Craft support with this problem – maybe they know more about this. 🤔

[martinhellwagner]

@bencroker

The Craft support suggested that Section isn't defined in the feed settings – and sent me this screenshot:

Is that something that might help with a solution?

[bencroker]

Um, section?? Did you select a mailing list to import contacts into?

[martinhellwagner]

Yes, all the necessary dropdowns are set:

(The Feed URL is not set to a URL because the actual call to the API is done with Guzzle in my code. However, the error doesn't change if I insert the correct endpoint URL.)

[bencroker]

In that case, the Craft support team’s suggestion is not particularly helpful.

[martinhellwagner]

What I'm surprised is that it works locally, but not on Stage. I have the same code deployed, and the same DB for both environments. Both environments run PHP 8.1.

[olivierbon]

Right, I got to the bottom of this.

This only happens when 'runQueueAutomatically' => false, and the queue is offset to a separate worker.

The reason is this.

The plugin will only register new elements on CP request.

I'll leave the awesome @bencroker do his thing now :)

[bencroker]

Great find, @olivierbon, and thank you so much for reporting it!
Fixed in 5b55002 and released in version 2.8.1.

[martinhellwagner]

Great news! Really appreciate your work on this, @olivierbon & @bencroker! 🙏🏼

Will try it out first thing next week.

[martinhellwagner]

Just tested it – looks good. Thanks again! 🚀

[martinhellwagner]

@bencroker

We found one issue where I'm not exactly sure if this has to do with the implementation of the feature. I have two feeds that each fetch a set of contacts from an API endpoint. One contact is found in both of these API responses.

My understanding would be that this contact would be created and subscribed to the first mailing list when running the first feed, and then subscribed to the second mailing list when running the second feed (I have both feeds set to "Add & Update"). The contact indeed gets created, but it only ever gets subscribed to the mailing list whose corresponding feed I'm running first. Am I doing something wrong, or is this the way it's currently implemented?

Thanks for your answer! 🙏🏼

[bencroker]

That process should work as you expect it to. Does the issue only happen with automated API fetches, or can you reproduce it manually? If you can, then please outline the steps involved so I can test locally.

[martinhellwagner]

If I extract the JSON data in question from the feed and add it manually, then the same thing happens. This is the data I'm using – the one contact that should go into both mailing lists (the businessMail is the unique identifier):

{
	"content": [{
		"id": 5357,
		"createdDate": 1685450968784,
		"modifiedDate": 1685524744466,
		"firstName": "Maria",
		"lastName": "Musterfrau",
		"fullName": "Musterfrau Maria",
		"reverseFullName": "Maria Musterfrau",
		"mail": null,
		"phone": null,
		"mobile": null,
		"address": null,
		"localDescription": null,
		"businessMail": "mamu@text.com",
		"businessPhone": null,
		"businessMobile": null
	}]
}

These are the feeds:

[bencroker]

The compareContent config setting determines whether to check against existing element content before updating. If the content of the user has not changed then saving is skipped, so setting it to false (the default is true) should have the desired effect.
https://docs.craftcms.com/feed-me/v4/get-started/configuration.html

[martinhellwagner]

@bencroker

Yep, that was it. Thanks for pointing me in the right direction! 🚀

[maximilian-schwarz]

I have the same error message as @martinhellwagner

This only happens when 'runQueueAutomatically' => false, and the queue is offset to a separate worker.

I think you need this event to register new elements for Feed me:
https://github.com/craftcms/feed-me/blob/c30fb38b4ab334cd1081727750c65cb42d900a4d/src/services/Elements.php#L32

Craft Version: 4.5.5
Feed me: 5.2.0
Campaign: 2.8.6

[bencroker]

This issue has already been resolved, @maximilian-schwarz, and the event to register elements in Feed Me is already being used.

craft-campaign/src/Campaign.php

Line 711 in 05ad834

Event::on(FeedMeElements::class, FeedMeElements::EVENT_REGISTER_FEED_ME_ELEMENTS,

If this is still an issue for you then please update to the latest version of the plugin and create a new issue with further details and I’ll be happy to look into it

[maximilian-schwarz]

@bencroker thanks for the fast reply. I tested it with the current version and the issue still exist.

I have created a new issue with a working solution ;)